* [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping
@ 2016-07-07 8:41 Chris Wilson
2016-07-07 8:41 ` [PATCH 02/64] drm/i915: Preserve current RPS frequency across init Chris Wilson
` (63 more replies)
0 siblings, 64 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Never go to sleep waiting on the GPU without first ensuring that we will
get woken up.
We have a choice of queuing the hangcheck before every schedule() or the
first time we wakeup. In order to simply accommodate both the signaler
and the ordinary waiter, move the queuing to the common point of
enabling the irq. We lose the paranoid safety of ensuring that the
hangcheck is active before the sleep, but avoid code duplication (and
redundant hangcheck queuing).
Testcase: igt/prime_busy
Fixes: c81d46138da6 ("drm/i915: Convert trace-irq to the breadcrumb waiter")
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
---
drivers/gpu/drm/i915/i915_gem.c | 9 ---------
drivers/gpu/drm/i915/intel_breadcrumbs.c | 9 +++++++++
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 8f50919ba9b4..7fd44980798f 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1501,15 +1501,6 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
break;
}
- /* Ensure that even if the GPU hangs, we get woken up.
- *
- * However, note that if no one is waiting, we never notice
- * a gpu hang. Eventually, we will have to wait for a resource
- * held by the GPU and so trigger a hangcheck. In the most
- * pathological case, this will be upon memory starvation!
- */
- i915_queue_hangcheck(req->i915);
-
timeout_remain = io_schedule_timeout(timeout_remain);
if (timeout_remain == 0) {
ret = -ETIME;
diff --git a/drivers/gpu/drm/i915/intel_breadcrumbs.c b/drivers/gpu/drm/i915/intel_breadcrumbs.c
index d89b2c963618..b074f3d6d127 100644
--- a/drivers/gpu/drm/i915/intel_breadcrumbs.c
+++ b/drivers/gpu/drm/i915/intel_breadcrumbs.c
@@ -93,6 +93,15 @@ static void __intel_breadcrumbs_enable_irq(struct intel_breadcrumbs *b)
if (!b->irq_enabled ||
test_bit(engine->id, &i915->gpu_error.missed_irq_rings))
mod_timer(&b->fake_irq, jiffies + 1);
+
+ /* Ensure that even if the GPU hangs, we get woken up.
+ *
+ * However, note that if no one is waiting, we never notice
+ * a gpu hang. Eventually, we will have to wait for a resource
+ * held by the GPU and so trigger a hangcheck. In the most
+ * pathological case, this will be upon memory starvation!
+ */
+ i915_queue_hangcheck(i915);
}
static void __intel_breadcrumbs_disable_irq(struct intel_breadcrumbs *b)
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 02/64] drm/i915: Preserve current RPS frequency across init
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 03/64] drm/i915: Remove superfluous powersave work flushing Chris Wilson
` (62 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx; +Cc: Mika Kuoppala
Select idle frequency during initialisation, then reset the last known
frequency when re-enabling. This allows us to preserve the user selected
frequency across resets.
v2: Stop CHV from overriding the user's choice in cherryview_enable_rps()
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
---
drivers/gpu/drm/i915/intel_pm.c | 48 +++++++++++++++++++----------------------
1 file changed, 22 insertions(+), 26 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index 5a8ee0c76593..49b5bc86e7c8 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -5149,6 +5149,7 @@ static void gen6_init_rps_frequencies(struct drm_i915_private *dev_priv)
}
dev_priv->rps.idle_freq = dev_priv->rps.min_freq;
+ dev_priv->rps.cur_freq = dev_priv->rps.idle_freq;
/* Preserve min/max settings in case of re-init */
if (dev_priv->rps.max_freq_softlimit == 0)
@@ -5165,6 +5166,20 @@ static void gen6_init_rps_frequencies(struct drm_i915_private *dev_priv)
}
}
+static void reset_rps(struct drm_i915_private *dev_priv,
+ void (*set)(struct drm_i915_private *, u8))
+{
+ u8 freq;
+
+ freq = dev_priv->rps.cur_freq;
+
+ /* force a reset */
+ dev_priv->rps.power = -1;
+ dev_priv->rps.cur_freq = -1;
+
+ set(dev_priv, freq);
+}
+
/* See the Gen9_GT_PM_Programming_Guide doc for the below */
static void gen9_enable_rps(struct drm_i915_private *dev_priv)
{
@@ -5201,8 +5216,7 @@ static void gen9_enable_rps(struct drm_i915_private *dev_priv)
/* Leaning on the below call to gen6_set_rps to program/setup the
* Up/Down EI & threshold registers, as well as the RP_CONTROL,
* RP_INTERRUPT_LIMITS & RPNSWREQ registers */
- dev_priv->rps.power = HIGH_POWER; /* force a reset */
- gen6_set_rps(dev_priv, dev_priv->rps.idle_freq);
+ reset_rps(dev_priv, gen6_set_rps);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}
@@ -5348,8 +5362,7 @@ static void gen8_enable_rps(struct drm_i915_private *dev_priv)
/* 6: Ring frequency + overclocking (our driver does this later */
- dev_priv->rps.power = HIGH_POWER; /* force a reset */
- gen6_set_rps(dev_priv, dev_priv->rps.idle_freq);
+ reset_rps(dev_priv, gen6_set_rps);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}
@@ -5442,8 +5455,7 @@ static void gen6_enable_rps(struct drm_i915_private *dev_priv)
dev_priv->rps.max_freq = pcu_mbox & 0xff;
}
- dev_priv->rps.power = HIGH_POWER; /* force a reset */
- gen6_set_rps(dev_priv, dev_priv->rps.idle_freq);
+ reset_rps(dev_priv, gen6_set_rps);
rc6vids = 0;
ret = sandybridge_pcode_read(dev_priv, GEN6_PCODE_READ_RC6VIDS, &rc6vids);
@@ -5807,6 +5819,7 @@ static void valleyview_init_gt_powersave(struct drm_i915_private *dev_priv)
dev_priv->rps.min_freq);
dev_priv->rps.idle_freq = dev_priv->rps.min_freq;
+ dev_priv->rps.cur_freq = dev_priv->rps.idle_freq;
/* Preserve min/max settings in case of re-init */
if (dev_priv->rps.max_freq_softlimit == 0)
@@ -5871,6 +5884,7 @@ static void cherryview_init_gt_powersave(struct drm_i915_private *dev_priv)
"Odd GPU freq values\n");
dev_priv->rps.idle_freq = dev_priv->rps.min_freq;
+ dev_priv->rps.cur_freq = dev_priv->rps.idle_freq;
/* Preserve min/max settings in case of re-init */
if (dev_priv->rps.max_freq_softlimit == 0)
@@ -5970,16 +5984,7 @@ static void cherryview_enable_rps(struct drm_i915_private *dev_priv)
DRM_DEBUG_DRIVER("GPLL enabled? %s\n", yesno(val & GPLLENABLE));
DRM_DEBUG_DRIVER("GPU status: 0x%08x\n", val);
- dev_priv->rps.cur_freq = (val >> 8) & 0xff;
- DRM_DEBUG_DRIVER("current GPU freq: %d MHz (%u)\n",
- intel_gpu_freq(dev_priv, dev_priv->rps.cur_freq),
- dev_priv->rps.cur_freq);
-
- DRM_DEBUG_DRIVER("setting GPU freq to %d MHz (%u)\n",
- intel_gpu_freq(dev_priv, dev_priv->rps.idle_freq),
- dev_priv->rps.idle_freq);
-
- valleyview_set_rps(dev_priv, dev_priv->rps.idle_freq);
+ reset_rps(dev_priv, valleyview_set_rps);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}
@@ -6059,16 +6064,7 @@ static void valleyview_enable_rps(struct drm_i915_private *dev_priv)
DRM_DEBUG_DRIVER("GPLL enabled? %s\n", yesno(val & GPLLENABLE));
DRM_DEBUG_DRIVER("GPU status: 0x%08x\n", val);
- dev_priv->rps.cur_freq = (val >> 8) & 0xff;
- DRM_DEBUG_DRIVER("current GPU freq: %d MHz (%u)\n",
- intel_gpu_freq(dev_priv, dev_priv->rps.cur_freq),
- dev_priv->rps.cur_freq);
-
- DRM_DEBUG_DRIVER("setting GPU freq to %d MHz (%u)\n",
- intel_gpu_freq(dev_priv, dev_priv->rps.idle_freq),
- dev_priv->rps.idle_freq);
-
- valleyview_set_rps(dev_priv, dev_priv->rps.idle_freq);
+ reset_rps(dev_priv, valleyview_set_rps);
intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 03/64] drm/i915: Remove superfluous powersave work flushing
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
2016-07-07 8:41 ` [PATCH 02/64] drm/i915: Preserve current RPS frequency across init Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 04/64] drm/i915: Defer enabling rc6 til after we submit the first batch/context Chris Wilson
` (61 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Instead of flushing the outstanding enabling, remember the requested
frequency to apply when the powersave work runs.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
drivers/gpu/drm/i915/i915_debugfs.c | 30 ++---------------------
drivers/gpu/drm/i915/i915_sysfs.c | 48 +++++++++----------------------------
2 files changed, 13 insertions(+), 65 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 3d05cae0e92d..e686704a310a 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -1205,8 +1205,6 @@ static int i915_frequency_info(struct seq_file *m, void *unused)
intel_runtime_pm_get(dev_priv);
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
if (IS_GEN5(dev)) {
u16 rgvswctl = I915_READ16(MEMSWCTL);
u16 rgvstat = I915_READ16(MEMSTAT_ILK);
@@ -1896,8 +1894,6 @@ static int i915_ring_freq_table(struct seq_file *m, void *unused)
intel_runtime_pm_get(dev_priv);
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
ret = mutex_lock_interruptible(&dev_priv->rps.hw_lock);
if (ret)
goto out;
@@ -4950,20 +4946,11 @@ i915_max_freq_get(void *data, u64 *val)
{
struct drm_device *dev = data;
struct drm_i915_private *dev_priv = to_i915(dev);
- int ret;
if (INTEL_INFO(dev)->gen < 6)
return -ENODEV;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
- ret = mutex_lock_interruptible(&dev_priv->rps.hw_lock);
- if (ret)
- return ret;
-
*val = intel_gpu_freq(dev_priv, dev_priv->rps.max_freq_softlimit);
- mutex_unlock(&dev_priv->rps.hw_lock);
-
return 0;
}
@@ -4978,8 +4965,6 @@ i915_max_freq_set(void *data, u64 val)
if (INTEL_INFO(dev)->gen < 6)
return -ENODEV;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
DRM_DEBUG_DRIVER("Manually setting max freq to %llu\n", val);
ret = mutex_lock_interruptible(&dev_priv->rps.hw_lock);
@@ -5017,20 +5002,11 @@ i915_min_freq_get(void *data, u64 *val)
{
struct drm_device *dev = data;
struct drm_i915_private *dev_priv = to_i915(dev);
- int ret;
- if (INTEL_INFO(dev)->gen < 6)
+ if (INTEL_GEN(dev_priv) < 6)
return -ENODEV;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
- ret = mutex_lock_interruptible(&dev_priv->rps.hw_lock);
- if (ret)
- return ret;
-
*val = intel_gpu_freq(dev_priv, dev_priv->rps.min_freq_softlimit);
- mutex_unlock(&dev_priv->rps.hw_lock);
-
return 0;
}
@@ -5042,11 +5018,9 @@ i915_min_freq_set(void *data, u64 val)
u32 hw_max, hw_min;
int ret;
- if (INTEL_INFO(dev)->gen < 6)
+ if (INTEL_GEN(dev_priv) < 6)
return -ENODEV;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
DRM_DEBUG_DRIVER("Manually setting min freq to %llu\n", val);
ret = mutex_lock_interruptible(&dev_priv->rps.hw_lock);
diff --git a/drivers/gpu/drm/i915/i915_sysfs.c b/drivers/gpu/drm/i915/i915_sysfs.c
index d61829e54f93..58d016afd131 100644
--- a/drivers/gpu/drm/i915/i915_sysfs.c
+++ b/drivers/gpu/drm/i915/i915_sysfs.c
@@ -271,8 +271,6 @@ static ssize_t gt_act_freq_mhz_show(struct device *kdev,
struct drm_i915_private *dev_priv = to_i915(dev);
int ret;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
intel_runtime_pm_get(dev_priv);
mutex_lock(&dev_priv->rps.hw_lock);
@@ -303,19 +301,9 @@ static ssize_t gt_cur_freq_mhz_show(struct device *kdev,
struct drm_minor *minor = dev_to_drm_minor(kdev);
struct drm_device *dev = minor->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- int ret;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
- intel_runtime_pm_get(dev_priv);
-
- mutex_lock(&dev_priv->rps.hw_lock);
- ret = intel_gpu_freq(dev_priv, dev_priv->rps.cur_freq);
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- intel_runtime_pm_put(dev_priv);
-
- return snprintf(buf, PAGE_SIZE, "%d\n", ret);
+ return snprintf(buf, PAGE_SIZE, "%d\n",
+ intel_gpu_freq(dev_priv, dev_priv->rps.cur_freq));
}
static ssize_t vlv_rpe_freq_mhz_show(struct device *kdev,
@@ -325,9 +313,9 @@ static ssize_t vlv_rpe_freq_mhz_show(struct device *kdev,
struct drm_device *dev = minor->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- return snprintf(buf, PAGE_SIZE,
- "%d\n",
- intel_gpu_freq(dev_priv, dev_priv->rps.efficient_freq));
+ return snprintf(buf, PAGE_SIZE, "%d\n",
+ intel_gpu_freq(dev_priv,
+ dev_priv->rps.efficient_freq));
}
static ssize_t gt_max_freq_mhz_show(struct device *kdev, struct device_attribute *attr, char *buf)
@@ -335,15 +323,10 @@ static ssize_t gt_max_freq_mhz_show(struct device *kdev, struct device_attribute
struct drm_minor *minor = dev_to_drm_minor(kdev);
struct drm_device *dev = minor->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- int ret;
-
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
- mutex_lock(&dev_priv->rps.hw_lock);
- ret = intel_gpu_freq(dev_priv, dev_priv->rps.max_freq_softlimit);
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- return snprintf(buf, PAGE_SIZE, "%d\n", ret);
+ return snprintf(buf, PAGE_SIZE, "%d\n",
+ intel_gpu_freq(dev_priv,
+ dev_priv->rps.max_freq_softlimit));
}
static ssize_t gt_max_freq_mhz_store(struct device *kdev,
@@ -360,8 +343,6 @@ static ssize_t gt_max_freq_mhz_store(struct device *kdev,
if (ret)
return ret;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
intel_runtime_pm_get(dev_priv);
mutex_lock(&dev_priv->rps.hw_lock);
@@ -403,15 +384,10 @@ static ssize_t gt_min_freq_mhz_show(struct device *kdev, struct device_attribute
struct drm_minor *minor = dev_to_drm_minor(kdev);
struct drm_device *dev = minor->dev;
struct drm_i915_private *dev_priv = to_i915(dev);
- int ret;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
- mutex_lock(&dev_priv->rps.hw_lock);
- ret = intel_gpu_freq(dev_priv, dev_priv->rps.min_freq_softlimit);
- mutex_unlock(&dev_priv->rps.hw_lock);
-
- return snprintf(buf, PAGE_SIZE, "%d\n", ret);
+ return snprintf(buf, PAGE_SIZE, "%d\n",
+ intel_gpu_freq(dev_priv,
+ dev_priv->rps.min_freq_softlimit));
}
static ssize_t gt_min_freq_mhz_store(struct device *kdev,
@@ -428,8 +404,6 @@ static ssize_t gt_min_freq_mhz_store(struct device *kdev,
if (ret)
return ret;
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
intel_runtime_pm_get(dev_priv);
mutex_lock(&dev_priv->rps.hw_lock);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 04/64] drm/i915: Defer enabling rc6 til after we submit the first batch/context
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
2016-07-07 8:41 ` [PATCH 02/64] drm/i915: Preserve current RPS frequency across init Chris Wilson
2016-07-07 8:41 ` [PATCH 03/64] drm/i915: Remove superfluous powersave work flushing Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 05/64] drm/i915: Remove temporary RPM wakeref assert disables Chris Wilson
` (60 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx; +Cc: Mika Kuoppala
Some hardware requires a valid render context before it can initiate
rc6 power gating of the GPU; the default state of the GPU is not
sufficient and may lead to undefined behaviour. The first execution of
any batch will load the "golden render state", at which point it is safe
to enable rc6. As we do not forcibly load the kernel context at resume,
we have to hook into the batch submission to be sure that the render
state is setup before enabling rc6.
However, since we don't enable powersaving until that first batch, we
queued a delayed task in order to guarantee that the batch is indeed
submitted.
v2: Rearrange intel_disable_gt_powersave() to match.
v3: Apply user specified cur_freq (or idle_freq if not set).
v4: Give in, and supply a delayed work to autoenable rc6
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Mika Kuoppala <mika.kuoppala@intel.com>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
drivers/gpu/drm/i915/i915_drv.c | 7 +-
drivers/gpu/drm/i915/i915_gem.c | 1 +
drivers/gpu/drm/i915/intel_display.c | 2 +-
drivers/gpu/drm/i915/intel_drv.h | 2 +
drivers/gpu/drm/i915/intel_pm.c | 148 ++++++++++++++++++++---------------
drivers/gpu/drm/i915/intel_uncore.c | 2 +-
6 files changed, 95 insertions(+), 67 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 694edac2c703..2739603d5862 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -1339,7 +1339,7 @@ void i915_driver_unload(struct drm_device *dev)
i915_destroy_error_state(dev);
/* Flush any outstanding unpin_work. */
- flush_workqueue(dev_priv->wq);
+ drain_workqueue(dev_priv->wq);
intel_guc_fini(dev);
i915_gem_fini(dev);
@@ -1648,6 +1648,7 @@ static int i915_drm_resume(struct drm_device *dev)
intel_opregion_notify_adapter(dev_priv, PCI_D0);
+ intel_autoenable_gt_powersave(dev_priv);
drm_kms_helper_poll_enable(dev);
enable_rpm_wakeref_asserts(dev_priv);
@@ -1831,8 +1832,7 @@ int i915_reset(struct drm_i915_private *dev_priv)
* previous concerns that it doesn't respond well to some forms
* of re-init after reset.
*/
- if (INTEL_INFO(dev)->gen > 5)
- intel_enable_gt_powersave(dev_priv);
+ intel_autoenable_gt_powersave(dev_priv);
return 0;
@@ -2455,7 +2455,6 @@ static int intel_runtime_resume(struct device *device)
* we can do is to hope that things will still work (and disable RPM).
*/
i915_gem_init_swizzling(dev);
- gen6_update_ring_freq(dev_priv);
intel_runtime_pm_enable_interrupts(dev_priv);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 7fd44980798f..a8de083f8cb9 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2842,6 +2842,7 @@ static void i915_gem_mark_busy(const struct intel_engine_cs *engine)
intel_runtime_pm_get_noresume(dev_priv);
dev_priv->gt.awake = true;
+ intel_enable_gt_powersave(dev_priv);
i915_update_gfx_val(dev_priv);
if (INTEL_GEN(dev_priv) >= 6)
gen6_rps_busy(dev_priv);
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 111b350d1d7e..5fb7b3926537 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -15549,7 +15549,6 @@ void intel_modeset_init_hw(struct drm_device *dev)
dev_priv->atomic_cdclk_freq = dev_priv->cdclk_freq;
intel_init_clock_gating(dev);
- intel_enable_gt_powersave(dev_priv);
}
/*
@@ -16344,6 +16343,7 @@ void intel_modeset_cleanup(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
+ intel_suspend_gt_powersave(dev_priv);
intel_disable_gt_powersave(dev_priv);
/*
diff --git a/drivers/gpu/drm/i915/intel_drv.h b/drivers/gpu/drm/i915/intel_drv.h
index e6a24d2154d1..9ac8f8cdd64e 100644
--- a/drivers/gpu/drm/i915/intel_drv.h
+++ b/drivers/gpu/drm/i915/intel_drv.h
@@ -1686,7 +1686,9 @@ void intel_gpu_ips_init(struct drm_i915_private *dev_priv);
void intel_gpu_ips_teardown(void);
void intel_init_gt_powersave(struct drm_i915_private *dev_priv);
void intel_cleanup_gt_powersave(struct drm_i915_private *dev_priv);
+void intel_sanitize_gt_powersave(struct drm_i915_private *dev_priv);
void intel_enable_gt_powersave(struct drm_i915_private *dev_priv);
+void intel_autoenable_gt_powersave(struct drm_i915_private *dev_priv);
void intel_disable_gt_powersave(struct drm_i915_private *dev_priv);
void intel_suspend_gt_powersave(struct drm_i915_private *dev_priv);
void intel_reset_gt_powersave(struct drm_i915_private *dev_priv);
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index 49b5bc86e7c8..b12916a21b7b 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -6578,13 +6578,6 @@ void intel_cleanup_gt_powersave(struct drm_i915_private *dev_priv)
intel_runtime_pm_put(dev_priv);
}
-static void gen6_suspend_rps(struct drm_i915_private *dev_priv)
-{
- flush_delayed_work(&dev_priv->rps.delayed_resume_work);
-
- gen6_disable_rps_interrupts(dev_priv);
-}
-
/**
* intel_suspend_gt_powersave - suspend PM work and helper threads
* @dev_priv: i915 device
@@ -6598,50 +6591,65 @@ void intel_suspend_gt_powersave(struct drm_i915_private *dev_priv)
if (INTEL_GEN(dev_priv) < 6)
return;
- gen6_suspend_rps(dev_priv);
+ cancel_delayed_work_sync(&dev_priv->rps.delayed_resume_work);
+
+ gen6_disable_rps_interrupts(dev_priv);
/* Force GPU to min freq during suspend */
gen6_rps_idle(dev_priv);
}
+void intel_sanitize_gt_powersave(struct drm_i915_private *dev_priv)
+{
+ dev_priv->rps.enabled = true; /* force disabling */
+ intel_disable_gt_powersave(dev_priv);
+
+ gen6_reset_rps_interrupts(dev_priv);
+}
+
void intel_disable_gt_powersave(struct drm_i915_private *dev_priv)
{
- if (IS_IRONLAKE_M(dev_priv)) {
- ironlake_disable_drps(dev_priv);
- } else if (INTEL_INFO(dev_priv)->gen >= 6) {
- intel_suspend_gt_powersave(dev_priv);
+ if (!READ_ONCE(dev_priv->rps.enabled))
+ return;
- mutex_lock(&dev_priv->rps.hw_lock);
- if (INTEL_INFO(dev_priv)->gen >= 9) {
- gen9_disable_rc6(dev_priv);
- gen9_disable_rps(dev_priv);
- } else if (IS_CHERRYVIEW(dev_priv))
- cherryview_disable_rps(dev_priv);
- else if (IS_VALLEYVIEW(dev_priv))
- valleyview_disable_rps(dev_priv);
- else
- gen6_disable_rps(dev_priv);
+ mutex_lock(&dev_priv->rps.hw_lock);
- dev_priv->rps.enabled = false;
- mutex_unlock(&dev_priv->rps.hw_lock);
+ if (INTEL_GEN(dev_priv) >= 9) {
+ gen9_disable_rc6(dev_priv);
+ gen9_disable_rps(dev_priv);
+ } else if (IS_CHERRYVIEW(dev_priv)) {
+ cherryview_disable_rps(dev_priv);
+ } else if (IS_VALLEYVIEW(dev_priv)) {
+ valleyview_disable_rps(dev_priv);
+ } else if (INTEL_GEN(dev_priv) >= 6) {
+ gen6_disable_rps(dev_priv);
+ } else if (IS_IRONLAKE_M(dev_priv)) {
+ ironlake_disable_drps(dev_priv);
}
+
+ dev_priv->rps.enabled = false;
+ mutex_unlock(&dev_priv->rps.hw_lock);
}
-static void intel_gen6_powersave_work(struct work_struct *work)
+void intel_enable_gt_powersave(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv =
- container_of(work, struct drm_i915_private,
- rps.delayed_resume_work.work);
+ /* We shouldn't be disabling as we submit, so this should be less
+ * racy than it appears!
+ */
+ if (READ_ONCE(dev_priv->rps.enabled))
+ return;
- mutex_lock(&dev_priv->rps.hw_lock);
+ /* Powersaving is controlled by the host when inside a VM */
+ if (intel_vgpu_active(dev_priv))
+ return;
- gen6_reset_rps_interrupts(dev_priv);
+ mutex_lock(&dev_priv->rps.hw_lock);
if (IS_CHERRYVIEW(dev_priv)) {
cherryview_enable_rps(dev_priv);
} else if (IS_VALLEYVIEW(dev_priv)) {
valleyview_enable_rps(dev_priv);
- } else if (INTEL_INFO(dev_priv)->gen >= 9) {
+ } else if (INTEL_GEN(dev_priv) >= 9) {
gen9_enable_rc6(dev_priv);
gen9_enable_rps(dev_priv);
if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv))
@@ -6649,9 +6657,12 @@ static void intel_gen6_powersave_work(struct work_struct *work)
} else if (IS_BROADWELL(dev_priv)) {
gen8_enable_rps(dev_priv);
__gen6_update_ring_freq(dev_priv);
- } else {
+ } else if (INTEL_GEN(dev_priv) >= 6) {
gen6_enable_rps(dev_priv);
__gen6_update_ring_freq(dev_priv);
+ } else if (IS_IRONLAKE_M(dev_priv)) {
+ ironlake_enable_drps(dev_priv);
+ intel_init_emon(dev_priv);
}
WARN_ON(dev_priv->rps.max_freq < dev_priv->rps.min_freq);
@@ -6661,42 +6672,57 @@ static void intel_gen6_powersave_work(struct work_struct *work)
WARN_ON(dev_priv->rps.efficient_freq > dev_priv->rps.max_freq);
dev_priv->rps.enabled = true;
+ mutex_unlock(&dev_priv->rps.hw_lock);
+}
- gen6_enable_rps_interrupts(dev_priv);
+static void __intel_autoenable_gt_powersave(struct work_struct *work)
+{
+ struct drm_i915_private *dev_priv =
+ container_of(work, typeof(*dev_priv), rps.delayed_resume_work.work);
+ struct intel_engine_cs *rcs;
+ struct drm_i915_gem_request *req;
+ int ret;
- mutex_unlock(&dev_priv->rps.hw_lock);
+ if (READ_ONCE(dev_priv->rps.enabled))
+ return;
+
+ rcs = &dev_priv->engine[RCS];
+ if (rcs->last_context)
+ return;
+
+ if (!rcs->init_context)
+ return;
+
+ intel_runtime_pm_get(dev_priv);
+ mutex_lock(&dev_priv->drm.struct_mutex);
+ req = i915_gem_request_alloc(rcs, dev_priv->kernel_context);
+ if (IS_ERR(req))
+ goto unlock;
+
+ ret = 0;
+ if (!i915.enable_execlists)
+ ret = i915_switch_context(req);
+ if (ret == 0)
+ rcs->init_context(req);
+ i915_add_request_no_flush(req);
+
+unlock:
+ mutex_unlock(&dev_priv->drm.struct_mutex);
intel_runtime_pm_put(dev_priv);
}
-void intel_enable_gt_powersave(struct drm_i915_private *dev_priv)
+void intel_autoenable_gt_powersave(struct drm_i915_private *dev_priv)
{
- /* Powersaving is controlled by the host when inside a VM */
- if (intel_vgpu_active(dev_priv))
+ if (READ_ONCE(dev_priv->rps.enabled))
return;
- if (IS_IRONLAKE_M(dev_priv)) {
- ironlake_enable_drps(dev_priv);
- mutex_lock(&dev_priv->drm.struct_mutex);
- intel_init_emon(dev_priv);
- mutex_unlock(&dev_priv->drm.struct_mutex);
- } else if (INTEL_INFO(dev_priv)->gen >= 6) {
- /*
- * PCU communication is slow and this doesn't need to be
- * done at any specific time, so do this out of our fast path
- * to make resume and init faster.
- *
- * We depend on the HW RC6 power context save/restore
- * mechanism when entering D3 through runtime PM suspend. So
- * disable RPM until RPS/RC6 is properly setup. We can only
- * get here via the driver load/system resume/runtime resume
- * paths, so the _noresume version is enough (and in case of
- * runtime resume it's necessary).
- */
- if (schedule_delayed_work(&dev_priv->rps.delayed_resume_work,
- round_jiffies_up_relative(HZ)))
- intel_runtime_pm_get_noresume(dev_priv);
- }
+ if (INTEL_GEN(dev_priv) < 6)
+ return;
+
+ queue_delayed_work(dev_priv->wq,
+ &dev_priv->rps.delayed_resume_work,
+ round_jiffies_up_relative(2*HZ));
}
void intel_reset_gt_powersave(struct drm_i915_private *dev_priv)
@@ -6704,7 +6730,7 @@ void intel_reset_gt_powersave(struct drm_i915_private *dev_priv)
if (INTEL_INFO(dev_priv)->gen < 6)
return;
- gen6_suspend_rps(dev_priv);
+ gen6_disable_rps_interrupts(dev_priv);
dev_priv->rps.enabled = false;
}
@@ -7815,7 +7841,7 @@ void intel_pm_setup(struct drm_device *dev)
spin_lock_init(&dev_priv->rps.client_lock);
INIT_DELAYED_WORK(&dev_priv->rps.delayed_resume_work,
- intel_gen6_powersave_work);
+ __intel_autoenable_gt_powersave);
INIT_LIST_HEAD(&dev_priv->rps.clients);
INIT_LIST_HEAD(&dev_priv->rps.semaphores.link);
INIT_LIST_HEAD(&dev_priv->rps.mmioflips.link);
diff --git a/drivers/gpu/drm/i915/intel_uncore.c b/drivers/gpu/drm/i915/intel_uncore.c
index ff80a81b1a84..eeb4cbce19ff 100644
--- a/drivers/gpu/drm/i915/intel_uncore.c
+++ b/drivers/gpu/drm/i915/intel_uncore.c
@@ -435,7 +435,7 @@ void intel_uncore_sanitize(struct drm_i915_private *dev_priv)
i915.enable_rc6 = sanitize_rc6_option(dev_priv, i915.enable_rc6);
/* BIOS often leaves RC6 enabled, but disable it for hw init */
- intel_disable_gt_powersave(dev_priv);
+ intel_sanitize_gt_powersave(dev_priv);
}
static void __intel_uncore_forcewake_get(struct drm_i915_private *dev_priv,
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 05/64] drm/i915: Remove temporary RPM wakeref assert disables
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (2 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 04/64] drm/i915: Defer enabling rc6 til after we submit the first batch/context Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 06/64] drm: Restore double clflush on the last partial cacheline Chris Wilson
` (59 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Now that the last couple of hacks have been removed from the runtime
powermanagement users, we can fully enable the asserts.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_drv.h | 7 -------
1 file changed, 7 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_drv.h b/drivers/gpu/drm/i915/intel_drv.h
index 9ac8f8cdd64e..17b66240936f 100644
--- a/drivers/gpu/drm/i915/intel_drv.h
+++ b/drivers/gpu/drm/i915/intel_drv.h
@@ -1654,13 +1654,6 @@ enable_rpm_wakeref_asserts(struct drm_i915_private *dev_priv)
atomic_dec(&dev_priv->pm.wakeref_count);
}
-/* TODO: convert users of these to rely instead on proper RPM refcounting */
-#define DISABLE_RPM_WAKEREF_ASSERTS(dev_priv) \
- disable_rpm_wakeref_asserts(dev_priv)
-
-#define ENABLE_RPM_WAKEREF_ASSERTS(dev_priv) \
- enable_rpm_wakeref_asserts(dev_priv)
-
void intel_runtime_pm_get(struct drm_i915_private *dev_priv);
bool intel_runtime_pm_get_if_in_use(struct drm_i915_private *dev_priv);
void intel_runtime_pm_get_noresume(struct drm_i915_private *dev_priv);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 06/64] drm: Restore double clflush on the last partial cacheline
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (3 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 05/64] drm/i915: Remove temporary RPM wakeref assert disables Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-12 13:57 ` Daniel Vetter
2016-07-07 8:41 ` [PATCH 07/64] drm/i915: Move GEM request routines to i915_gem_request.c Chris Wilson
` (58 subsequent siblings)
63 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Cc: Chris Wilson, dri-devel, Akash Goel, Imre Deak, Daniel Vetter,
Jason Ekstrand, stable
This effectively reverts
commit afcd950cafea6e27b739fe7772cbbeed37d05b8b
Author: Chris Wilson <chris@chris-wilson.co.uk>
Date: Wed Jun 10 15:58:01 2015 +0100
drm: Avoid the double clflush on the last cache line in drm_clflush_virt_range()
as we have observed issues with serialisation of the clflush operations
on Baytrail+ Atoms with partial updates. Applying the double flush on the
last cacheline forces that clflush to be ordered with respect to the
previous clflush, and the mfence then protects against prefetches crossing
the clflush boundary.
The same issue can be demonstrated in userspace with igt/gem_exec_flush.
Fixes: afcd950cafea6 (drm: Avoid the double clflush on the last cache...)
Testcase: igt/gem_concurrent_blit
Testcase: igt/gem_partial_pread_pwrite
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92845
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: dri-devel@lists.freedesktop.org
Cc: Akash Goel <akash.goel@intel.com>
Cc: Imre Deak <imre.deak@intel.com>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Jason Ekstrand <jason.ekstrand@intel.com>
Cc: stable@vger.kernel.org
Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
---
drivers/gpu/drm/drm_cache.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c
index 059f7c39c582..a7916e5f8864 100644
--- a/drivers/gpu/drm/drm_cache.c
+++ b/drivers/gpu/drm/drm_cache.c
@@ -136,6 +136,7 @@ drm_clflush_virt_range(void *addr, unsigned long length)
mb();
for (; addr < end; addr += size)
clflushopt(addr);
+ clflushopt(end - 1); /* force serialisation */
mb();
return;
}
--
2.8.1
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 07/64] drm/i915: Move GEM request routines to i915_gem_request.c
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (4 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 06/64] drm: Restore double clflush on the last partial cacheline Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next Chris Wilson
` (57 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Migrate the request operations out of the main body of i915_gem.c and
into their own C file for easier expansion.
v2: Move __i915_add_request() across as well
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/Makefile | 1 +
drivers/gpu/drm/i915/i915_drv.h | 209 +---------
drivers/gpu/drm/i915/i915_gem.c | 655 +------------------------------
drivers/gpu/drm/i915/i915_gem_request.c | 658 ++++++++++++++++++++++++++++++++
drivers/gpu/drm/i915/i915_gem_request.h | 238 ++++++++++++
5 files changed, 905 insertions(+), 856 deletions(-)
create mode 100644 drivers/gpu/drm/i915/i915_gem_request.c
create mode 100644 drivers/gpu/drm/i915/i915_gem_request.h
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index 684fc1cd08fa..f7a3f1421869 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -33,6 +33,7 @@ i915-y += i915_cmd_parser.o \
i915_gem_gtt.o \
i915_gem.o \
i915_gem_render_state.o \
+ i915_gem_request.o \
i915_gem_shrinker.o \
i915_gem_stolen.o \
i915_gem_tiling.o \
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index d2c60992a508..9b062fe28483 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -61,6 +61,7 @@
#include "i915_gem.h"
#include "i915_gem_gtt.h"
#include "i915_gem_render_state.h"
+#include "i915_gem_request.h"
#include "intel_gvt.h"
@@ -2361,171 +2362,6 @@ static inline struct scatterlist *__sg_next(struct scatterlist *sg)
(((__iter).curr += PAGE_SIZE) < (__iter).max) || \
((__iter) = __sgt_iter(__sg_next((__iter).sgp), false), 0))
-/**
- * Request queue structure.
- *
- * The request queue allows us to note sequence numbers that have been emitted
- * and may be associated with active buffers to be retired.
- *
- * By keeping this list, we can avoid having to do questionable sequence
- * number comparisons on buffer last_read|write_seqno. It also allows an
- * emission time to be associated with the request for tracking how far ahead
- * of the GPU the submission is.
- *
- * The requests are reference counted, so upon creation they should have an
- * initial reference taken using kref_init
- */
-struct drm_i915_gem_request {
- struct kref ref;
-
- /** On Which ring this request was generated */
- struct drm_i915_private *i915;
- struct intel_engine_cs *engine;
- struct intel_signal_node signaling;
-
- /** GEM sequence number associated with the previous request,
- * when the HWS breadcrumb is equal to this the GPU is processing
- * this request.
- */
- u32 previous_seqno;
-
- /** GEM sequence number associated with this request,
- * when the HWS breadcrumb is equal or greater than this the GPU
- * has finished processing this request.
- */
- u32 seqno;
-
- /** Position in the ringbuffer of the start of the request */
- u32 head;
-
- /**
- * Position in the ringbuffer of the start of the postfix.
- * This is required to calculate the maximum available ringbuffer
- * space without overwriting the postfix.
- */
- u32 postfix;
-
- /** Position in the ringbuffer of the end of the whole request */
- u32 tail;
-
- /** Preallocate space in the ringbuffer for the emitting the request */
- u32 reserved_space;
-
- /**
- * Context and ring buffer related to this request
- * Contexts are refcounted, so when this request is associated with a
- * context, we must increment the context's refcount, to guarantee that
- * it persists while any request is linked to it. Requests themselves
- * are also refcounted, so the request will only be freed when the last
- * reference to it is dismissed, and the code in
- * i915_gem_request_free() will then decrement the refcount on the
- * context.
- */
- struct i915_gem_context *ctx;
- struct intel_ringbuffer *ringbuf;
-
- /**
- * Context related to the previous request.
- * As the contexts are accessed by the hardware until the switch is
- * completed to a new context, the hardware may still be writing
- * to the context object after the breadcrumb is visible. We must
- * not unpin/unbind/prune that object whilst still active and so
- * we keep the previous context pinned until the following (this)
- * request is retired.
- */
- struct i915_gem_context *previous_context;
-
- /** Batch buffer related to this request if any (used for
- error state dump only) */
- struct drm_i915_gem_object *batch_obj;
-
- /** Time at which this request was emitted, in jiffies. */
- unsigned long emitted_jiffies;
-
- /** global list entry for this request */
- struct list_head list;
-
- struct drm_i915_file_private *file_priv;
- /** file_priv list entry for this request */
- struct list_head client_list;
-
- /** process identifier submitting this request */
- struct pid *pid;
-
- /**
- * The ELSP only accepts two elements at a time, so we queue
- * context/tail pairs on a given queue (ring->execlist_queue) until the
- * hardware is available. The queue serves a double purpose: we also use
- * it to keep track of the up to 2 contexts currently in the hardware
- * (usually one in execution and the other queued up by the GPU): We
- * only remove elements from the head of the queue when the hardware
- * informs us that an element has been completed.
- *
- * All accesses to the queue are mediated by a spinlock
- * (ring->execlist_lock).
- */
-
- /** Execlist link in the submission queue.*/
- struct list_head execlist_link;
-
- /** Execlists no. of times this request has been sent to the ELSP */
- int elsp_submitted;
-
- /** Execlists context hardware id. */
- unsigned ctx_hw_id;
-};
-
-struct drm_i915_gem_request * __must_check
-i915_gem_request_alloc(struct intel_engine_cs *engine,
- struct i915_gem_context *ctx);
-void i915_gem_request_free(struct kref *req_ref);
-int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
- struct drm_file *file);
-
-static inline uint32_t
-i915_gem_request_get_seqno(struct drm_i915_gem_request *req)
-{
- return req ? req->seqno : 0;
-}
-
-static inline struct intel_engine_cs *
-i915_gem_request_get_engine(struct drm_i915_gem_request *req)
-{
- return req ? req->engine : NULL;
-}
-
-static inline struct drm_i915_gem_request *
-i915_gem_request_reference(struct drm_i915_gem_request *req)
-{
- if (req)
- kref_get(&req->ref);
- return req;
-}
-
-static inline void
-i915_gem_request_unreference(struct drm_i915_gem_request *req)
-{
- kref_put(&req->ref, i915_gem_request_free);
-}
-
-static inline void i915_gem_request_assign(struct drm_i915_gem_request **pdst,
- struct drm_i915_gem_request *src)
-{
- if (src)
- i915_gem_request_reference(src);
-
- if (*pdst)
- i915_gem_request_unreference(*pdst);
-
- *pdst = src;
-}
-
-/*
- * XXX: i915_gem_request_completed should be here but currently needs the
- * definition of i915_seqno_passed() which is below. It will be moved in
- * a later patch when the call to i915_seqno_passed() is obsoleted...
- */
-
/*
* A command that requires special handling by the command parser.
*/
@@ -3280,37 +3116,6 @@ void i915_gem_track_fb(struct drm_i915_gem_object *old,
struct drm_i915_gem_object *new,
unsigned frontbuffer_bits);
-/**
- * Returns true if seq1 is later than seq2.
- */
-static inline bool
-i915_seqno_passed(uint32_t seq1, uint32_t seq2)
-{
- return (int32_t)(seq1 - seq2) >= 0;
-}
-
-static inline bool i915_gem_request_started(const struct drm_i915_gem_request *req)
-{
- return i915_seqno_passed(intel_engine_get_seqno(req->engine),
- req->previous_seqno);
-}
-
-static inline bool i915_gem_request_completed(const struct drm_i915_gem_request *req)
-{
- return i915_seqno_passed(intel_engine_get_seqno(req->engine),
- req->seqno);
-}
-
-bool __i915_spin_request(const struct drm_i915_gem_request *request,
- int state, unsigned long timeout_us);
-static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
- int state, unsigned long timeout_us)
-{
- return (i915_gem_request_started(request) &&
- __i915_spin_request(request, state, timeout_us));
-}
-
-int __must_check i915_gem_get_seqno(struct drm_i915_private *dev_priv, u32 *seqno);
int __must_check i915_gem_set_seqno(struct drm_device *dev, u32 seqno);
struct drm_i915_gem_request *
@@ -3368,18 +3173,6 @@ void i915_gem_init_swizzling(struct drm_device *dev);
void i915_gem_cleanup_engines(struct drm_device *dev);
int __must_check i915_gem_wait_for_idle(struct drm_i915_private *dev_priv);
int __must_check i915_gem_suspend(struct drm_device *dev);
-void __i915_add_request(struct drm_i915_gem_request *req,
- struct drm_i915_gem_object *batch_obj,
- bool flush_caches);
-#define i915_add_request(req) \
- __i915_add_request(req, NULL, true)
-#define i915_add_request_no_flush(req) \
- __i915_add_request(req, NULL, false)
-int __i915_wait_request(struct drm_i915_gem_request *req,
- bool interruptible,
- s64 *timeout,
- struct intel_rps_client *rps);
-int __must_check i915_wait_request(struct drm_i915_gem_request *req);
int i915_gem_fault(struct vm_area_struct *vma, struct vm_fault *vmf);
int __must_check
i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a8de083f8cb9..14cc87b916c6 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1325,365 +1325,6 @@ put_rpm:
return ret;
}
-static int
-i915_gem_check_wedge(unsigned reset_counter, bool interruptible)
-{
- if (__i915_terminally_wedged(reset_counter))
- return -EIO;
-
- if (__i915_reset_in_progress(reset_counter)) {
- /* Non-interruptible callers can't handle -EAGAIN, hence return
- * -EIO unconditionally for these. */
- if (!interruptible)
- return -EIO;
-
- return -EAGAIN;
- }
-
- return 0;
-}
-
-static unsigned long local_clock_us(unsigned *cpu)
-{
- unsigned long t;
-
- /* Cheaply and approximately convert from nanoseconds to microseconds.
- * The result and subsequent calculations are also defined in the same
- * approximate microseconds units. The principal source of timing
- * error here is from the simple truncation.
- *
- * Note that local_clock() is only defined wrt to the current CPU;
- * the comparisons are no longer valid if we switch CPUs. Instead of
- * blocking preemption for the entire busywait, we can detect the CPU
- * switch and use that as indicator of system load and a reason to
- * stop busywaiting, see busywait_stop().
- */
- *cpu = get_cpu();
- t = local_clock() >> 10;
- put_cpu();
-
- return t;
-}
-
-static bool busywait_stop(unsigned long timeout, unsigned cpu)
-{
- unsigned this_cpu;
-
- if (time_after(local_clock_us(&this_cpu), timeout))
- return true;
-
- return this_cpu != cpu;
-}
-
-bool __i915_spin_request(const struct drm_i915_gem_request *req,
- int state, unsigned long timeout_us)
-{
- unsigned cpu;
-
- /* When waiting for high frequency requests, e.g. during synchronous
- * rendering split between the CPU and GPU, the finite amount of time
- * required to set up the irq and wait upon it limits the response
- * rate. By busywaiting on the request completion for a short while we
- * can service the high frequency waits as quick as possible. However,
- * if it is a slow request, we want to sleep as quickly as possible.
- * The tradeoff between waiting and sleeping is roughly the time it
- * takes to sleep on a request, on the order of a microsecond.
- */
-
- timeout_us += local_clock_us(&cpu);
- do {
- if (i915_gem_request_completed(req))
- return true;
-
- if (signal_pending_state(state, current))
- break;
-
- if (busywait_stop(timeout_us, cpu))
- break;
-
- cpu_relax_lowlatency();
- } while (!need_resched());
-
- return false;
-}
-
-/**
- * __i915_wait_request - wait until execution of request has finished
- * @req: duh!
- * @interruptible: do an interruptible wait (normally yes)
- * @timeout: in - how long to wait (NULL forever); out - how much time remaining
- * @rps: RPS client
- *
- * Note: It is of utmost importance that the passed in seqno and reset_counter
- * values have been read by the caller in an smp safe manner. Where read-side
- * locks are involved, it is sufficient to read the reset_counter before
- * unlocking the lock that protects the seqno. For lockless tricks, the
- * reset_counter _must_ be read before, and an appropriate smp_rmb must be
- * inserted.
- *
- * Returns 0 if the request was found within the alloted time. Else returns the
- * errno with remaining time filled in timeout argument.
- */
-int __i915_wait_request(struct drm_i915_gem_request *req,
- bool interruptible,
- s64 *timeout,
- struct intel_rps_client *rps)
-{
- int state = interruptible ? TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE;
- DEFINE_WAIT(reset);
- struct intel_wait wait;
- unsigned long timeout_remain;
- s64 before = 0; /* Only to silence a compiler warning. */
- int ret = 0;
-
- might_sleep();
-
- if (list_empty(&req->list))
- return 0;
-
- if (i915_gem_request_completed(req))
- return 0;
-
- timeout_remain = MAX_SCHEDULE_TIMEOUT;
- if (timeout) {
- if (WARN_ON(*timeout < 0))
- return -EINVAL;
-
- if (*timeout == 0)
- return -ETIME;
-
- timeout_remain = nsecs_to_jiffies_timeout(*timeout);
-
- /*
- * Record current time in case interrupted by signal, or wedged.
- */
- before = ktime_get_raw_ns();
- }
-
- trace_i915_gem_request_wait_begin(req);
-
- /* This client is about to stall waiting for the GPU. In many cases
- * this is undesirable and limits the throughput of the system, as
- * many clients cannot continue processing user input/output whilst
- * blocked. RPS autotuning may take tens of milliseconds to respond
- * to the GPU load and thus incurs additional latency for the client.
- * We can circumvent that by promoting the GPU frequency to maximum
- * before we wait. This makes the GPU throttle up much more quickly
- * (good for benchmarks and user experience, e.g. window animations),
- * but at a cost of spending more power processing the workload
- * (bad for battery). Not all clients even want their results
- * immediately and for them we should just let the GPU select its own
- * frequency to maximise efficiency. To prevent a single client from
- * forcing the clocks too high for the whole system, we only allow
- * each client to waitboost once in a busy period.
- */
- if (INTEL_INFO(req->i915)->gen >= 6)
- gen6_rps_boost(req->i915, rps, req->emitted_jiffies);
-
- /* Optimistic spin for the next ~jiffie before touching IRQs */
- if (i915_spin_request(req, state, 5))
- goto complete;
-
- set_current_state(state);
- add_wait_queue(&req->i915->gpu_error.wait_queue, &reset);
-
- intel_wait_init(&wait, req->seqno);
- if (intel_engine_add_wait(req->engine, &wait))
- /* In order to check that we haven't missed the interrupt
- * as we enabled it, we need to kick ourselves to do a
- * coherent check on the seqno before we sleep.
- */
- goto wakeup;
-
- for (;;) {
- if (signal_pending_state(state, current)) {
- ret = -ERESTARTSYS;
- break;
- }
-
- timeout_remain = io_schedule_timeout(timeout_remain);
- if (timeout_remain == 0) {
- ret = -ETIME;
- break;
- }
-
- if (intel_wait_complete(&wait))
- break;
-
- set_current_state(state);
-
-wakeup:
- /* Carefully check if the request is complete, giving time
- * for the seqno to be visible following the interrupt.
- * We also have to check in case we are kicked by the GPU
- * reset in order to drop the struct_mutex.
- */
- if (__i915_request_irq_complete(req))
- break;
-
- /* Only spin if we know the GPU is processing this request */
- if (i915_spin_request(req, state, 2))
- break;
- }
- remove_wait_queue(&req->i915->gpu_error.wait_queue, &reset);
-
- intel_engine_remove_wait(req->engine, &wait);
- __set_current_state(TASK_RUNNING);
-complete:
- trace_i915_gem_request_wait_end(req);
-
- if (timeout) {
- s64 tres = *timeout - (ktime_get_raw_ns() - before);
-
- *timeout = tres < 0 ? 0 : tres;
-
- /*
- * Apparently ktime isn't accurate enough and occasionally has a
- * bit of mismatch in the jiffies<->nsecs<->ktime loop. So patch
- * things up to make the test happy. We allow up to 1 jiffy.
- *
- * This is a regrssion from the timespec->ktime conversion.
- */
- if (ret == -ETIME && *timeout < jiffies_to_usecs(1)*1000)
- *timeout = 0;
- }
-
- if (rps && req->seqno == req->engine->last_submitted_seqno) {
- /* The GPU is now idle and this client has stalled.
- * Since no other client has submitted a request in the
- * meantime, assume that this client is the only one
- * supplying work to the GPU but is unable to keep that
- * work supplied because it is waiting. Since the GPU is
- * then never kept fully busy, RPS autoclocking will
- * keep the clocks relatively low, causing further delays.
- * Compensate by giving the synchronous client credit for
- * a waitboost next time.
- */
- spin_lock(&req->i915->rps.client_lock);
- list_del_init(&rps->link);
- spin_unlock(&req->i915->rps.client_lock);
- }
-
- return ret;
-}
-
-int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
- struct drm_file *file)
-{
- struct drm_i915_file_private *file_priv;
-
- WARN_ON(!req || !file || req->file_priv);
-
- if (!req || !file)
- return -EINVAL;
-
- if (req->file_priv)
- return -EINVAL;
-
- file_priv = file->driver_priv;
-
- spin_lock(&file_priv->mm.lock);
- req->file_priv = file_priv;
- list_add_tail(&req->client_list, &file_priv->mm.request_list);
- spin_unlock(&file_priv->mm.lock);
-
- req->pid = get_pid(task_pid(current));
-
- return 0;
-}
-
-static inline void
-i915_gem_request_remove_from_client(struct drm_i915_gem_request *request)
-{
- struct drm_i915_file_private *file_priv = request->file_priv;
-
- if (!file_priv)
- return;
-
- spin_lock(&file_priv->mm.lock);
- list_del(&request->client_list);
- request->file_priv = NULL;
- spin_unlock(&file_priv->mm.lock);
-
- put_pid(request->pid);
- request->pid = NULL;
-}
-
-static void i915_gem_request_retire(struct drm_i915_gem_request *request)
-{
- trace_i915_gem_request_retire(request);
-
- /* We know the GPU must have read the request to have
- * sent us the seqno + interrupt, so use the position
- * of tail of the request to update the last known position
- * of the GPU head.
- *
- * Note this requires that we are always called in request
- * completion order.
- */
- request->ringbuf->last_retired_head = request->postfix;
-
- list_del_init(&request->list);
- i915_gem_request_remove_from_client(request);
-
- if (request->previous_context) {
- if (i915.enable_execlists)
- intel_lr_context_unpin(request->previous_context,
- request->engine);
- }
-
- i915_gem_context_unreference(request->ctx);
- i915_gem_request_unreference(request);
-}
-
-static void
-__i915_gem_request_retire__upto(struct drm_i915_gem_request *req)
-{
- struct intel_engine_cs *engine = req->engine;
- struct drm_i915_gem_request *tmp;
-
- lockdep_assert_held(&engine->i915->drm.struct_mutex);
-
- if (list_empty(&req->list))
- return;
-
- do {
- tmp = list_first_entry(&engine->request_list,
- typeof(*tmp), list);
-
- i915_gem_request_retire(tmp);
- } while (tmp != req);
-
- WARN_ON(i915_verify_lists(engine->dev));
-}
-
-/**
- * Waits for a request to be signaled, and cleans up the
- * request and object lists appropriately for that event.
- * @req: request to wait on
- */
-int
-i915_wait_request(struct drm_i915_gem_request *req)
-{
- struct drm_i915_private *dev_priv = req->i915;
- bool interruptible;
- int ret;
-
- interruptible = dev_priv->mm.interruptible;
-
- BUG_ON(!mutex_is_locked(&dev_priv->drm.struct_mutex));
-
- ret = __i915_wait_request(req, interruptible, NULL, NULL);
- if (ret)
- return ret;
-
- /* If the GPU hung, we want to keep the requests to find the guilty. */
- if (!i915_reset_in_progress(&dev_priv->gpu_error))
- __i915_gem_request_retire__upto(req);
-
- return 0;
-}
-
/**
* Ensures that all rendering to the object has completed and the object is
* safe to unbind from the GTT or access from the CPU.
@@ -1740,7 +1381,7 @@ i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
i915_gem_object_retire__write(obj);
if (!i915_reset_in_progress(&req->i915->gpu_error))
- __i915_gem_request_retire__upto(req);
+ i915_gem_request_retire_upto(req);
}
/* A nonblocking variant of the above wait. This is a highly dangerous routine
@@ -2761,193 +2402,6 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
drm_gem_object_unreference(&obj->base);
}
-static int
-i915_gem_init_seqno(struct drm_i915_private *dev_priv, u32 seqno)
-{
- struct intel_engine_cs *engine;
- int ret;
-
- /* Carefully retire all requests without writing to the rings */
- for_each_engine(engine, dev_priv) {
- ret = intel_engine_idle(engine);
- if (ret)
- return ret;
- }
- i915_gem_retire_requests(dev_priv);
-
- /* If the seqno wraps around, we need to clear the breadcrumb rbtree */
- if (!i915_seqno_passed(seqno, dev_priv->next_seqno)) {
- while (intel_kick_waiters(dev_priv) ||
- intel_kick_signalers(dev_priv))
- yield();
- }
-
- /* Finally reset hw state */
- for_each_engine(engine, dev_priv)
- intel_ring_init_seqno(engine, seqno);
-
- return 0;
-}
-
-int i915_gem_set_seqno(struct drm_device *dev, u32 seqno)
-{
- struct drm_i915_private *dev_priv = to_i915(dev);
- int ret;
-
- if (seqno == 0)
- return -EINVAL;
-
- /* HWS page needs to be set less than what we
- * will inject to ring
- */
- ret = i915_gem_init_seqno(dev_priv, seqno - 1);
- if (ret)
- return ret;
-
- /* Carefully set the last_seqno value so that wrap
- * detection still works
- */
- dev_priv->next_seqno = seqno;
- dev_priv->last_seqno = seqno - 1;
- if (dev_priv->last_seqno == 0)
- dev_priv->last_seqno--;
-
- return 0;
-}
-
-int
-i915_gem_get_seqno(struct drm_i915_private *dev_priv, u32 *seqno)
-{
- /* reserve 0 for non-seqno */
- if (dev_priv->next_seqno == 0) {
- int ret = i915_gem_init_seqno(dev_priv, 0);
- if (ret)
- return ret;
-
- dev_priv->next_seqno = 1;
- }
-
- *seqno = dev_priv->last_seqno = dev_priv->next_seqno++;
- return 0;
-}
-
-static void i915_gem_mark_busy(const struct intel_engine_cs *engine)
-{
- struct drm_i915_private *dev_priv = engine->i915;
-
- dev_priv->gt.active_engines |= intel_engine_flag(engine);
- if (dev_priv->gt.awake)
- return;
-
- intel_runtime_pm_get_noresume(dev_priv);
- dev_priv->gt.awake = true;
-
- intel_enable_gt_powersave(dev_priv);
- i915_update_gfx_val(dev_priv);
- if (INTEL_GEN(dev_priv) >= 6)
- gen6_rps_busy(dev_priv);
-
- queue_delayed_work(dev_priv->wq,
- &dev_priv->gt.retire_work,
- round_jiffies_up_relative(HZ));
-}
-
-/*
- * NB: This function is not allowed to fail. Doing so would mean the the
- * request is not being tracked for completion but the work itself is
- * going to happen on the hardware. This would be a Bad Thing(tm).
- */
-void __i915_add_request(struct drm_i915_gem_request *request,
- struct drm_i915_gem_object *obj,
- bool flush_caches)
-{
- struct intel_engine_cs *engine;
- struct intel_ringbuffer *ringbuf;
- u32 request_start;
- u32 reserved_tail;
- int ret;
-
- if (WARN_ON(request == NULL))
- return;
-
- engine = request->engine;
- ringbuf = request->ringbuf;
-
- /*
- * To ensure that this call will not fail, space for its emissions
- * should already have been reserved in the ring buffer. Let the ring
- * know that it is time to use that space up.
- */
- request_start = intel_ring_get_tail(ringbuf);
- reserved_tail = request->reserved_space;
- request->reserved_space = 0;
-
- /*
- * Emit any outstanding flushes - execbuf can fail to emit the flush
- * after having emitted the batchbuffer command. Hence we need to fix
- * things up similar to emitting the lazy request. The difference here
- * is that the flush _must_ happen before the next request, no matter
- * what.
- */
- if (flush_caches) {
- if (i915.enable_execlists)
- ret = logical_ring_flush_all_caches(request);
- else
- ret = intel_ring_flush_all_caches(request);
- /* Not allowed to fail! */
- WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
- }
-
- trace_i915_gem_request_add(request);
-
- request->head = request_start;
-
- /* Whilst this request exists, batch_obj will be on the
- * active_list, and so will hold the active reference. Only when this
- * request is retired will the the batch_obj be moved onto the
- * inactive_list and lose its active reference. Hence we do not need
- * to explicitly hold another reference here.
- */
- request->batch_obj = obj;
-
- /* Seal the request and mark it as pending execution. Note that
- * we may inspect this state, without holding any locks, during
- * hangcheck. Hence we apply the barrier to ensure that we do not
- * see a more recent value in the hws than we are tracking.
- */
- request->emitted_jiffies = jiffies;
- request->previous_seqno = engine->last_submitted_seqno;
- smp_store_mb(engine->last_submitted_seqno, request->seqno);
- list_add_tail(&request->list, &engine->request_list);
-
- /* Record the position of the start of the request so that
- * should we detect the updated seqno part-way through the
- * GPU processing the request, we never over-estimate the
- * position of the head.
- */
- request->postfix = intel_ring_get_tail(ringbuf);
-
- if (i915.enable_execlists)
- ret = engine->emit_request(request);
- else {
- ret = engine->add_request(request);
-
- request->tail = intel_ring_get_tail(ringbuf);
- }
- /* Not allowed to fail! */
- WARN(ret, "emit|add_request failed: %d!\n", ret);
- /* Sanity check that the reserved size was large enough. */
- ret = intel_ring_get_tail(ringbuf) - request_start;
- if (ret < 0)
- ret += ringbuf->size;
- WARN_ONCE(ret > reserved_tail,
- "Not enough space reserved (%d bytes) "
- "for adding the request (%d bytes)\n",
- reserved_tail, ret);
-
- i915_gem_mark_busy(engine);
-}
-
static bool i915_context_is_banned(const struct i915_gem_context *ctx)
{
unsigned long elapsed;
@@ -2979,101 +2433,6 @@ static void i915_set_reset_status(struct i915_gem_context *ctx,
}
}
-void i915_gem_request_free(struct kref *req_ref)
-{
- struct drm_i915_gem_request *req = container_of(req_ref,
- typeof(*req), ref);
- kmem_cache_free(req->i915->requests, req);
-}
-
-static inline int
-__i915_gem_request_alloc(struct intel_engine_cs *engine,
- struct i915_gem_context *ctx,
- struct drm_i915_gem_request **req_out)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- unsigned reset_counter = i915_reset_counter(&dev_priv->gpu_error);
- struct drm_i915_gem_request *req;
- int ret;
-
- if (!req_out)
- return -EINVAL;
-
- *req_out = NULL;
-
- /* ABI: Before userspace accesses the GPU (e.g. execbuffer), report
- * EIO if the GPU is already wedged, or EAGAIN to drop the struct_mutex
- * and restart.
- */
- ret = i915_gem_check_wedge(reset_counter, dev_priv->mm.interruptible);
- if (ret)
- return ret;
-
- req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
- if (req == NULL)
- return -ENOMEM;
-
- ret = i915_gem_get_seqno(engine->i915, &req->seqno);
- if (ret)
- goto err;
-
- kref_init(&req->ref);
- req->i915 = dev_priv;
- req->engine = engine;
- req->ctx = ctx;
- i915_gem_context_reference(req->ctx);
-
- /*
- * Reserve space in the ring buffer for all the commands required to
- * eventually emit this request. This is to guarantee that the
- * i915_add_request() call can't fail. Note that the reserve may need
- * to be redone if the request is not actually submitted straight
- * away, e.g. because a GPU scheduler has deferred it.
- */
- req->reserved_space = MIN_SPACE_FOR_ADD_REQUEST;
-
- if (i915.enable_execlists)
- ret = intel_logical_ring_alloc_request_extras(req);
- else
- ret = intel_ring_alloc_request_extras(req);
- if (ret)
- goto err_ctx;
-
- *req_out = req;
- return 0;
-
-err_ctx:
- i915_gem_context_unreference(ctx);
-err:
- kmem_cache_free(dev_priv->requests, req);
- return ret;
-}
-
-/**
- * i915_gem_request_alloc - allocate a request structure
- *
- * @engine: engine that we wish to issue the request on.
- * @ctx: context that the request will be associated with.
- * This can be NULL if the request is not directly related to
- * any specific user context, in which case this function will
- * choose an appropriate context to use.
- *
- * Returns a pointer to the allocated request if successful,
- * or an error code if not.
- */
-struct drm_i915_gem_request *
-i915_gem_request_alloc(struct intel_engine_cs *engine,
- struct i915_gem_context *ctx)
-{
- struct drm_i915_gem_request *req;
- int err;
-
- if (ctx == NULL)
- ctx = engine->i915->kernel_context;
- err = __i915_gem_request_alloc(engine, ctx, &req);
- return err ? ERR_PTR(err) : req;
-}
-
struct drm_i915_gem_request *
i915_gem_find_active_request(struct intel_engine_cs *engine)
{
@@ -3147,14 +2506,14 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
* implicit references on things like e.g. ppgtt address spaces through
* the request.
*/
- while (!list_empty(&engine->request_list)) {
+ if (!list_empty(&engine->request_list)) {
struct drm_i915_gem_request *request;
- request = list_first_entry(&engine->request_list,
- struct drm_i915_gem_request,
- list);
+ request = list_last_entry(&engine->request_list,
+ struct drm_i915_gem_request,
+ list);
- i915_gem_request_retire(request);
+ i915_gem_request_retire_upto(request);
}
/* Having flushed all requests from all queues, we know that all
@@ -3219,7 +2578,7 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
if (!i915_gem_request_completed(request))
break;
- i915_gem_request_retire(request);
+ i915_gem_request_retire_upto(request);
}
/* Move any buffers on the active list that are no longer referenced
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
new file mode 100644
index 000000000000..9e9aa6b725f7
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -0,0 +1,658 @@
+/*
+ * Copyright © 2008-2015 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#include "i915_drv.h"
+
+int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
+ struct drm_file *file)
+{
+ struct drm_i915_private *dev_private;
+ struct drm_i915_file_private *file_priv;
+
+ WARN_ON(!req || !file || req->file_priv);
+
+ if (!req || !file)
+ return -EINVAL;
+
+ if (req->file_priv)
+ return -EINVAL;
+
+ dev_private = req->i915;
+ file_priv = file->driver_priv;
+
+ spin_lock(&file_priv->mm.lock);
+ req->file_priv = file_priv;
+ list_add_tail(&req->client_list, &file_priv->mm.request_list);
+ spin_unlock(&file_priv->mm.lock);
+
+ req->pid = get_pid(task_pid(current));
+
+ return 0;
+}
+
+static inline void
+i915_gem_request_remove_from_client(struct drm_i915_gem_request *request)
+{
+ struct drm_i915_file_private *file_priv = request->file_priv;
+
+ if (!file_priv)
+ return;
+
+ spin_lock(&file_priv->mm.lock);
+ list_del(&request->client_list);
+ request->file_priv = NULL;
+ spin_unlock(&file_priv->mm.lock);
+
+ put_pid(request->pid);
+ request->pid = NULL;
+}
+
+static void i915_gem_request_retire(struct drm_i915_gem_request *request)
+{
+ trace_i915_gem_request_retire(request);
+ list_del_init(&request->list);
+
+ /* We know the GPU must have read the request to have
+ * sent us the seqno + interrupt, so use the position
+ * of tail of the request to update the last known position
+ * of the GPU head.
+ *
+ * Note this requires that we are always called in request
+ * completion order.
+ */
+ request->ringbuf->last_retired_head = request->postfix;
+
+ i915_gem_request_remove_from_client(request);
+
+ if (request->previous_context) {
+ if (i915.enable_execlists)
+ intel_lr_context_unpin(request->previous_context,
+ request->engine);
+ }
+
+ i915_gem_context_unreference(request->ctx);
+ i915_gem_request_unreference(request);
+}
+
+void i915_gem_request_retire_upto(struct drm_i915_gem_request *req)
+{
+ struct intel_engine_cs *engine = req->engine;
+ struct drm_i915_gem_request *tmp;
+
+ lockdep_assert_held(&req->i915->drm.struct_mutex);
+
+ if (list_empty(&req->list))
+ return;
+
+ do {
+ tmp = list_first_entry(&engine->request_list,
+ typeof(*tmp), list);
+
+ i915_gem_request_retire(tmp);
+ } while (tmp != req);
+
+ WARN_ON(i915_verify_lists(engine->dev));
+}
+
+static int i915_gem_check_wedge(unsigned int reset_counter, bool interruptible)
+{
+ if (__i915_terminally_wedged(reset_counter))
+ return -EIO;
+
+ if (__i915_reset_in_progress(reset_counter)) {
+ /* Non-interruptible callers can't handle -EAGAIN, hence return
+ * -EIO unconditionally for these.
+ */
+ if (!interruptible)
+ return -EIO;
+
+ return -EAGAIN;
+ }
+
+ return 0;
+}
+
+static int i915_gem_init_seqno(struct drm_i915_private *dev_priv, u32 seqno)
+{
+ struct intel_engine_cs *engine;
+ int ret;
+
+ /* Carefully retire all requests without writing to the rings */
+ for_each_engine(engine, dev_priv) {
+ ret = intel_engine_idle(engine);
+ if (ret)
+ return ret;
+ }
+ i915_gem_retire_requests(dev_priv);
+
+ /* If the seqno wraps around, we need to clear the breadcrumb rbtree */
+ if (!i915_seqno_passed(seqno, dev_priv->next_seqno)) {
+ while (intel_kick_waiters(dev_priv) ||
+ intel_kick_signalers(dev_priv))
+ yield();
+ }
+
+ /* Finally reset hw state */
+ for_each_engine(engine, dev_priv)
+ intel_ring_init_seqno(engine, seqno);
+
+ return 0;
+}
+
+int i915_gem_set_seqno(struct drm_device *dev, u32 seqno)
+{
+ struct drm_i915_private *dev_priv = to_i915(dev);
+ int ret;
+
+ if (seqno == 0)
+ return -EINVAL;
+
+ /* HWS page needs to be set less than what we
+ * will inject to ring
+ */
+ ret = i915_gem_init_seqno(dev_priv, seqno - 1);
+ if (ret)
+ return ret;
+
+ /* Carefully set the last_seqno value so that wrap
+ * detection still works
+ */
+ dev_priv->next_seqno = seqno;
+ dev_priv->last_seqno = seqno - 1;
+ if (dev_priv->last_seqno == 0)
+ dev_priv->last_seqno--;
+
+ return 0;
+}
+
+static int i915_gem_get_seqno(struct drm_i915_private *dev_priv, u32 *seqno)
+{
+ /* reserve 0 for non-seqno */
+ if (unlikely(dev_priv->next_seqno == 0)) {
+ int ret;
+
+ ret = i915_gem_init_seqno(dev_priv, 0);
+ if (ret)
+ return ret;
+
+ dev_priv->next_seqno = 1;
+ }
+
+ *seqno = dev_priv->last_seqno = dev_priv->next_seqno++;
+ return 0;
+}
+
+static inline int
+__i915_gem_request_alloc(struct intel_engine_cs *engine,
+ struct i915_gem_context *ctx,
+ struct drm_i915_gem_request **req_out)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+ unsigned int reset_counter = i915_reset_counter(&dev_priv->gpu_error);
+ struct drm_i915_gem_request *req;
+ int ret;
+
+ if (!req_out)
+ return -EINVAL;
+
+ *req_out = NULL;
+
+ /* ABI: Before userspace accesses the GPU (e.g. execbuffer), report
+ * EIO if the GPU is already wedged, or EAGAIN to drop the struct_mutex
+ * and restart.
+ */
+ ret = i915_gem_check_wedge(reset_counter, dev_priv->mm.interruptible);
+ if (ret)
+ return ret;
+
+ req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
+ if (!req)
+ return -ENOMEM;
+
+ ret = i915_gem_get_seqno(dev_priv, &req->seqno);
+ if (ret)
+ goto err;
+
+ kref_init(&req->ref);
+ req->i915 = dev_priv;
+ req->engine = engine;
+ req->ctx = ctx;
+ i915_gem_context_reference(ctx);
+
+ /*
+ * Reserve space in the ring buffer for all the commands required to
+ * eventually emit this request. This is to guarantee that the
+ * i915_add_request() call can't fail. Note that the reserve may need
+ * to be redone if the request is not actually submitted straight
+ * away, e.g. because a GPU scheduler has deferred it.
+ */
+ req->reserved_space = MIN_SPACE_FOR_ADD_REQUEST;
+
+ if (i915.enable_execlists)
+ ret = intel_logical_ring_alloc_request_extras(req);
+ else
+ ret = intel_ring_alloc_request_extras(req);
+ if (ret)
+ goto err_ctx;
+
+ *req_out = req;
+ return 0;
+
+err_ctx:
+ i915_gem_context_unreference(ctx);
+err:
+ kmem_cache_free(dev_priv->requests, req);
+ return ret;
+}
+
+/**
+ * i915_gem_request_alloc - allocate a request structure
+ *
+ * @engine: engine that we wish to issue the request on.
+ * @ctx: context that the request will be associated with.
+ * This can be NULL if the request is not directly related to
+ * any specific user context, in which case this function will
+ * choose an appropriate context to use.
+ *
+ * Returns a pointer to the allocated request if successful,
+ * or an error code if not.
+ */
+struct drm_i915_gem_request *
+i915_gem_request_alloc(struct intel_engine_cs *engine,
+ struct i915_gem_context *ctx)
+{
+ struct drm_i915_gem_request *req;
+ int err;
+
+ if (!ctx)
+ ctx = engine->i915->kernel_context;
+ err = __i915_gem_request_alloc(engine, ctx, &req);
+ return err ? ERR_PTR(err) : req;
+}
+
+static void i915_gem_mark_busy(const struct intel_engine_cs *engine)
+{
+ struct drm_i915_private *dev_priv = engine->i915;
+
+ dev_priv->gt.active_engines |= intel_engine_flag(engine);
+ if (dev_priv->gt.awake)
+ return;
+
+ intel_runtime_pm_get_noresume(dev_priv);
+ dev_priv->gt.awake = true;
+
+ intel_enable_gt_powersave(dev_priv);
+ i915_update_gfx_val(dev_priv);
+ if (INTEL_GEN(dev_priv) >= 6)
+ gen6_rps_busy(dev_priv);
+
+ queue_delayed_work(dev_priv->wq,
+ &dev_priv->gt.retire_work,
+ round_jiffies_up_relative(HZ));
+}
+
+/*
+ * NB: This function is not allowed to fail. Doing so would mean the the
+ * request is not being tracked for completion but the work itself is
+ * going to happen on the hardware. This would be a Bad Thing(tm).
+ */
+void __i915_add_request(struct drm_i915_gem_request *request,
+ struct drm_i915_gem_object *obj,
+ bool flush_caches)
+{
+ struct intel_engine_cs *engine;
+ struct intel_ringbuffer *ringbuf;
+ u32 request_start;
+ u32 reserved_tail;
+ int ret;
+
+ if (WARN_ON(!request))
+ return;
+
+ engine = request->engine;
+ ringbuf = request->ringbuf;
+
+ /*
+ * To ensure that this call will not fail, space for its emissions
+ * should already have been reserved in the ring buffer. Let the ring
+ * know that it is time to use that space up.
+ */
+ request_start = intel_ring_get_tail(ringbuf);
+ reserved_tail = request->reserved_space;
+ request->reserved_space = 0;
+
+ /*
+ * Emit any outstanding flushes - execbuf can fail to emit the flush
+ * after having emitted the batchbuffer command. Hence we need to fix
+ * things up similar to emitting the lazy request. The difference here
+ * is that the flush _must_ happen before the next request, no matter
+ * what.
+ */
+ if (flush_caches) {
+ if (i915.enable_execlists)
+ ret = logical_ring_flush_all_caches(request);
+ else
+ ret = intel_ring_flush_all_caches(request);
+ /* Not allowed to fail! */
+ WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
+ }
+
+ trace_i915_gem_request_add(request);
+
+ request->head = request_start;
+
+ /* Whilst this request exists, batch_obj will be on the
+ * active_list, and so will hold the active reference. Only when this
+ * request is retired will the the batch_obj be moved onto the
+ * inactive_list and lose its active reference. Hence we do not need
+ * to explicitly hold another reference here.
+ */
+ request->batch_obj = obj;
+
+ /* Seal the request and mark it as pending execution. Note that
+ * we may inspect this state, without holding any locks, during
+ * hangcheck. Hence we apply the barrier to ensure that we do not
+ * see a more recent value in the hws than we are tracking.
+ */
+ request->emitted_jiffies = jiffies;
+ request->previous_seqno = engine->last_submitted_seqno;
+ smp_store_mb(engine->last_submitted_seqno, request->seqno);
+ list_add_tail(&request->list, &engine->request_list);
+
+ /* Record the position of the start of the request so that
+ * should we detect the updated seqno part-way through the
+ * GPU processing the request, we never over-estimate the
+ * position of the head.
+ */
+ request->postfix = intel_ring_get_tail(ringbuf);
+
+ if (i915.enable_execlists) {
+ ret = engine->emit_request(request);
+ } else {
+ ret = engine->add_request(request);
+
+ request->tail = intel_ring_get_tail(ringbuf);
+ }
+ /* Not allowed to fail! */
+ WARN(ret, "emit|add_request failed: %d!\n", ret);
+ /* Sanity check that the reserved size was large enough. */
+ ret = intel_ring_get_tail(ringbuf) - request_start;
+ if (ret < 0)
+ ret += ringbuf->size;
+ WARN_ONCE(ret > reserved_tail,
+ "Not enough space reserved (%d bytes) "
+ "for adding the request (%d bytes)\n",
+ reserved_tail, ret);
+
+ i915_gem_mark_busy(engine);
+}
+
+static unsigned long local_clock_us(unsigned int *cpu)
+{
+ unsigned long t;
+
+ /* Cheaply and approximately convert from nanoseconds to microseconds.
+ * The result and subsequent calculations are also defined in the same
+ * approximate microseconds units. The principal source of timing
+ * error here is from the simple truncation.
+ *
+ * Note that local_clock() is only defined wrt to the current CPU;
+ * the comparisons are no longer valid if we switch CPUs. Instead of
+ * blocking preemption for the entire busywait, we can detect the CPU
+ * switch and use that as indicator of system load and a reason to
+ * stop busywaiting, see busywait_stop().
+ */
+ *cpu = get_cpu();
+ t = local_clock() >> 10;
+ put_cpu();
+
+ return t;
+}
+
+static bool busywait_stop(unsigned long timeout, unsigned int cpu)
+{
+ unsigned int this_cpu;
+
+ if (time_after(local_clock_us(&this_cpu), timeout))
+ return true;
+
+ return this_cpu != cpu;
+}
+
+bool __i915_spin_request(const struct drm_i915_gem_request *req,
+ int state, unsigned long timeout_us)
+{
+ unsigned int cpu;
+
+ /* When waiting for high frequency requests, e.g. during synchronous
+ * rendering split between the CPU and GPU, the finite amount of time
+ * required to set up the irq and wait upon it limits the response
+ * rate. By busywaiting on the request completion for a short while we
+ * can service the high frequency waits as quick as possible. However,
+ * if it is a slow request, we want to sleep as quickly as possible.
+ * The tradeoff between waiting and sleeping is roughly the time it
+ * takes to sleep on a request, on the order of a microsecond.
+ */
+
+ timeout_us += local_clock_us(&cpu);
+ do {
+ if (i915_gem_request_completed(req))
+ return true;
+
+ if (signal_pending_state(state, current))
+ break;
+
+ if (busywait_stop(timeout_us, cpu))
+ break;
+
+ cpu_relax_lowlatency();
+ } while (!need_resched());
+
+ return false;
+}
+
+/**
+ * __i915_wait_request - wait until execution of request has finished
+ * @req: duh!
+ * @interruptible: do an interruptible wait (normally yes)
+ * @timeout: in - how long to wait (NULL forever); out - how much time remaining
+ * @rps: client to charge for RPS boosting
+ *
+ * Note: It is of utmost importance that the passed in seqno and reset_counter
+ * values have been read by the caller in an smp safe manner. Where read-side
+ * locks are involved, it is sufficient to read the reset_counter before
+ * unlocking the lock that protects the seqno. For lockless tricks, the
+ * reset_counter _must_ be read before, and an appropriate smp_rmb must be
+ * inserted.
+ *
+ * Returns 0 if the request was found within the alloted time. Else returns the
+ * errno with remaining time filled in timeout argument.
+ */
+int __i915_wait_request(struct drm_i915_gem_request *req,
+ bool interruptible,
+ s64 *timeout,
+ struct intel_rps_client *rps)
+{
+ int state = interruptible ? TASK_INTERRUPTIBLE : TASK_UNINTERRUPTIBLE;
+ DEFINE_WAIT(reset);
+ struct intel_wait wait;
+ unsigned long timeout_remain;
+ int ret = 0;
+
+ might_sleep();
+
+ if (list_empty(&req->list))
+ return 0;
+
+ if (i915_gem_request_completed(req))
+ return 0;
+
+ timeout_remain = MAX_SCHEDULE_TIMEOUT;
+ if (timeout) {
+ if (WARN_ON(*timeout < 0))
+ return -EINVAL;
+
+ if (*timeout == 0)
+ return -ETIME;
+
+ /* Record current time in case interrupted, or wedged */
+ timeout_remain = nsecs_to_jiffies_timeout(*timeout);
+ *timeout += ktime_get_raw_ns();
+ }
+
+ trace_i915_gem_request_wait_begin(req);
+
+ /* This client is about to stall waiting for the GPU. In many cases
+ * this is undesirable and limits the throughput of the system, as
+ * many clients cannot continue processing user input/output whilst
+ * blocked. RPS autotuning may take tens of milliseconds to respond
+ * to the GPU load and thus incurs additional latency for the client.
+ * We can circumvent that by promoting the GPU frequency to maximum
+ * before we wait. This makes the GPU throttle up much more quickly
+ * (good for benchmarks and user experience, e.g. window animations),
+ * but at a cost of spending more power processing the workload
+ * (bad for battery). Not all clients even want their results
+ * immediately and for them we should just let the GPU select its own
+ * frequency to maximise efficiency. To prevent a single client from
+ * forcing the clocks too high for the whole system, we only allow
+ * each client to waitboost once in a busy period.
+ */
+ if (INTEL_GEN(req->i915) >= 6)
+ gen6_rps_boost(req->i915, rps, req->emitted_jiffies);
+
+ /* Optimistic spin for the next ~jiffie before touching IRQs */
+ if (i915_spin_request(req, state, 5))
+ goto complete;
+
+ set_current_state(state);
+ add_wait_queue(&req->i915->gpu_error.wait_queue, &reset);
+
+ intel_wait_init(&wait, req->seqno);
+ if (intel_engine_add_wait(req->engine, &wait))
+ /* In order to check that we haven't missed the interrupt
+ * as we enabled it, we need to kick ourselves to do a
+ * coherent check on the seqno before we sleep.
+ */
+ goto wakeup;
+
+ for (;;) {
+ if (signal_pending_state(state, current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
+
+ timeout_remain = io_schedule_timeout(timeout_remain);
+ if (timeout_remain == 0) {
+ ret = -ETIME;
+ break;
+ }
+
+ if (intel_wait_complete(&wait))
+ break;
+
+ set_current_state(state);
+
+wakeup:
+ /* Carefully check if the request is complete, giving time
+ * for the seqno to be visible following the interrupt.
+ * We also have to check in case we are kicked by the GPU
+ * reset in order to drop the struct_mutex.
+ */
+ if (__i915_request_irq_complete(req))
+ break;
+
+ /* Only spin if we know the GPU is processing this request */
+ if (i915_spin_request(req, state, 2))
+ break;
+ }
+ remove_wait_queue(&req->i915->gpu_error.wait_queue, &reset);
+
+ intel_engine_remove_wait(req->engine, &wait);
+ __set_current_state(TASK_RUNNING);
+complete:
+ trace_i915_gem_request_wait_end(req);
+
+ if (timeout) {
+ *timeout -= ktime_get_raw_ns();
+ if (*timeout < 0)
+ *timeout = 0;
+
+ /*
+ * Apparently ktime isn't accurate enough and occasionally has a
+ * bit of mismatch in the jiffies<->nsecs<->ktime loop. So patch
+ * things up to make the test happy. We allow up to 1 jiffy.
+ *
+ * This is a regrssion from the timespec->ktime conversion.
+ */
+ if (ret == -ETIME && *timeout < jiffies_to_usecs(1)*1000)
+ *timeout = 0;
+ }
+
+ if (rps && req->seqno == req->engine->last_submitted_seqno) {
+ /* The GPU is now idle and this client has stalled.
+ * Since no other client has submitted a request in the
+ * meantime, assume that this client is the only one
+ * supplying work to the GPU but is unable to keep that
+ * work supplied because it is waiting. Since the GPU is
+ * then never kept fully busy, RPS autoclocking will
+ * keep the clocks relatively low, causing further delays.
+ * Compensate by giving the synchronous client credit for
+ * a waitboost next time.
+ */
+ spin_lock(&req->i915->rps.client_lock);
+ list_del_init(&rps->link);
+ spin_unlock(&req->i915->rps.client_lock);
+ }
+
+ return ret;
+}
+
+/**
+ * Waits for a request to be signaled, and cleans up the
+ * request and object lists appropriately for that event.
+ */
+int i915_wait_request(struct drm_i915_gem_request *req)
+{
+ int ret;
+
+ GEM_BUG_ON(!req);
+ lockdep_assert_held(&req->i915->drm.struct_mutex);
+
+ ret = __i915_wait_request(req, req->i915->mm.interruptible, NULL, NULL);
+ if (ret)
+ return ret;
+
+ /* If the GPU hung, we want to keep the requests to find the guilty. */
+ if (!i915_reset_in_progress(&req->i915->gpu_error))
+ i915_gem_request_retire_upto(req);
+
+ return 0;
+}
+
+void i915_gem_request_free(struct kref *req_ref)
+{
+ struct drm_i915_gem_request *req =
+ container_of(req_ref, typeof(*req), ref);
+ kmem_cache_free(req->i915->requests, req);
+}
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
new file mode 100644
index 000000000000..ea700befcc28
--- /dev/null
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -0,0 +1,238 @@
+/*
+ * Copyright © 2008-2015 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ */
+
+#ifndef I915_GEM_REQUEST_H
+#define I915_GEM_REQUEST_H
+
+/**
+ * Request queue structure.
+ *
+ * The request queue allows us to note sequence numbers that have been emitted
+ * and may be associated with active buffers to be retired.
+ *
+ * By keeping this list, we can avoid having to do questionable sequence
+ * number comparisons on buffer last_read|write_seqno. It also allows an
+ * emission time to be associated with the request for tracking how far ahead
+ * of the GPU the submission is.
+ *
+ * The requests are reference counted, so upon creation they should have an
+ * initial reference taken using kref_init
+ */
+struct drm_i915_gem_request {
+ struct kref ref;
+
+ /** On Which ring this request was generated */
+ struct drm_i915_private *i915;
+
+ /**
+ * Context and ring buffer related to this request
+ * Contexts are refcounted, so when this request is associated with a
+ * context, we must increment the context's refcount, to guarantee that
+ * it persists while any request is linked to it. Requests themselves
+ * are also refcounted, so the request will only be freed when the last
+ * reference to it is dismissed, and the code in
+ * i915_gem_request_free() will then decrement the refcount on the
+ * context.
+ */
+ struct i915_gem_context *ctx;
+ struct intel_engine_cs *engine;
+ struct intel_ringbuffer *ringbuf;
+ struct intel_signal_node signaling;
+
+ /** GEM sequence number associated with the previous request,
+ * when the HWS breadcrumb is equal to this the GPU is processing
+ * this request.
+ */
+ u32 previous_seqno;
+
+ /** GEM sequence number associated with this request,
+ * when the HWS breadcrumb is equal or greater than this the GPU
+ * has finished processing this request.
+ */
+ u32 seqno;
+
+ /** Position in the ringbuffer of the start of the request */
+ u32 head;
+
+ /**
+ * Position in the ringbuffer of the start of the postfix.
+ * This is required to calculate the maximum available ringbuffer
+ * space without overwriting the postfix.
+ */
+ u32 postfix;
+
+ /** Position in the ringbuffer of the end of the whole request */
+ u32 tail;
+
+ /** Preallocate space in the ringbuffer for the emitting the request */
+ u32 reserved_space;
+
+ /**
+ * Context related to the previous request.
+ * As the contexts are accessed by the hardware until the switch is
+ * completed to a new context, the hardware may still be writing
+ * to the context object after the breadcrumb is visible. We must
+ * not unpin/unbind/prune that object whilst still active and so
+ * we keep the previous context pinned until the following (this)
+ * request is retired.
+ */
+ struct i915_gem_context *previous_context;
+
+ /** Batch buffer related to this request if any (used for
+ * error state dump only).
+ */
+ struct drm_i915_gem_object *batch_obj;
+
+ /** Time at which this request was emitted, in jiffies. */
+ unsigned long emitted_jiffies;
+
+ /** global list entry for this request */
+ struct list_head list;
+
+ struct drm_i915_file_private *file_priv;
+ /** file_priv list entry for this request */
+ struct list_head client_list;
+
+ /** process identifier submitting this request */
+ struct pid *pid;
+
+ /**
+ * The ELSP only accepts two elements at a time, so we queue
+ * context/tail pairs on a given queue (ring->execlist_queue) until the
+ * hardware is available. The queue serves a double purpose: we also use
+ * it to keep track of the up to 2 contexts currently in the hardware
+ * (usually one in execution and the other queued up by the GPU): We
+ * only remove elements from the head of the queue when the hardware
+ * informs us that an element has been completed.
+ *
+ * All accesses to the queue are mediated by a spinlock
+ * (ring->execlist_lock).
+ */
+
+ /** Execlist link in the submission queue.*/
+ struct list_head execlist_link;
+
+ /** Execlists no. of times this request has been sent to the ELSP */
+ int elsp_submitted;
+
+ /** Execlists context hardware id. */
+ unsigned int ctx_hw_id;
+};
+
+struct drm_i915_gem_request * __must_check
+i915_gem_request_alloc(struct intel_engine_cs *engine,
+ struct i915_gem_context *ctx);
+void i915_gem_request_free(struct kref *req_ref);
+int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
+ struct drm_file *file);
+void i915_gem_request_retire_upto(struct drm_i915_gem_request *req);
+
+static inline u32
+i915_gem_request_get_seqno(struct drm_i915_gem_request *req)
+{
+ return req ? req->seqno : 0;
+}
+
+static inline struct intel_engine_cs *
+i915_gem_request_get_engine(struct drm_i915_gem_request *req)
+{
+ return req ? req->engine : NULL;
+}
+
+static inline struct drm_i915_gem_request *
+i915_gem_request_reference(struct drm_i915_gem_request *req)
+{
+ if (req)
+ kref_get(&req->ref);
+ return req;
+}
+
+static inline void
+i915_gem_request_unreference(struct drm_i915_gem_request *req)
+{
+ kref_put(&req->ref, i915_gem_request_free);
+}
+
+static inline void i915_gem_request_assign(struct drm_i915_gem_request **pdst,
+ struct drm_i915_gem_request *src)
+{
+ if (src)
+ i915_gem_request_reference(src);
+
+ if (*pdst)
+ i915_gem_request_unreference(*pdst);
+
+ *pdst = src;
+}
+
+void __i915_add_request(struct drm_i915_gem_request *req,
+ struct drm_i915_gem_object *batch_obj,
+ bool flush_caches);
+#define i915_add_request(req) \
+ __i915_add_request(req, NULL, true)
+#define i915_add_request_no_flush(req) \
+ __i915_add_request(req, NULL, false)
+
+struct intel_rps_client;
+
+int __i915_wait_request(struct drm_i915_gem_request *req,
+ bool interruptible,
+ s64 *timeout,
+ struct intel_rps_client *rps);
+int __must_check i915_wait_request(struct drm_i915_gem_request *req);
+
+static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine);
+
+/**
+ * Returns true if seq1 is later than seq2.
+ */
+static inline bool i915_seqno_passed(u32 seq1, u32 seq2)
+{
+ return (s32)(seq1 - seq2) >= 0;
+}
+
+static inline bool
+i915_gem_request_started(const struct drm_i915_gem_request *req)
+{
+ return i915_seqno_passed(intel_engine_get_seqno(req->engine),
+ req->previous_seqno);
+}
+
+static inline bool
+i915_gem_request_completed(const struct drm_i915_gem_request *req)
+{
+ return i915_seqno_passed(intel_engine_get_seqno(req->engine),
+ req->seqno);
+}
+
+bool __i915_spin_request(const struct drm_i915_gem_request *request,
+ int state, unsigned long timeout_us);
+static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
+ int state, unsigned long timeout_us)
+{
+ return (i915_gem_request_started(request) &&
+ __i915_spin_request(request, state, timeout_us));
+}
+
+#endif /* I915_GEM_REQUEST_H */
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (5 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 07/64] drm/i915: Move GEM request routines to i915_gem_request.c Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 9:41 ` Tvrtko Ursulin
2016-07-08 9:21 ` Mika Kuoppala
2016-07-07 8:41 ` [PATCH 09/64] drm/i915: Mark all current requests as complete before resetting them Chris Wilson
` (56 subsequent siblings)
63 siblings, 2 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
In order to keep the memory allocated for requests reasonably tight, try
to reuse the oldest request (so long as it is completed and has no
external references) for the next allocation.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_request.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 9e9aa6b725f7..ee1189c35509 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -226,6 +226,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
if (ret)
return ret;
+ if (!list_empty(&engine->request_list)) {
+ req = list_first_entry(&engine->request_list,
+ typeof(*req), list);
+ if (i915_gem_request_completed(req))
+ i915_gem_request_retire(req);
+ }
+
req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
if (!req)
return -ENOMEM;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 09/64] drm/i915: Mark all current requests as complete before resetting them
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (6 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 16:09 ` Mika Kuoppala
2016-07-07 8:41 ` [PATCH 10/64] drm/i915: Derive GEM requests from dma-fence Chris Wilson
` (55 subsequent siblings)
63 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Following a GPU reset upon hang, we retire all the requests and then
mark them all as complete. If we mark them as complete first, we both
keep the normal retirement order (completed first then retired) and
provide a small optimisation for concurrent lookups.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 14cc87b916c6..1c05e4573173 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2486,6 +2486,12 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
i915_gem_object_retire__read(obj, engine->id);
}
+ /* Mark all pending requests as complete so that any concurrent
+ * (lockless) lookup doesn't try and wait upon the request as we
+ * reset it.
+ */
+ intel_ring_init_seqno(engine, engine->last_submitted_seqno);
+
/*
* Clear the execlists queue up before freeing the requests, as those
* are the ones that keep the context and ringbuffer backing objects
@@ -2527,8 +2533,6 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
buffer->last_retired_head = buffer->tail;
intel_ring_update_space(buffer);
}
-
- intel_ring_init_seqno(engine, engine->last_submitted_seqno);
}
void i915_gem_reset(struct drm_device *dev)
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 10/64] drm/i915: Derive GEM requests from dma-fence
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (7 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 09/64] drm/i915: Mark all current requests as complete before resetting them Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 11/64] drm/i915: Disable waitboosting for fence_wait() Chris Wilson
` (54 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx; +Cc: Daniel Vetter, Jesse Barnes
dma-buf provides a generic fence class for interoperation between
drivers. Internally we use the request structure as a fence, and so with
only a little bit of interfacing we can rebase those requests on top of
dma-buf fences. This will allow us, in the future, to pass those fences
back to userspace or between drivers.
v2: The fence_context needs to be globally unique, not just unique to
this device.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---
drivers/gpu/drm/i915/i915_debugfs.c | 2 +-
drivers/gpu/drm/i915/i915_gem_request.c | 113 ++++++++++++++++++++++++++---
drivers/gpu/drm/i915/i915_gem_request.h | 43 +++++++----
drivers/gpu/drm/i915/i915_gpu_error.c | 2 +-
drivers/gpu/drm/i915/i915_guc_submission.c | 4 +-
drivers/gpu/drm/i915/i915_trace.h | 10 +--
drivers/gpu/drm/i915/intel_breadcrumbs.c | 7 +-
drivers/gpu/drm/i915/intel_lrc.c | 3 +-
drivers/gpu/drm/i915/intel_ringbuffer.c | 11 +--
drivers/gpu/drm/i915/intel_ringbuffer.h | 1 +
10 files changed, 150 insertions(+), 46 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index e686704a310a..87f259a09f21 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -768,7 +768,7 @@ static int i915_gem_request_info(struct seq_file *m, void *data)
if (req->pid)
task = pid_task(req->pid, PIDTYPE_PID);
seq_printf(m, " %x @ %d: %s [%d]\n",
- req->seqno,
+ req->fence.seqno,
(int) (jiffies - req->emitted_jiffies),
task ? task->comm : "<unknown>",
task ? task->pid : -1);
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index ee1189c35509..4184b5879aea 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -24,6 +24,95 @@
#include "i915_drv.h"
+static const char *i915_fence_get_driver_name(struct fence *fence)
+{
+ return "i915";
+}
+
+static const char *i915_fence_get_timeline_name(struct fence *fence)
+{
+ /* Timelines are bound by eviction to a VM. However, since
+ * we only have a global seqno at the moment, we only have
+ * a single timeline. Note that each timeline will have
+ * multiple execution contexts (fence contexts) as we allow
+ * engines within a single timeline to execute in parallel.
+ */
+ return "global";
+}
+
+static bool i915_fence_signaled(struct fence *fence)
+{
+ return i915_gem_request_completed(to_request(fence));
+}
+
+static bool i915_fence_enable_signaling(struct fence *fence)
+{
+ if (i915_fence_signaled(fence))
+ return false;
+
+ intel_engine_enable_signaling(to_request(fence));
+ return true;
+}
+
+static signed long i915_fence_wait(struct fence *fence,
+ bool interruptible,
+ signed long timeout_jiffies)
+{
+ s64 timeout_ns, *timeout;
+ int ret;
+
+ if (timeout_jiffies != MAX_SCHEDULE_TIMEOUT) {
+ timeout_ns = jiffies_to_nsecs(timeout_jiffies);
+ timeout = &timeout_ns;
+ } else {
+ timeout = NULL;
+ }
+
+ ret = __i915_wait_request(to_request(fence),
+ interruptible, timeout,
+ NULL);
+ if (ret == -ETIME)
+ return 0;
+
+ if (ret < 0)
+ return ret;
+
+ if (timeout_jiffies != MAX_SCHEDULE_TIMEOUT)
+ timeout_jiffies = nsecs_to_jiffies(timeout_ns);
+
+ return timeout_jiffies;
+}
+
+static void i915_fence_value_str(struct fence *fence, char *str, int size)
+{
+ snprintf(str, size, "%u", fence->seqno);
+}
+
+static void i915_fence_timeline_value_str(struct fence *fence, char *str,
+ int size)
+{
+ snprintf(str, size, "%u",
+ intel_engine_get_seqno(to_request(fence)->engine));
+}
+
+static void i915_fence_release(struct fence *fence)
+{
+ struct drm_i915_gem_request *req = to_request(fence);
+
+ kmem_cache_free(req->i915->requests, req);
+}
+
+const struct fence_ops i915_fence_ops = {
+ .get_driver_name = i915_fence_get_driver_name,
+ .get_timeline_name = i915_fence_get_timeline_name,
+ .enable_signaling = i915_fence_enable_signaling,
+ .signaled = i915_fence_signaled,
+ .wait = i915_fence_wait,
+ .release = i915_fence_release,
+ .fence_value_str = i915_fence_value_str,
+ .timeline_value_str = i915_fence_timeline_value_str,
+};
+
int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
struct drm_file *file)
{
@@ -211,6 +300,7 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
struct drm_i915_private *dev_priv = engine->i915;
unsigned int reset_counter = i915_reset_counter(&dev_priv->gpu_error);
struct drm_i915_gem_request *req;
+ u32 seqno;
int ret;
if (!req_out)
@@ -237,11 +327,17 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
if (!req)
return -ENOMEM;
- ret = i915_gem_get_seqno(dev_priv, &req->seqno);
+ ret = i915_gem_get_seqno(dev_priv, &seqno);
if (ret)
goto err;
- kref_init(&req->ref);
+ spin_lock_init(&req->lock);
+ fence_init(&req->fence,
+ &i915_fence_ops,
+ &req->lock,
+ engine->fence_context,
+ seqno);
+
req->i915 = dev_priv;
req->engine = engine;
req->ctx = ctx;
@@ -384,7 +480,7 @@ void __i915_add_request(struct drm_i915_gem_request *request,
*/
request->emitted_jiffies = jiffies;
request->previous_seqno = engine->last_submitted_seqno;
- smp_store_mb(engine->last_submitted_seqno, request->seqno);
+ smp_store_mb(engine->last_submitted_seqno, request->fence.seqno);
list_add_tail(&request->list, &engine->request_list);
/* Record the position of the start of the request so that
@@ -555,7 +651,7 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
set_current_state(state);
add_wait_queue(&req->i915->gpu_error.wait_queue, &reset);
- intel_wait_init(&wait, req->seqno);
+ intel_wait_init(&wait, req->fence.seqno);
if (intel_engine_add_wait(req->engine, &wait))
/* In order to check that we haven't missed the interrupt
* as we enabled it, we need to kick ourselves to do a
@@ -616,7 +712,7 @@ complete:
*timeout = 0;
}
- if (rps && req->seqno == req->engine->last_submitted_seqno) {
+ if (rps && req->fence.seqno == req->engine->last_submitted_seqno) {
/* The GPU is now idle and this client has stalled.
* Since no other client has submitted a request in the
* meantime, assume that this client is the only one
@@ -656,10 +752,3 @@ int i915_wait_request(struct drm_i915_gem_request *req)
return 0;
}
-
-void i915_gem_request_free(struct kref *req_ref)
-{
- struct drm_i915_gem_request *req =
- container_of(req_ref, typeof(*req), ref);
- kmem_cache_free(req->i915->requests, req);
-}
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index ea700befcc28..6f2c820785f3 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -25,6 +25,10 @@
#ifndef I915_GEM_REQUEST_H
#define I915_GEM_REQUEST_H
+#include <linux/fence.h>
+
+#include "i915_gem.h"
+
/**
* Request queue structure.
*
@@ -36,11 +40,11 @@
* emission time to be associated with the request for tracking how far ahead
* of the GPU the submission is.
*
- * The requests are reference counted, so upon creation they should have an
- * initial reference taken using kref_init
+ * The requests are reference counted.
*/
struct drm_i915_gem_request {
- struct kref ref;
+ struct fence fence;
+ spinlock_t lock;
/** On Which ring this request was generated */
struct drm_i915_private *i915;
@@ -66,12 +70,6 @@ struct drm_i915_gem_request {
*/
u32 previous_seqno;
- /** GEM sequence number associated with this request,
- * when the HWS breadcrumb is equal or greater than this the GPU
- * has finished processing this request.
- */
- u32 seqno;
-
/** Position in the ringbuffer of the start of the request */
u32 head;
@@ -140,10 +138,16 @@ struct drm_i915_gem_request {
unsigned int ctx_hw_id;
};
+extern const struct fence_ops i915_fence_ops;
+
+static inline bool fence_is_i915(struct fence *fence)
+{
+ return fence->ops == &i915_fence_ops;
+}
+
struct drm_i915_gem_request * __must_check
i915_gem_request_alloc(struct intel_engine_cs *engine,
struct i915_gem_context *ctx);
-void i915_gem_request_free(struct kref *req_ref);
int i915_gem_request_add_to_client(struct drm_i915_gem_request *req,
struct drm_file *file);
void i915_gem_request_retire_upto(struct drm_i915_gem_request *req);
@@ -151,7 +155,7 @@ void i915_gem_request_retire_upto(struct drm_i915_gem_request *req);
static inline u32
i915_gem_request_get_seqno(struct drm_i915_gem_request *req)
{
- return req ? req->seqno : 0;
+ return req ? req->fence.seqno : 0;
}
static inline struct intel_engine_cs *
@@ -161,17 +165,24 @@ i915_gem_request_get_engine(struct drm_i915_gem_request *req)
}
static inline struct drm_i915_gem_request *
+to_request(struct fence *fence)
+{
+ /* We assume that NULL fence/request are interoperable */
+ BUILD_BUG_ON(offsetof(struct drm_i915_gem_request, fence) != 0);
+ GEM_BUG_ON(fence && !fence_is_i915(fence));
+ return container_of(fence, struct drm_i915_gem_request, fence);
+}
+
+static inline struct drm_i915_gem_request *
i915_gem_request_reference(struct drm_i915_gem_request *req)
{
- if (req)
- kref_get(&req->ref);
- return req;
+ return to_request(fence_get(&req->fence));
}
static inline void
i915_gem_request_unreference(struct drm_i915_gem_request *req)
{
- kref_put(&req->ref, i915_gem_request_free);
+ fence_put(&req->fence);
}
static inline void i915_gem_request_assign(struct drm_i915_gem_request **pdst,
@@ -223,7 +234,7 @@ static inline bool
i915_gem_request_completed(const struct drm_i915_gem_request *req)
{
return i915_seqno_passed(intel_engine_get_seqno(req->engine),
- req->seqno);
+ req->fence.seqno);
}
bool __i915_spin_request(const struct drm_i915_gem_request *request,
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 9d73d2216adc..6daaf4ecd2da 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -1182,7 +1182,7 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
}
erq = &error->ring[i].requests[count++];
- erq->seqno = request->seqno;
+ erq->seqno = request->fence.seqno;
erq->jiffies = request->emitted_jiffies;
erq->tail = request->postfix;
}
diff --git a/drivers/gpu/drm/i915/i915_guc_submission.c b/drivers/gpu/drm/i915/i915_guc_submission.c
index 2112e029db6a..1cc5de129e76 100644
--- a/drivers/gpu/drm/i915/i915_guc_submission.c
+++ b/drivers/gpu/drm/i915/i915_guc_submission.c
@@ -506,7 +506,7 @@ static void guc_add_workqueue_item(struct i915_guc_client *gc,
rq->engine);
wqi->ring_tail = tail << WQ_RING_TAIL_SHIFT;
- wqi->fence_id = rq->seqno;
+ wqi->fence_id = rq->fence.seqno;
kunmap_atomic(base);
}
@@ -601,7 +601,7 @@ int i915_guc_submit(struct drm_i915_gem_request *rq)
client->b_fail += 1;
guc->submissions[engine_id] += 1;
- guc->last_seqno[engine_id] = rq->seqno;
+ guc->last_seqno[engine_id] = rq->fence.seqno;
return b_ret;
}
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index 534154e05fbe..007112d1e049 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -465,7 +465,7 @@ TRACE_EVENT(i915_gem_ring_sync_to,
__entry->dev = from->i915->drm.primary->index;
__entry->sync_from = from->id;
__entry->sync_to = to_req->engine->id;
- __entry->seqno = i915_gem_request_get_seqno(req);
+ __entry->seqno = req->fence.seqno;
),
TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u",
@@ -488,9 +488,9 @@ TRACE_EVENT(i915_gem_ring_dispatch,
TP_fast_assign(
__entry->dev = req->i915->drm.primary->index;
__entry->ring = req->engine->id;
- __entry->seqno = req->seqno;
+ __entry->seqno = req->fence.seqno;
__entry->flags = flags;
- intel_engine_enable_signaling(req);
+ fence_enable_sw_signaling(&req->fence);
),
TP_printk("dev=%u, ring=%u, seqno=%u, flags=%x",
@@ -533,7 +533,7 @@ DECLARE_EVENT_CLASS(i915_gem_request,
TP_fast_assign(
__entry->dev = req->i915->drm.primary->index;
__entry->ring = req->engine->id;
- __entry->seqno = req->seqno;
+ __entry->seqno = req->fence.seqno;
),
TP_printk("dev=%u, ring=%u, seqno=%u",
@@ -595,7 +595,7 @@ TRACE_EVENT(i915_gem_request_wait_begin,
TP_fast_assign(
__entry->dev = req->i915->drm.primary->index;
__entry->ring = req->engine->id;
- __entry->seqno = req->seqno;
+ __entry->seqno = req->fence.seqno;
__entry->blocking =
mutex_is_locked(&req->i915->drm.struct_mutex);
),
diff --git a/drivers/gpu/drm/i915/intel_breadcrumbs.c b/drivers/gpu/drm/i915/intel_breadcrumbs.c
index b074f3d6d127..32ada41dedfb 100644
--- a/drivers/gpu/drm/i915/intel_breadcrumbs.c
+++ b/drivers/gpu/drm/i915/intel_breadcrumbs.c
@@ -436,6 +436,7 @@ static int intel_breadcrumbs_signaler(void *arg)
*/
intel_engine_remove_wait(engine,
&request->signaling.wait);
+ fence_signal(&request->fence);
/* Find the next oldest signal. Note that as we have
* not been holding the lock, another client may
@@ -482,7 +483,7 @@ void intel_engine_enable_signaling(struct drm_i915_gem_request *request)
}
request->signaling.wait.tsk = b->signaler;
- request->signaling.wait.seqno = request->seqno;
+ request->signaling.wait.seqno = request->fence.seqno;
i915_gem_request_reference(request);
/* First add ourselves into the list of waiters, but register our
@@ -504,8 +505,8 @@ void intel_engine_enable_signaling(struct drm_i915_gem_request *request)
p = &b->signals.rb_node;
while (*p) {
parent = *p;
- if (i915_seqno_passed(request->seqno,
- to_signaler(parent)->seqno)) {
+ if (i915_seqno_passed(request->fence.seqno,
+ to_signaler(parent)->fence.seqno)) {
p = &parent->rb_right;
first = false;
} else {
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index d47d1a0dbb91..b22931890a4c 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1802,7 +1802,7 @@ static int gen8_emit_request(struct drm_i915_gem_request *request)
intel_hws_seqno_address(request->engine) |
MI_FLUSH_DW_USE_GTT);
intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, request->seqno);
+ intel_logical_ring_emit(ringbuf, request->fence.seqno);
intel_logical_ring_emit(ringbuf, MI_USER_INTERRUPT);
intel_logical_ring_emit(ringbuf, MI_NOOP);
return intel_logical_ring_advance_and_submit(request);
@@ -2120,6 +2120,7 @@ logical_ring_setup(struct drm_i915_private *dev_priv, enum intel_engine_id id)
engine->exec_id = info->exec_id;
engine->guc_id = info->guc_id;
engine->mmio_base = info->mmio_base;
+ engine->fence_context = fence_context_alloc(1);
engine->i915 = dev_priv;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 61e00bf9e87f..29da7a7aff86 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1352,7 +1352,7 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
PIPE_CONTROL_CS_STALL);
intel_ring_emit(signaller, lower_32_bits(gtt_offset));
intel_ring_emit(signaller, upper_32_bits(gtt_offset));
- intel_ring_emit(signaller, signaller_req->seqno);
+ intel_ring_emit(signaller, signaller_req->fence.seqno);
intel_ring_emit(signaller, 0);
intel_ring_emit(signaller, MI_SEMAPHORE_SIGNAL |
MI_SEMAPHORE_TARGET(waiter->hw_id));
@@ -1390,7 +1390,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
intel_ring_emit(signaller, lower_32_bits(gtt_offset) |
MI_FLUSH_DW_USE_GTT);
intel_ring_emit(signaller, upper_32_bits(gtt_offset));
- intel_ring_emit(signaller, signaller_req->seqno);
+ intel_ring_emit(signaller, signaller_req->fence.seqno);
intel_ring_emit(signaller, MI_SEMAPHORE_SIGNAL |
MI_SEMAPHORE_TARGET(waiter->hw_id));
intel_ring_emit(signaller, 0);
@@ -1423,7 +1423,7 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
if (i915_mmio_reg_valid(mbox_reg)) {
intel_ring_emit(signaller, MI_LOAD_REGISTER_IMM(1));
intel_ring_emit_reg(signaller, mbox_reg);
- intel_ring_emit(signaller, signaller_req->seqno);
+ intel_ring_emit(signaller, signaller_req->fence.seqno);
}
}
@@ -1459,7 +1459,7 @@ gen6_add_request(struct drm_i915_gem_request *req)
intel_ring_emit(engine, MI_STORE_DWORD_INDEX);
intel_ring_emit(engine,
I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
- intel_ring_emit(engine, req->seqno);
+ intel_ring_emit(engine, req->fence.seqno);
intel_ring_emit(engine, MI_USER_INTERRUPT);
__intel_ring_advance(engine);
@@ -1708,7 +1708,7 @@ i9xx_add_request(struct drm_i915_gem_request *req)
intel_ring_emit(engine, MI_STORE_DWORD_INDEX);
intel_ring_emit(engine,
I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
- intel_ring_emit(engine, req->seqno);
+ intel_ring_emit(engine, req->fence.seqno);
intel_ring_emit(engine, MI_USER_INTERRUPT);
__intel_ring_advance(engine);
@@ -2177,6 +2177,7 @@ static int intel_init_ring_buffer(struct drm_device *dev,
WARN_ON(engine->buffer);
engine->i915 = dev_priv;
+ engine->fence_context = fence_context_alloc(1);
INIT_LIST_HEAD(&engine->active_list);
INIT_LIST_HEAD(&engine->request_list);
INIT_LIST_HEAD(&engine->execlist_queue);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 12cb7ed90014..3d47e23991cc 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -146,6 +146,7 @@ struct intel_engine_cs {
unsigned int exec_id;
unsigned int hw_id;
unsigned int guc_id; /* XXX same as hw_id? */
+ u64 fence_context;
u32 mmio_base;
struct intel_ringbuffer *buffer;
struct list_head buffers;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 11/64] drm/i915: Disable waitboosting for fence_wait()
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (8 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 10/64] drm/i915: Derive GEM requests from dma-fence Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 12/64] drm/i915: Disable waitboosting for mmioflips/semaphores Chris Wilson
` (53 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
We want to restrict waitboosting to known process contexts, where we can
track which clients are receiving waitboosts and prevent excessive power
wasting. For fence_wait() we do not have any client tracking and so that
leaves it open to abuse.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_request.c | 7 ++++---
drivers/gpu/drm/i915/i915_gem_request.h | 1 +
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 4184b5879aea..19636859852f 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -70,7 +70,7 @@ static signed long i915_fence_wait(struct fence *fence,
ret = __i915_wait_request(to_request(fence),
interruptible, timeout,
- NULL);
+ NO_WAITBOOST);
if (ret == -ETIME)
return 0;
@@ -641,7 +641,7 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
* forcing the clocks too high for the whole system, we only allow
* each client to waitboost once in a busy period.
*/
- if (INTEL_GEN(req->i915) >= 6)
+ if (!IS_ERR(rps) && INTEL_GEN(req->i915) >= 6)
gen6_rps_boost(req->i915, rps, req->emitted_jiffies);
/* Optimistic spin for the next ~jiffie before touching IRQs */
@@ -712,7 +712,8 @@ complete:
*timeout = 0;
}
- if (rps && req->fence.seqno == req->engine->last_submitted_seqno) {
+ if (!IS_ERR_OR_NULL(rps) &&
+ req->fence.seqno == req->engine->last_submitted_seqno) {
/* The GPU is now idle and this client has stalled.
* Since no other client has submitted a request in the
* meantime, assume that this client is the only one
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index 6f2c820785f3..f7f497a07893 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -206,6 +206,7 @@ void __i915_add_request(struct drm_i915_gem_request *req,
__i915_add_request(req, NULL, false)
struct intel_rps_client;
+#define NO_WAITBOOST ERR_PTR(-1)
int __i915_wait_request(struct drm_i915_gem_request *req,
bool interruptible,
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 12/64] drm/i915: Disable waitboosting for mmioflips/semaphores
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (9 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 11/64] drm/i915: Disable waitboosting for fence_wait() Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 13/64] drm/i915: Export our request as a dma-buf fence on the reservation object Chris Wilson
` (52 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Since commit a6f766f39751 ("drm/i915: Limit ring synchronisation (sw
sempahores) RPS boosts") and commit bcafc4e38b6a ("drm/i915: Limit mmio
flip RPS boosts") we have limited the waitboosting for semaphores and
flips. Ideally we do not want to boost in either of these instances as no
consumer is waiting upon the results. With the introduction of NO_WAITBOOST
in the previous patch, we can finally disable these needless boosts.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 8 +-------
drivers/gpu/drm/i915/i915_drv.h | 2 --
drivers/gpu/drm/i915/i915_gem.c | 2 +-
drivers/gpu/drm/i915/intel_display.c | 2 +-
drivers/gpu/drm/i915/intel_pm.c | 2 --
5 files changed, 3 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 87f259a09f21..78819b6e6f8e 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -2463,13 +2463,7 @@ static int i915_rps_boost_info(struct seq_file *m, void *data)
list_empty(&file_priv->rps.link) ? "" : ", active");
rcu_read_unlock();
}
- seq_printf(m, "Semaphore boosts: %d%s\n",
- dev_priv->rps.semaphores.boosts,
- list_empty(&dev_priv->rps.semaphores.link) ? "" : ", active");
- seq_printf(m, "MMIO flip boosts: %d%s\n",
- dev_priv->rps.mmioflips.boosts,
- list_empty(&dev_priv->rps.mmioflips.link) ? "" : ", active");
- seq_printf(m, "Kernel boosts: %d\n", dev_priv->rps.boosts);
+ seq_printf(m, "Kernel (anonymous) boosts: %d\n", dev_priv->rps.boosts);
spin_unlock(&dev_priv->rps.client_lock);
mutex_unlock(&dev->filelist_mutex);
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 9b062fe28483..97387921d292 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1190,8 +1190,6 @@ struct intel_gen6_power_mgmt {
struct delayed_work delayed_resume_work;
unsigned boosts;
- struct intel_rps_client semaphores, mmioflips;
-
/* manual wa residency calculations */
struct intel_rps_ei up_ei, down_ei;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 1c05e4573173..aab21041a04e 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2844,7 +2844,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
ret = __i915_wait_request(from_req,
i915->mm.interruptible,
NULL,
- &i915->rps.semaphores);
+ NO_WAITBOOST);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 5fb7b3926537..7bf59ca8b801 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11560,7 +11560,7 @@ static void intel_mmio_flip_work_func(struct work_struct *w)
if (work->flip_queued_req)
WARN_ON(__i915_wait_request(work->flip_queued_req,
false, NULL,
- &dev_priv->rps.mmioflips));
+ NO_WAITBOOST));
/* For framebuffer backed by dmabuf, wait for fence */
resv = i915_gem_object_get_dmabuf_resv(obj);
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index b12916a21b7b..3772a5bb6cbd 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -7843,8 +7843,6 @@ void intel_pm_setup(struct drm_device *dev)
INIT_DELAYED_WORK(&dev_priv->rps.delayed_resume_work,
__intel_autoenable_gt_powersave);
INIT_LIST_HEAD(&dev_priv->rps.clients);
- INIT_LIST_HEAD(&dev_priv->rps.semaphores.link);
- INIT_LIST_HEAD(&dev_priv->rps.mmioflips.link);
dev_priv->pm.suspended = false;
atomic_set(&dev_priv->pm.wakeref_count, 0);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 13/64] drm/i915: Export our request as a dma-buf fence on the reservation object
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (10 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 12/64] drm/i915: Disable waitboosting for mmioflips/semaphores Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 14/64] drm/i915: Wait on external rendering for GEM objects Chris Wilson
` (51 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
If the GEM objects being rendered with in this request have been
exported via dma-buf to a third party, hook ourselves into the dma-buf
reservation object so that the third party can serialise with our
rendering via the dma-buf fences.
Testcase: igt/prime_busy
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 42 +++++++++++++++++++++++++++---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 30 +++++++++++++++++++--
2 files changed, 67 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
index 80bbe43a2e92..075d7a94e77c 100644
--- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
@@ -23,9 +23,13 @@
* Authors:
* Dave Airlie <airlied@redhat.com>
*/
+
+#include <linux/dma-buf.h>
+#include <linux/reservation.h>
+
#include <drm/drmP.h>
+
#include "i915_drv.h"
-#include <linux/dma-buf.h>
static struct drm_i915_gem_object *dma_buf_to_obj(struct dma_buf *buf)
{
@@ -218,25 +222,57 @@ static const struct dma_buf_ops i915_dmabuf_ops = {
.end_cpu_access = i915_gem_end_cpu_access,
};
+static void export_fences(struct drm_i915_gem_object *obj,
+ struct dma_buf *dma_buf)
+{
+ struct reservation_object *resv = dma_buf->resv;
+ struct drm_i915_gem_request *req;
+ int idx;
+
+ mutex_lock(&obj->base.dev->struct_mutex);
+ mutex_lock(&resv->lock.base);
+
+ for (idx = 0; idx < ARRAY_SIZE(obj->last_read_req); idx++) {
+ req = obj->last_read_req[idx];
+ if (!req)
+ continue;
+
+ if (reservation_object_reserve_shared(resv) == 0)
+ reservation_object_add_shared_fence(resv, &req->fence);
+ }
+
+ req = obj->last_write_req;
+ if (req)
+ reservation_object_add_excl_fence(resv, &req->fence);
+
+ mutex_unlock(&resv->lock.base);
+ mutex_unlock(&obj->base.dev->struct_mutex);
+}
+
struct dma_buf *i915_gem_prime_export(struct drm_device *dev,
struct drm_gem_object *gem_obj, int flags)
{
struct drm_i915_gem_object *obj = to_intel_bo(gem_obj);
DEFINE_DMA_BUF_EXPORT_INFO(exp_info);
+ struct dma_buf *dma_buf;
exp_info.ops = &i915_dmabuf_ops;
exp_info.size = gem_obj->size;
exp_info.flags = flags;
exp_info.priv = gem_obj;
-
if (obj->ops->dmabuf_export) {
int ret = obj->ops->dmabuf_export(obj);
if (ret)
return ERR_PTR(ret);
}
- return dma_buf_export(&exp_info);
+ dma_buf = dma_buf_export(&exp_info);
+ if (IS_ERR(dma_buf))
+ return dma_buf;
+
+ export_fences(obj, dma_buf);
+ return dma_buf;
}
static int i915_gem_object_get_pages_dmabuf(struct drm_i915_gem_object *obj)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 1978633e7549..359ef0afa312 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -26,13 +26,17 @@
*
*/
+#include <linux/dma_remapping.h>
+#include <linux/reservation.h>
+#include <linux/uaccess.h>
+
#include <drm/drmP.h>
#include <drm/i915_drm.h>
+
#include "i915_drv.h"
+#include "i915_gem_dmabuf.h"
#include "i915_trace.h"
#include "intel_drv.h"
-#include <linux/dma_remapping.h>
-#include <linux/uaccess.h>
#define __EXEC_OBJECT_HAS_PIN (1<<31)
#define __EXEC_OBJECT_HAS_FENCE (1<<30)
@@ -1086,6 +1090,27 @@ i915_gem_validate_context(struct drm_device *dev, struct drm_file *file,
return ctx;
}
+static void eb_export_fence(struct drm_i915_gem_object *obj,
+ struct drm_i915_gem_request *req)
+{
+ struct reservation_object *resv;
+
+ resv = i915_gem_object_get_dmabuf_resv(obj);
+ if (!resv)
+ return;
+
+ /* Ignore errors from failing to allocate the new fence, we can't
+ * handle an error right now. Worst case should be missed
+ * synchronisation leading to rendering corruption.
+ */
+ mutex_lock(&resv->lock.base);
+ if (obj->base.write_domain)
+ reservation_object_add_excl_fence(resv, &req->fence);
+ else if (reservation_object_reserve_shared(resv) == 0)
+ reservation_object_add_shared_fence(resv, &req->fence);
+ mutex_unlock(&resv->lock.base);
+}
+
void
i915_gem_execbuffer_move_to_active(struct list_head *vmas,
struct drm_i915_gem_request *req)
@@ -1106,6 +1131,7 @@ i915_gem_execbuffer_move_to_active(struct list_head *vmas,
obj->base.read_domains = obj->base.pending_read_domains;
i915_vma_move_to_active(vma, req);
+ eb_export_fence(obj, req);
if (obj->base.write_domain) {
i915_gem_request_assign(&obj->last_write_req, req);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 14/64] drm/i915: Wait on external rendering for GEM objects
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (11 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 13/64] drm/i915: Export our request as a dma-buf fence on the reservation object Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 15/64] drm/i915: Mark imported dma-buf objects as being coherent Chris Wilson
` (50 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
When transitioning to the GTT or CPU domain we wait on all rendering
from i915 to complete (with the optimisation of allowing concurrent read
access by both the GPU and client). We don't yet ensure all rendering
from third parties (tracked by implicit fences on the dma-buf) is
complete. Since implicitly tracked rendering by third parties will
ignore our cache-domain tracking, we have to always wait upon rendering
from third-parties when transitioning to direct access to the backing
store. We still rely on clients notifying us of cache domain changes
(i.e. they need to move to the GTT read or write domain after doing a CPU
access before letting the third party render again).
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem.c | 42 ++++++++++++++++++++++++++---------------
1 file changed, 27 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index aab21041a04e..82925b20f5f1 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -29,10 +29,12 @@
#include <drm/drm_vma_manager.h>
#include <drm/i915_drm.h>
#include "i915_drv.h"
+#include "i915_gem_dmabuf.h"
#include "i915_vgpu.h"
#include "i915_trace.h"
#include "intel_drv.h"
#include "intel_mocs.h"
+#include <linux/reservation.h>
#include <linux/shmem_fs.h>
#include <linux/slab.h>
#include <linux/swap.h>
@@ -511,6 +513,10 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,
if (WARN_ON(!i915_gem_object_has_struct_page(obj)))
return -EINVAL;
+ ret = i915_gem_object_wait_rendering(obj, true);
+ if (ret)
+ return ret;
+
if (!(obj->base.read_domains & I915_GEM_DOMAIN_CPU)) {
/* If we're not in the cpu read domain, set ourself into the gtt
* read domain and manually flush cachelines (if required). This
@@ -518,9 +524,6 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj,
* anyway again before the next pread happens. */
*needs_clflush = !cpu_cache_is_coherent(obj->base.dev,
obj->cache_level);
- ret = i915_gem_object_wait_rendering(obj, true);
- if (ret)
- return ret;
}
ret = i915_gem_object_get_pages(obj);
@@ -1132,15 +1135,16 @@ i915_gem_shmem_pwrite(struct drm_device *dev,
obj_do_bit17_swizzling = i915_gem_object_needs_bit17_swizzle(obj);
+ ret = i915_gem_object_wait_rendering(obj, false);
+ if (ret)
+ return ret;
+
if (obj->base.write_domain != I915_GEM_DOMAIN_CPU) {
/* If we're not in the cpu write domain, set ourself into the gtt
* write domain and manually flush cachelines (if required). This
* optimizes for the case when the gpu will use the data
* right away and we therefore have to clflush anyway. */
needs_clflush_after = cpu_write_needs_clflush(obj);
- ret = i915_gem_object_wait_rendering(obj, false);
- if (ret)
- return ret;
}
/* Same trick applies to invalidate partially written cachelines read
* before writing. */
@@ -1335,11 +1339,9 @@ int
i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
bool readonly)
{
+ struct reservation_object *resv;
int ret, i;
- if (!obj->active)
- return 0;
-
if (readonly) {
if (obj->last_write_req != NULL) {
ret = i915_wait_request(obj->last_write_req);
@@ -1366,6 +1368,16 @@ i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
GEM_BUG_ON(obj->active);
}
+ resv = i915_gem_object_get_dmabuf_resv(obj);
+ if (resv) {
+ long err;
+
+ err = reservation_object_wait_timeout_rcu(resv, !readonly, true,
+ MAX_SCHEDULE_TIMEOUT);
+ if (err < 0)
+ return err;
+ }
+
return 0;
}
@@ -3397,13 +3409,13 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
struct i915_vma *vma;
int ret;
- if (obj->base.write_domain == I915_GEM_DOMAIN_GTT)
- return 0;
-
ret = i915_gem_object_wait_rendering(obj, !write);
if (ret)
return ret;
+ if (obj->base.write_domain == I915_GEM_DOMAIN_GTT)
+ return 0;
+
/* Flush and acquire obj->pages so that we are coherent through
* direct access in memory with previous cached writes through
* shmemfs and that our cache domain tracking remains valid.
@@ -3747,13 +3759,13 @@ i915_gem_object_set_to_cpu_domain(struct drm_i915_gem_object *obj, bool write)
uint32_t old_write_domain, old_read_domains;
int ret;
- if (obj->base.write_domain == I915_GEM_DOMAIN_CPU)
- return 0;
-
ret = i915_gem_object_wait_rendering(obj, !write);
if (ret)
return ret;
+ if (obj->base.write_domain == I915_GEM_DOMAIN_CPU)
+ return 0;
+
i915_gem_object_flush_gtt_write_domain(obj);
old_write_domain = obj->base.write_domain;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 15/64] drm/i915: Mark imported dma-buf objects as being coherent
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (12 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 14/64] drm/i915: Wait on external rendering for GEM objects Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 16/64] drm/i915: Rename request reference/unreference to get/put Chris Wilson
` (49 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
A foreign dma-buf does not share our cache domain tracking, and we rely
on the producer ensuring cache coherency. Marking them as being in the
CPU domain is incorrect.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
index 075d7a94e77c..67f7fc5c59ae 100644
--- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
@@ -335,6 +335,8 @@ struct drm_gem_object *i915_gem_prime_import(struct drm_device *dev,
drm_gem_private_object_init(dev, &obj->base, dma_buf->size);
i915_gem_object_init(obj, &i915_gem_object_dmabuf_ops);
obj->base.import_attach = attach;
+ obj->base.read_domains = I915_GEM_DOMAIN_GTT;
+ obj->base.write_domain = 0;
return &obj->base;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 16/64] drm/i915: Rename request reference/unreference to get/put
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (13 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 15/64] drm/i915: Mark imported dma-buf objects as being coherent Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 17/64] drm/i915: Rename i915_gem_context_reference/unreference() Chris Wilson
` (48 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Now that we derive requests from struct fence, swap over to its
nomenclature for references. It's shorter and more idiomatic across the
kernel.
s/i915_gem_request_reference/i915_gem_request_get/
s/i915_gem_request_unreference/i915_gem_request_put/
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch>
---
drivers/gpu/drm/i915/i915_gem.c | 14 +++++++-------
drivers/gpu/drm/i915/i915_gem_request.c | 2 +-
drivers/gpu/drm/i915/i915_gem_request.h | 8 ++++----
drivers/gpu/drm/i915/i915_gem_userptr.c | 4 ++--
drivers/gpu/drm/i915/intel_breadcrumbs.c | 4 ++--
drivers/gpu/drm/i915/intel_display.c | 5 ++---
drivers/gpu/drm/i915/intel_lrc.c | 10 +++++-----
drivers/gpu/drm/i915/intel_pm.c | 5 ++---
8 files changed, 25 insertions(+), 27 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 82925b20f5f1..abd9577c87dc 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1422,7 +1422,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
if (req == NULL)
return 0;
- requests[n++] = i915_gem_request_reference(req);
+ requests[n++] = i915_gem_request_get(req);
} else {
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
@@ -1431,7 +1431,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
if (req == NULL)
continue;
- requests[n++] = i915_gem_request_reference(req);
+ requests[n++] = i915_gem_request_get(req);
}
}
@@ -1444,7 +1444,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
for (i = 0; i < n; i++) {
if (ret == 0)
i915_gem_object_retire_request(obj, requests[i]);
- i915_gem_request_unreference(requests[i]);
+ i915_gem_request_put(requests[i]);
}
return ret;
@@ -2815,7 +2815,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
if (obj->last_read_req[i] == NULL)
continue;
- req[n++] = i915_gem_request_reference(obj->last_read_req[i]);
+ req[n++] = i915_gem_request_get(obj->last_read_req[i]);
}
mutex_unlock(&dev->struct_mutex);
@@ -2825,7 +2825,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
ret = __i915_wait_request(req[i], true,
args->timeout_ns > 0 ? &args->timeout_ns : NULL,
to_rps_client(file));
- i915_gem_request_unreference(req[i]);
+ i915_gem_request_put(req[i]);
}
return ret;
@@ -3840,14 +3840,14 @@ i915_gem_ring_throttle(struct drm_device *dev, struct drm_file *file)
target = request;
}
if (target)
- i915_gem_request_reference(target);
+ i915_gem_request_get(target);
spin_unlock(&file_priv->mm.lock);
if (target == NULL)
return 0;
ret = __i915_wait_request(target, true, NULL, NULL);
- i915_gem_request_unreference(target);
+ i915_gem_request_put(target);
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 19636859852f..a3635dc9ca15 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -181,7 +181,7 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
}
i915_gem_context_unreference(request->ctx);
- i915_gem_request_unreference(request);
+ i915_gem_request_put(request);
}
void i915_gem_request_retire_upto(struct drm_i915_gem_request *req)
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index f7f497a07893..ed3f34dfeecd 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -174,13 +174,13 @@ to_request(struct fence *fence)
}
static inline struct drm_i915_gem_request *
-i915_gem_request_reference(struct drm_i915_gem_request *req)
+i915_gem_request_get(struct drm_i915_gem_request *req)
{
return to_request(fence_get(&req->fence));
}
static inline void
-i915_gem_request_unreference(struct drm_i915_gem_request *req)
+i915_gem_request_put(struct drm_i915_gem_request *req)
{
fence_put(&req->fence);
}
@@ -189,10 +189,10 @@ static inline void i915_gem_request_assign(struct drm_i915_gem_request **pdst,
struct drm_i915_gem_request *src)
{
if (src)
- i915_gem_request_reference(src);
+ i915_gem_request_get(src);
if (*pdst)
- i915_gem_request_unreference(*pdst);
+ i915_gem_request_put(*pdst);
*pdst = src;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index 2314c88323e3..ba16e044fac6 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -78,7 +78,7 @@ static void wait_rendering(struct drm_i915_gem_object *obj)
if (req == NULL)
continue;
- requests[n++] = i915_gem_request_reference(req);
+ requests[n++] = i915_gem_request_get(req);
}
mutex_unlock(&dev->struct_mutex);
@@ -89,7 +89,7 @@ static void wait_rendering(struct drm_i915_gem_object *obj)
mutex_lock(&dev->struct_mutex);
for (i = 0; i < n; i++)
- i915_gem_request_unreference(requests[i]);
+ i915_gem_request_put(requests[i]);
}
static void cancel_userptr(struct work_struct *work)
diff --git a/drivers/gpu/drm/i915/intel_breadcrumbs.c b/drivers/gpu/drm/i915/intel_breadcrumbs.c
index 32ada41dedfb..f0b56e3f4abe 100644
--- a/drivers/gpu/drm/i915/intel_breadcrumbs.c
+++ b/drivers/gpu/drm/i915/intel_breadcrumbs.c
@@ -453,7 +453,7 @@ static int intel_breadcrumbs_signaler(void *arg)
rb_erase(&request->signaling.node, &b->signals);
spin_unlock(&b->lock);
- i915_gem_request_unreference(request);
+ i915_gem_request_put(request);
} else {
if (kthread_should_stop())
break;
@@ -484,7 +484,7 @@ void intel_engine_enable_signaling(struct drm_i915_gem_request *request)
request->signaling.wait.tsk = b->signaler;
request->signaling.wait.seqno = request->fence.seqno;
- i915_gem_request_reference(request);
+ i915_gem_request_get(request);
/* First add ourselves into the list of waiters, but register our
* bottom-half as the signaller thread. As per usual, only the oldest
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 7bf59ca8b801..c614d30271d2 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11041,11 +11041,10 @@ static void intel_unpin_work_fn(struct work_struct *__work)
mutex_lock(&dev->struct_mutex);
intel_unpin_fb_obj(work->old_fb, primary->state->rotation);
drm_gem_object_unreference(&work->pending_flip_obj->base);
-
- if (work->flip_queued_req)
- i915_gem_request_assign(&work->flip_queued_req, NULL);
mutex_unlock(&dev->struct_mutex);
+ i915_gem_request_put(work->flip_queued_req);
+
intel_frontbuffer_flip_complete(dev, to_intel_plane(primary)->frontbuffer_bit);
intel_fbc_post_update(crtc);
drm_framebuffer_unreference(work->old_fb);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index b22931890a4c..481a5c836fb5 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -441,7 +441,7 @@ static void execlists_context_unqueue(struct intel_engine_cs *engine)
* will update tail past first request's workload */
cursor->elsp_submitted = req0->elsp_submitted;
list_del(&req0->execlist_link);
- i915_gem_request_unreference(req0);
+ i915_gem_request_put(req0);
req0 = cursor;
} else {
if (IS_ENABLED(CONFIG_DRM_I915_GVT)) {
@@ -514,7 +514,7 @@ execlists_check_remove_request(struct intel_engine_cs *engine, u32 ctx_id)
execlists_context_status_change(head_req, INTEL_CONTEXT_SCHEDULE_OUT);
list_del(&head_req->execlist_link);
- i915_gem_request_unreference(head_req);
+ i915_gem_request_put(head_req);
return 1;
}
@@ -635,11 +635,11 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
WARN(tail_req->elsp_submitted != 0,
"More than 2 already-submitted reqs queued\n");
list_del(&tail_req->execlist_link);
- i915_gem_request_unreference(tail_req);
+ i915_gem_request_put(tail_req);
}
}
- i915_gem_request_reference(request);
+ i915_gem_request_get(request);
list_add_tail(&request->execlist_link, &engine->execlist_queue);
request->ctx_hw_id = request->ctx->hw_id;
if (num_elements == 0)
@@ -907,7 +907,7 @@ void intel_execlists_cancel_requests(struct intel_engine_cs *engine)
list_for_each_entry_safe(req, tmp, &cancel_list, execlist_link) {
list_del(&req->execlist_link);
- i915_gem_request_unreference(req);
+ i915_gem_request_put(req);
}
}
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index 3772a5bb6cbd..960eaf5051ed 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -7808,7 +7808,7 @@ static void __intel_rps_boost_work(struct work_struct *work)
if (!i915_gem_request_completed(req))
gen6_rps_boost(req->i915, NULL, req->emitted_jiffies);
- i915_gem_request_unreference(req);
+ i915_gem_request_put(req);
kfree(boost);
}
@@ -7826,8 +7826,7 @@ void intel_queue_rps_boost_for_request(struct drm_i915_gem_request *req)
if (boost == NULL)
return;
- i915_gem_request_reference(req);
- boost->req = req;
+ boost->req = i915_gem_request_get(req);
INIT_WORK(&boost->work, __intel_rps_boost_work);
queue_work(req->i915->wq, &boost->work);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 17/64] drm/i915: Rename i915_gem_context_reference/unreference()
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (14 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 16/64] drm/i915: Rename request reference/unreference to get/put Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 18/64] drm/i915: Wrap drm_gem_object_lookup in i915_gem_object_lookup Chris Wilson
` (47 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
As these are wrappers around kref_get/kref_put() it is preferable to
follow the naming convention and use the same verb get/put in our
wrapper names for manipulating a reference to the context.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Reviewed-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
---
drivers/gpu/drm/i915/i915_drv.h | 6 ++++--
drivers/gpu/drm/i915/i915_gem_context.c | 22 ++++++++++------------
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 6 +++---
drivers/gpu/drm/i915/i915_gem_request.c | 7 +++----
drivers/gpu/drm/i915/intel_lrc.c | 4 ++--
drivers/gpu/drm/i915/intel_ringbuffer.c | 4 ++--
6 files changed, 24 insertions(+), 25 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 97387921d292..6fca16e32b08 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3320,12 +3320,14 @@ i915_gem_context_lookup(struct drm_i915_file_private *file_priv, u32 id)
return ctx;
}
-static inline void i915_gem_context_reference(struct i915_gem_context *ctx)
+static inline struct i915_gem_context *
+i915_gem_context_get(struct i915_gem_context *ctx)
{
kref_get(&ctx->ref);
+ return ctx;
}
-static inline void i915_gem_context_unreference(struct i915_gem_context *ctx)
+static inline void i915_gem_context_put(struct i915_gem_context *ctx)
{
lockdep_assert_held(&ctx->i915->drm.struct_mutex);
kref_put(&ctx->ref, i915_gem_context_free);
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 3c97f0e7a003..3f151b12893b 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -305,7 +305,7 @@ __create_hw_context(struct drm_device *dev,
return ctx;
err_out:
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
return ERR_PTR(ret);
}
@@ -333,7 +333,7 @@ i915_gem_create_context(struct drm_device *dev,
DRM_DEBUG_DRIVER("PPGTT setup failed (%ld)\n",
PTR_ERR(ppgtt));
idr_remove(&file_priv->context_idr, ctx->user_handle);
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
return ERR_CAST(ppgtt);
}
@@ -390,7 +390,7 @@ static void i915_gem_context_unpin(struct i915_gem_context *ctx,
if (ce->state)
i915_gem_object_ggtt_unpin(ce->state);
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
}
}
@@ -504,7 +504,7 @@ void i915_gem_context_fini(struct drm_device *dev)
lockdep_assert_held(&dev->struct_mutex);
- i915_gem_context_unreference(dctx);
+ i915_gem_context_put(dctx);
dev_priv->kernel_context = NULL;
ida_destroy(&dev_priv->context_hw_ida);
@@ -515,7 +515,7 @@ static int context_idr_cleanup(int id, void *p, void *data)
struct i915_gem_context *ctx = p;
ctx->file_priv = ERR_PTR(-EBADF);
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
return 0;
}
@@ -827,10 +827,9 @@ static int do_rcs_switch(struct drm_i915_gem_request *req)
/* obj is kept alive until the next request by its active ref */
i915_gem_object_ggtt_unpin(from->engine[RCS].state);
- i915_gem_context_unreference(from);
+ i915_gem_context_put(from);
}
- i915_gem_context_reference(to);
- engine->last_context = to;
+ engine->last_context = i915_gem_context_get(to);
/* GEN8 does *not* require an explicit reload if the PDPs have been
* setup, and we do not wish to move them.
@@ -914,10 +913,9 @@ int i915_switch_context(struct drm_i915_gem_request *req)
}
if (to != engine->last_context) {
- i915_gem_context_reference(to);
if (engine->last_context)
- i915_gem_context_unreference(engine->last_context);
- engine->last_context = to;
+ i915_gem_context_put(engine->last_context);
+ engine->last_context = i915_gem_context_get(to);
}
return 0;
@@ -985,7 +983,7 @@ int i915_gem_context_destroy_ioctl(struct drm_device *dev, void *data,
}
idr_remove(&file_priv->context_idr, ctx->user_handle);
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
mutex_unlock(&dev->struct_mutex);
DRM_DEBUG_DRIVER("HW context %d destroyed\n", args->ctx_id);
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 359ef0afa312..d9dd919fa48b 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1522,7 +1522,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
goto pre_mutex_err;
}
- i915_gem_context_reference(ctx);
+ i915_gem_context_get(ctx);
if (ctx->ppgtt)
vm = &ctx->ppgtt->base;
@@ -1533,7 +1533,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
eb = eb_create(args);
if (eb == NULL) {
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
mutex_unlock(&dev->struct_mutex);
ret = -ENOMEM;
goto pre_mutex_err;
@@ -1677,7 +1677,7 @@ err_batch_unpin:
err:
/* the request owns the ref now */
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
eb_destroy(eb);
mutex_unlock(&dev->struct_mutex);
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index a3635dc9ca15..f15924c8a26e 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -180,7 +180,7 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
request->engine);
}
- i915_gem_context_unreference(request->ctx);
+ i915_gem_context_put(request->ctx);
i915_gem_request_put(request);
}
@@ -340,8 +340,7 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
req->i915 = dev_priv;
req->engine = engine;
- req->ctx = ctx;
- i915_gem_context_reference(ctx);
+ req->ctx = i915_gem_context_get(ctx);
/*
* Reserve space in the ring buffer for all the commands required to
@@ -363,7 +362,7 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
return 0;
err_ctx:
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
err:
kmem_cache_free(dev_priv->requests, req);
return ret;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 481a5c836fb5..e55a78c1250c 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -983,7 +983,6 @@ static int intel_lr_context_pin(struct i915_gem_context *ctx,
if (ret)
goto unpin_map;
- i915_gem_context_reference(ctx);
ce->lrc_vma = i915_gem_obj_to_ggtt(ce->state);
intel_lr_context_descriptor_update(ctx, engine);
@@ -995,6 +994,7 @@ static int intel_lr_context_pin(struct i915_gem_context *ctx,
if (i915.enable_guc_submission)
I915_WRITE(GEN8_GTCR, GEN8_GTCR_INVALIDATE);
+ i915_gem_context_get(ctx);
return 0;
unpin_map:
@@ -1026,7 +1026,7 @@ void intel_lr_context_unpin(struct i915_gem_context *ctx,
ce->lrc_desc = 0;
ce->lrc_reg_state = NULL;
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
}
static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 29da7a7aff86..ac079dc41c23 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2143,7 +2143,7 @@ static int intel_ring_context_pin(struct i915_gem_context *ctx,
if (ctx == ctx->i915->kernel_context)
ce->initialised = true;
- i915_gem_context_reference(ctx);
+ i915_gem_context_get(ctx);
return 0;
error:
@@ -2164,7 +2164,7 @@ static void intel_ring_context_unpin(struct i915_gem_context *ctx,
if (ce->state)
i915_gem_object_ggtt_unpin(ce->state);
- i915_gem_context_unreference(ctx);
+ i915_gem_context_put(ctx);
}
static int intel_init_ring_buffer(struct drm_device *dev,
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 18/64] drm/i915: Wrap drm_gem_object_lookup in i915_gem_object_lookup
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (15 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 17/64] drm/i915: Rename i915_gem_context_reference/unreference() Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 19/64] drm/i915: Wrap drm_gem_object_reference in i915_gem_object_get Chris Wilson
` (46 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
For symmetry with a forthcoming i915_gem_object_get() and
i915_gem_object_pu().
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 20 +++++++++++-
drivers/gpu/drm/i915/i915_gem.c | 58 +++++++++++++++++-----------------
drivers/gpu/drm/i915/i915_gem_tiling.c | 8 ++---
drivers/gpu/drm/i915/intel_display.c | 4 +--
drivers/gpu/drm/i915/intel_overlay.c | 5 ++-
5 files changed, 56 insertions(+), 39 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 6fca16e32b08..43456b895a91 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2282,7 +2282,25 @@ struct drm_i915_gem_object {
} userptr;
};
};
-#define to_intel_bo(x) container_of(x, struct drm_i915_gem_object, base)
+
+static inline struct drm_i915_gem_object *
+to_intel_bo(struct drm_gem_object *gem)
+{
+ /* Assert that to_intel_bo(NULL) == NULL */
+ BUILD_BUG_ON(offsetof(struct drm_i915_gem_object, base));
+
+ return container_of(gem, struct drm_i915_gem_object, base);
+}
+
+static inline struct drm_i915_gem_object *
+i915_gem_object_lookup(struct drm_file *file, u32 handle)
+{
+ return to_intel_bo(drm_gem_object_lookup(file, handle));
+}
+
+__deprecated
+extern struct drm_gem_object *
+drm_gem_object_lookup(struct drm_file *file, u32 handle);
static inline bool
i915_gem_object_has_struct_page(const struct drm_i915_gem_object *obj)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index abd9577c87dc..363c6180e774 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -864,8 +864,8 @@ i915_gem_pread_ioctl(struct drm_device *dev, void *data,
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
@@ -1280,8 +1280,8 @@ i915_gem_pwrite_ioctl(struct drm_device *dev, void *data,
if (ret)
goto put_rpm;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
@@ -1497,8 +1497,8 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
@@ -1546,8 +1546,8 @@ i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
@@ -1587,7 +1587,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
struct drm_file *file)
{
struct drm_i915_gem_mmap *args = data;
- struct drm_gem_object *obj;
+ struct drm_i915_gem_object *obj;
unsigned long addr;
if (args->flags & ~(I915_MMAP_WC))
@@ -1596,19 +1596,19 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
if (args->flags & I915_MMAP_WC && !boot_cpu_has(X86_FEATURE_PAT))
return -ENODEV;
- obj = drm_gem_object_lookup(file, args->handle);
- if (obj == NULL)
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj)
return -ENOENT;
/* prime objects have no backing filp to GEM mmap
* pages from.
*/
- if (!obj->filp) {
- drm_gem_object_unreference_unlocked(obj);
+ if (!obj->base.filp) {
+ drm_gem_object_unreference_unlocked(&obj->base);
return -EINVAL;
}
- addr = vm_mmap(obj->filp, 0, args->size,
+ addr = vm_mmap(obj->base.filp, 0, args->size,
PROT_READ | PROT_WRITE, MAP_SHARED,
args->offset);
if (args->flags & I915_MMAP_WC) {
@@ -1616,7 +1616,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
struct vm_area_struct *vma;
if (down_write_killable(&mm->mmap_sem)) {
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_unreference_unlocked(&obj->base);
return -EINTR;
}
vma = find_vma(mm, addr);
@@ -1628,9 +1628,9 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
up_write(&mm->mmap_sem);
/* This may race, but that's ok, it only gets set */
- WRITE_ONCE(to_intel_bo(obj)->has_wc_mmap, true);
+ WRITE_ONCE(obj->has_wc_mmap, true);
}
- drm_gem_object_unreference_unlocked(obj);
+ drm_gem_object_unreference_unlocked(&obj->base);
if (IS_ERR((void *)addr))
return addr;
@@ -1968,8 +1968,8 @@ i915_gem_mmap_gtt(struct drm_file *file,
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(file, handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
@@ -2787,8 +2787,8 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->bo_handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, args->bo_handle);
+ if (!obj) {
mutex_unlock(&dev->struct_mutex);
return -ENOENT;
}
@@ -3591,8 +3591,8 @@ int i915_gem_get_caching_ioctl(struct drm_device *dev, void *data,
struct drm_i915_gem_caching *args = data;
struct drm_i915_gem_object *obj;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL)
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj)
return -ENOENT;
switch (obj->cache_level) {
@@ -3652,8 +3652,8 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,
if (ret)
goto rpm_put;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
@@ -4021,8 +4021,8 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
@@ -4086,8 +4086,8 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
if (ret)
return ret;
- obj = to_intel_bo(drm_gem_object_lookup(file_priv, args->handle));
- if (&obj->base == NULL) {
+ obj = i915_gem_object_lookup(file_priv, args->handle);
+ if (!obj) {
ret = -ENOENT;
goto unlock;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
index 8030199731db..46e80f30afcd 100644
--- a/drivers/gpu/drm/i915/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
@@ -166,8 +166,8 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
struct drm_i915_gem_object *obj;
int ret = 0;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL)
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj)
return -ENOENT;
if (!i915_tiling_ok(dev,
@@ -297,8 +297,8 @@ i915_gem_get_tiling(struct drm_device *dev, void *data,
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_i915_gem_object *obj;
- obj = to_intel_bo(drm_gem_object_lookup(file, args->handle));
- if (&obj->base == NULL)
+ obj = i915_gem_object_lookup(file, args->handle);
+ if (!obj)
return -ENOENT;
mutex_lock(&dev->struct_mutex);
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index c614d30271d2..384801788e0d 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -15134,8 +15134,8 @@ intel_user_framebuffer_create(struct drm_device *dev,
struct drm_i915_gem_object *obj;
struct drm_mode_fb_cmd2 mode_cmd = *user_mode_cmd;
- obj = to_intel_bo(drm_gem_object_lookup(filp, mode_cmd.handles[0]));
- if (&obj->base == NULL)
+ obj = i915_gem_object_lookup(filp, mode_cmd.handles[0]);
+ if (!obj)
return ERR_PTR(-ENOENT);
fb = intel_framebuffer_create(dev, &mode_cmd, obj);
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 3212d8806b5a..5ca797b01ccb 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -1122,9 +1122,8 @@ int intel_overlay_put_image_ioctl(struct drm_device *dev, void *data,
}
crtc = to_intel_crtc(drmmode_crtc);
- new_bo = to_intel_bo(drm_gem_object_lookup(file_priv,
- put_image_rec->bo_handle));
- if (&new_bo->base == NULL) {
+ new_bo = i915_gem_object_lookup(file_priv, put_image_rec->bo_handle);
+ if (!new_bo) {
ret = -ENOENT;
goto out_free;
}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 19/64] drm/i915: Wrap drm_gem_object_reference in i915_gem_object_get
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (16 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 18/64] drm/i915: Wrap drm_gem_object_lookup in i915_gem_object_lookup Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 20/64] drm/i915: Rename drm_gem_object_unreference in preparation for lockless free Chris Wilson
` (45 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 11 +++++++++++
drivers/gpu/drm/i915/i915_gem.c | 4 ++--
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 3 +--
drivers/gpu/drm/i915/i915_gem_evict.c | 2 +-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 4 ++--
drivers/gpu/drm/i915/i915_gem_shrinker.c | 2 +-
drivers/gpu/drm/i915/i915_gem_userptr.c | 3 +--
drivers/gpu/drm/i915/intel_display.c | 3 +--
8 files changed, 20 insertions(+), 12 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 43456b895a91..aee4c708637e 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2302,6 +2302,17 @@ __deprecated
extern struct drm_gem_object *
drm_gem_object_lookup(struct drm_file *file, u32 handle);
+__attribute__((nonnull))
+static inline struct drm_i915_gem_object *
+i915_gem_object_get(struct drm_i915_gem_object *obj)
+{
+ drm_gem_object_reference(&obj->base);
+ return obj;
+}
+
+__deprecated
+extern void drm_gem_object_reference(struct drm_gem_object *);
+
static inline bool
i915_gem_object_has_struct_page(const struct drm_i915_gem_object *obj)
{
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 363c6180e774..e2881ae12b93 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -289,7 +289,7 @@ drop_pages(struct drm_i915_gem_object *obj)
struct i915_vma *vma, *next;
int ret;
- drm_gem_object_reference(&obj->base);
+ i915_gem_object_get(obj);
list_for_each_entry_safe(vma, next, &obj->vma_list, obj_link)
if (i915_vma_unbind(vma))
break;
@@ -2361,7 +2361,7 @@ void i915_vma_move_to_active(struct i915_vma *vma,
/* Add a reference if we're newly entering the active list. */
if (obj->active == 0)
- drm_gem_object_reference(&obj->base);
+ i915_gem_object_get(obj);
obj->active |= intel_engine_flag(engine);
list_move_tail(&obj->engine_list[engine->id], &engine->active_list);
diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
index 67f7fc5c59ae..85a5c86ba80a 100644
--- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
@@ -314,8 +314,7 @@ struct drm_gem_object *i915_gem_prime_import(struct drm_device *dev,
* Importing dmabuf exported from out own gem increases
* refcount on gem itself instead of f_count of dmabuf.
*/
- drm_gem_object_reference(&obj->base);
- return &obj->base;
+ return &i915_gem_object_get(obj)->base;
}
}
diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c
index 3c1280ec7ff6..d5777a0750f0 100644
--- a/drivers/gpu/drm/i915/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/i915_gem_evict.c
@@ -214,7 +214,7 @@ found:
exec_list);
if (drm_mm_scan_remove_block(&vma->node)) {
list_move(&vma->exec_list, &eviction_list);
- drm_gem_object_reference(&vma->obj->base);
+ i915_gem_object_get(vma->obj);
continue;
}
list_del_init(&vma->exec_list);
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index d9dd919fa48b..e61751beabf8 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -126,7 +126,7 @@ eb_lookup_vmas(struct eb_vmas *eb,
goto err;
}
- drm_gem_object_reference(&obj->base);
+ i915_gem_object_get(obj);
list_add_tail(&obj->obj_exec_link, &objects);
}
spin_unlock(&file->table_lock);
@@ -1229,7 +1229,7 @@ i915_gem_execbuffer_parse(struct intel_engine_cs *engine,
vma = i915_gem_obj_to_ggtt(shadow_batch_obj);
vma->exec_entry = shadow_exec_entry;
vma->exec_entry->flags = __EXEC_OBJECT_HAS_PIN;
- drm_gem_object_reference(&shadow_batch_obj->base);
+ i915_gem_object_get(shadow_batch_obj);
list_add_tail(&vma->exec_list, &eb->vmas);
shadow_batch_obj->base.pending_read_domains = I915_GEM_DOMAIN_COMMAND;
diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
index 067632ad2f29..3c9d771d499b 100644
--- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
@@ -190,7 +190,7 @@ i915_gem_shrink(struct drm_i915_private *dev_priv,
if (!can_release_pages(obj))
continue;
- drm_gem_object_reference(&obj->base);
+ i915_gem_object_get(obj);
/* For the unbound phase, this should be a no-op! */
list_for_each_entry_safe(vma, v,
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index ba16e044fac6..c41bf74f926e 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -622,8 +622,7 @@ __i915_gem_userptr_get_pages_schedule(struct drm_i915_gem_object *obj,
obj->userptr.work = &work->work;
obj->userptr.workers++;
- work->obj = obj;
- drm_gem_object_reference(&obj->base);
+ work->obj = i915_gem_object_get(obj);
work->task = current;
get_task_struct(work->task);
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 384801788e0d..bde2bf838d64 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11736,7 +11736,6 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
/* Reference the objects for the scheduled work. */
drm_framebuffer_reference(work->old_fb);
- drm_gem_object_reference(&obj->base);
crtc->primary->fb = fb;
update_state_fb(crtc->primary);
@@ -11744,7 +11743,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
intel_fbc_pre_update(intel_crtc, intel_crtc->config,
to_intel_plane_state(primary->state));
- work->pending_flip_obj = obj;
+ work->pending_flip_obj = i915_gem_object_get(obj);
ret = i915_mutex_lock_interruptible(dev);
if (ret)
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 20/64] drm/i915: Rename drm_gem_object_unreference in preparation for lockless free
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (17 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 19/64] drm/i915: Wrap drm_gem_object_reference in i915_gem_object_get Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 21/64] drm/i915: Rename drm_gem_object_unreference_unlocked " Chris Wilson
` (44 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++
drivers/gpu/drm/i915/i915_gem.c | 26 +++++++++++++-------------
drivers/gpu/drm/i915/i915_gem_batch_pool.c | 4 ++--
drivers/gpu/drm/i915/i915_gem_context.c | 4 ++--
drivers/gpu/drm/i915/i915_gem_evict.c | 7 ++++---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 6 +++---
drivers/gpu/drm/i915/i915_gem_render_state.c | 4 ++--
drivers/gpu/drm/i915/i915_gem_shrinker.c | 2 +-
drivers/gpu/drm/i915/i915_gem_stolen.c | 2 +-
drivers/gpu/drm/i915/i915_gem_tiling.c | 4 ++--
drivers/gpu/drm/i915/i915_gem_userptr.c | 4 ++--
drivers/gpu/drm/i915/i915_guc_submission.c | 6 +++---
drivers/gpu/drm/i915/intel_display.c | 6 +++---
drivers/gpu/drm/i915/intel_fbdev.c | 2 +-
drivers/gpu/drm/i915/intel_guc_loader.c | 8 +++++---
drivers/gpu/drm/i915/intel_lrc.c | 6 +++---
drivers/gpu/drm/i915/intel_overlay.c | 8 ++++----
drivers/gpu/drm/i915/intel_ringbuffer.c | 14 +++++++-------
18 files changed, 68 insertions(+), 55 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index aee4c708637e..845596888e53 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2313,6 +2313,16 @@ i915_gem_object_get(struct drm_i915_gem_object *obj)
__deprecated
extern void drm_gem_object_reference(struct drm_gem_object *);
+__attribute__((nonnull))
+static inline void
+i915_gem_object_put(struct drm_i915_gem_object *obj)
+{
+ drm_gem_object_unreference(&obj->base);
+}
+
+__deprecated
+extern void drm_gem_object_unreference(struct drm_gem_object *);
+
static inline bool
i915_gem_object_has_struct_page(const struct drm_i915_gem_object *obj)
{
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index e2881ae12b93..a42cf4fc3e31 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -295,7 +295,7 @@ drop_pages(struct drm_i915_gem_object *obj)
break;
ret = i915_gem_object_put_pages(obj);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
return ret;
}
@@ -887,7 +887,7 @@ i915_gem_pread_ioctl(struct drm_device *dev, void *data,
args->offset, args->data_ptr);
out:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
@@ -1320,7 +1320,7 @@ i915_gem_pwrite_ioctl(struct drm_device *dev, void *data,
}
out:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
put_rpm:
@@ -1522,7 +1522,7 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
intel_fb_obj_invalidate(obj, write_origin(obj, write_domain));
unref:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
@@ -1556,7 +1556,7 @@ i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
if (obj->pin_display)
i915_gem_object_flush_cpu_write_domain(obj);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
@@ -1987,7 +1987,7 @@ i915_gem_mmap_gtt(struct drm_file *file,
*offset = drm_vma_node_offset_addr(&obj->base.vma_node);
out:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
@@ -2411,7 +2411,7 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
}
i915_gem_request_assign(&obj->last_fenced_req, NULL);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
}
static bool i915_context_is_banned(const struct i915_gem_context *ctx)
@@ -2809,7 +2809,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
goto out;
}
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
for (i = 0; i < I915_NUM_ENGINES; i++) {
if (obj->last_read_req[i] == NULL)
@@ -2830,7 +2830,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
return ret;
out:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
mutex_unlock(&dev->struct_mutex);
return ret;
}
@@ -3660,7 +3660,7 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,
ret = i915_gem_object_set_cache_level(obj, level);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
rpm_put:
@@ -4052,7 +4052,7 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
}
unref:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
@@ -4116,7 +4116,7 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
args->retained = obj->madv != __I915_MADV_PURGED;
out:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
return ret;
@@ -4960,6 +4960,6 @@ i915_gem_object_create_from_data(struct drm_device *dev,
return obj;
fail:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
return ERR_PTR(ret);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_batch_pool.c b/drivers/gpu/drm/i915/i915_gem_batch_pool.c
index 3752d5daa4b2..3507b2753fd3 100644
--- a/drivers/gpu/drm/i915/i915_gem_batch_pool.c
+++ b/drivers/gpu/drm/i915/i915_gem_batch_pool.c
@@ -75,7 +75,7 @@ void i915_gem_batch_pool_fini(struct i915_gem_batch_pool *pool)
batch_pool_link);
list_del(&obj->batch_pool_link);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
}
}
}
@@ -121,7 +121,7 @@ i915_gem_batch_pool_get(struct i915_gem_batch_pool *pool,
/* While we're looping, do some clean up */
if (tmp->madv == __I915_MADV_PURGED) {
list_del(&tmp->batch_pool_link);
- drm_gem_object_unreference(&tmp->base);
+ i915_gem_object_put(tmp);
continue;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 3f151b12893b..6ad82ab35145 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -176,7 +176,7 @@ void i915_gem_context_free(struct kref *ctx_ref)
if (ce->ringbuf)
intel_ringbuffer_free(ce->ringbuf);
- drm_gem_object_unreference(&ce->state->base);
+ i915_gem_object_put(ce->state);
}
list_del(&ctx->link);
@@ -216,7 +216,7 @@ i915_gem_alloc_context_obj(struct drm_device *dev, size_t size)
ret = i915_gem_object_set_cache_level(obj, I915_CACHE_L3_LLC);
/* Failure shouldn't ever happen this early */
if (WARN_ON(ret)) {
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
return ERR_PTR(ret);
}
}
diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c
index d5777a0750f0..5a02c32e9ae6 100644
--- a/drivers/gpu/drm/i915/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/i915_gem_evict.c
@@ -222,17 +222,18 @@ found:
/* Unbinding will emit any required flushes */
while (!list_empty(&eviction_list)) {
- struct drm_gem_object *obj;
+ struct drm_i915_gem_object *obj;
+
vma = list_first_entry(&eviction_list,
struct i915_vma,
exec_list);
- obj = &vma->obj->base;
+ obj = vma->obj;
list_del_init(&vma->exec_list);
if (ret == 0)
ret = i915_vma_unbind(vma);
- drm_gem_object_unreference(obj);
+ i915_gem_object_put(obj);
}
return ret;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index e61751beabf8..da541d3c52eb 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -179,7 +179,7 @@ err:
struct drm_i915_gem_object,
obj_exec_link);
list_del_init(&obj->obj_exec_link);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
}
/*
* Objects already transfered to the vmas list will be unreferenced by
@@ -238,7 +238,7 @@ static void eb_destroy(struct eb_vmas *eb)
exec_list);
list_del_init(&vma->exec_list);
i915_gem_execbuffer_unreserve_vma(vma);
- drm_gem_object_unreference(&vma->obj->base);
+ i915_gem_object_put(vma->obj);
}
kfree(eb);
}
@@ -847,7 +847,7 @@ i915_gem_execbuffer_relocate_slow(struct drm_device *dev,
vma = list_first_entry(&eb->vmas, struct i915_vma, exec_list);
list_del_init(&vma->exec_list);
i915_gem_execbuffer_unreserve_vma(vma);
- drm_gem_object_unreference(&vma->obj->base);
+ i915_gem_object_put(vma->obj);
}
mutex_unlock(&dev->struct_mutex);
diff --git a/drivers/gpu/drm/i915/i915_gem_render_state.c b/drivers/gpu/drm/i915/i915_gem_render_state.c
index f75bbd67a13a..b2be4676a5cf 100644
--- a/drivers/gpu/drm/i915/i915_gem_render_state.c
+++ b/drivers/gpu/drm/i915/i915_gem_render_state.c
@@ -70,7 +70,7 @@ static int render_state_init(struct render_state *so,
return 0;
free_gem:
- drm_gem_object_unreference(&so->obj->base);
+ i915_gem_object_put(so->obj);
return ret;
}
@@ -195,7 +195,7 @@ err_out:
void i915_gem_render_state_fini(struct render_state *so)
{
i915_gem_object_ggtt_unpin(so->obj);
- drm_gem_object_unreference(&so->obj->base);
+ i915_gem_object_put(so->obj);
}
int i915_gem_render_state_prepare(struct intel_engine_cs *engine,
diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
index 3c9d771d499b..e93a837cf2a1 100644
--- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
@@ -201,7 +201,7 @@ i915_gem_shrink(struct drm_i915_private *dev_priv,
if (i915_gem_object_put_pages(obj) == 0)
count += obj->base.size >> PAGE_SHIFT;
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
}
list_splice(&still_in_list, phase->list);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c
index 66be299a1486..310756c30723 100644
--- a/drivers/gpu/drm/i915/i915_gem_stolen.c
+++ b/drivers/gpu/drm/i915/i915_gem_stolen.c
@@ -716,6 +716,6 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_device *dev,
return obj;
err:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
return NULL;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
index 46e80f30afcd..2d9d190bd6ef 100644
--- a/drivers/gpu/drm/i915/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
@@ -268,7 +268,7 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
}
err:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
mutex_unlock(&dev->struct_mutex);
intel_runtime_pm_put(dev_priv);
@@ -328,7 +328,7 @@ i915_gem_get_tiling(struct drm_device *dev, void *data,
if (args->swizzle_mode == I915_BIT_6_SWIZZLE_9_10_17)
args->swizzle_mode = I915_BIT_6_SWIZZLE_9_10;
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
mutex_unlock(&dev->struct_mutex);
return 0;
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index c41bf74f926e..cd4af22b8c59 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -119,7 +119,7 @@ static void cancel_userptr(struct work_struct *work)
dev_priv->mm.interruptible = was_interruptible;
}
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
mutex_unlock(&dev->struct_mutex);
}
@@ -577,7 +577,7 @@ __i915_gem_userptr_get_pages_worker(struct work_struct *_work)
}
obj->userptr.workers--;
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
mutex_unlock(&dev->struct_mutex);
release_pages(pvec, pinned, 0);
diff --git a/drivers/gpu/drm/i915/i915_guc_submission.c b/drivers/gpu/drm/i915/i915_guc_submission.c
index 1cc5de129e76..01c1c1671811 100644
--- a/drivers/gpu/drm/i915/i915_guc_submission.c
+++ b/drivers/gpu/drm/i915/i915_guc_submission.c
@@ -633,13 +633,13 @@ gem_allocate_guc_obj(struct drm_i915_private *dev_priv, u32 size)
return NULL;
if (i915_gem_object_get_pages(obj)) {
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
return NULL;
}
if (i915_gem_obj_ggtt_pin(obj, PAGE_SIZE,
PIN_OFFSET_BIAS | GUC_WOPCM_TOP)) {
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
return NULL;
}
@@ -661,7 +661,7 @@ static void gem_release_guc_obj(struct drm_i915_gem_object *obj)
if (i915_gem_obj_is_pinned(obj))
i915_gem_object_ggtt_unpin(obj);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
}
static void
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index bde2bf838d64..9a7a9ad9be9d 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -2542,7 +2542,7 @@ intel_alloc_initial_plane_obj(struct intel_crtc *crtc,
return true;
out_unref_obj:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
mutex_unlock(&dev->struct_mutex);
return false;
}
@@ -11040,7 +11040,7 @@ static void intel_unpin_work_fn(struct work_struct *__work)
mutex_lock(&dev->struct_mutex);
intel_unpin_fb_obj(work->old_fb, primary->state->rotation);
- drm_gem_object_unreference(&work->pending_flip_obj->base);
+ i915_gem_object_put(work->pending_flip_obj);
mutex_unlock(&dev->struct_mutex);
i915_gem_request_put(work->flip_queued_req);
@@ -14896,7 +14896,7 @@ static void intel_user_framebuffer_destroy(struct drm_framebuffer *fb)
drm_framebuffer_cleanup(fb);
mutex_lock(&dev->struct_mutex);
WARN_ON(!intel_fb->obj->framebuffer_references--);
- drm_gem_object_unreference(&intel_fb->obj->base);
+ i915_gem_object_put(intel_fb->obj);
mutex_unlock(&dev->struct_mutex);
kfree(intel_fb);
}
diff --git a/drivers/gpu/drm/i915/intel_fbdev.c b/drivers/gpu/drm/i915/intel_fbdev.c
index 86b00c6db1a6..fde36d3af1ad 100644
--- a/drivers/gpu/drm/i915/intel_fbdev.c
+++ b/drivers/gpu/drm/i915/intel_fbdev.c
@@ -159,7 +159,7 @@ static int intelfb_alloc(struct drm_fb_helper *helper,
fb = __intel_framebuffer_create(dev, &mode_cmd, obj);
if (IS_ERR(fb)) {
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
ret = PTR_ERR(fb);
goto out;
}
diff --git a/drivers/gpu/drm/i915/intel_guc_loader.c b/drivers/gpu/drm/i915/intel_guc_loader.c
index 605c69658d2c..75a1496ceb6f 100644
--- a/drivers/gpu/drm/i915/intel_guc_loader.c
+++ b/drivers/gpu/drm/i915/intel_guc_loader.c
@@ -662,7 +662,7 @@ fail:
mutex_lock(&dev->struct_mutex);
obj = guc_fw->guc_fw_obj;
if (obj)
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
guc_fw->guc_fw_obj = NULL;
mutex_unlock(&dev->struct_mutex);
@@ -737,13 +737,15 @@ void intel_guc_fini(struct drm_device *dev)
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_guc_fw *guc_fw = &dev_priv->guc.guc_fw;
+ if (!guc_fw->guc_fw_obj)
+ return;
+
mutex_lock(&dev->struct_mutex);
direct_interrupts_to_host(dev_priv);
i915_guc_submission_disable(dev_priv);
i915_guc_submission_fini(dev_priv);
- if (guc_fw->guc_fw_obj)
- drm_gem_object_unreference(&guc_fw->guc_fw_obj->base);
+ i915_gem_object_put(guc_fw->guc_fw_obj);
guc_fw->guc_fw_obj = NULL;
mutex_unlock(&dev->struct_mutex);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index e55a78c1250c..edd7b0f9bef3 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1366,7 +1366,7 @@ static int lrc_setup_wa_ctx_obj(struct intel_engine_cs *engine, u32 size)
if (ret) {
DRM_DEBUG_DRIVER("pin LRC WA ctx backing obj failed: %d\n",
ret);
- drm_gem_object_unreference(&engine->wa_ctx.obj->base);
+ i915_gem_object_put(engine->wa_ctx.obj);
return ret;
}
@@ -1377,7 +1377,7 @@ static void lrc_destroy_wa_ctx_obj(struct intel_engine_cs *engine)
{
if (engine->wa_ctx.obj) {
i915_gem_object_ggtt_unpin(engine->wa_ctx.obj);
- drm_gem_object_unreference(&engine->wa_ctx.obj->base);
+ i915_gem_object_put(engine->wa_ctx.obj);
engine->wa_ctx.obj = NULL;
}
}
@@ -2514,7 +2514,7 @@ static int execlists_context_deferred_alloc(struct i915_gem_context *ctx,
error_ringbuf:
intel_ringbuffer_free(ringbuf);
error_deref_obj:
- drm_gem_object_unreference(&ctx_obj->base);
+ i915_gem_object_put(ctx_obj);
ce->ringbuf = NULL;
ce->state = NULL;
return ret;
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 5ca797b01ccb..30d2f5db7d87 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -308,7 +308,7 @@ static void intel_overlay_release_old_vid_tail(struct intel_overlay *overlay)
struct drm_i915_gem_object *obj = overlay->old_vid_bo;
i915_gem_object_ggtt_unpin(obj);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
overlay->old_vid_bo = NULL;
}
@@ -322,7 +322,7 @@ static void intel_overlay_off_tail(struct intel_overlay *overlay)
return;
i915_gem_object_ggtt_unpin(obj);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
overlay->vid_bo = NULL;
overlay->crtc->overlay = NULL;
@@ -1219,7 +1219,7 @@ int intel_overlay_put_image_ioctl(struct drm_device *dev, void *data,
out_unlock:
mutex_unlock(&dev->struct_mutex);
drm_modeset_unlock_all(dev);
- drm_gem_object_unreference_unlocked(&new_bo->base);
+ i915_gem_object_put(new_bo);
out_free:
kfree(params);
@@ -1443,7 +1443,7 @@ out_unpin_bo:
if (!OVERLAY_NEEDS_PHYSICAL(dev_priv))
i915_gem_object_ggtt_unpin(reg_bo);
out_free_bo:
- drm_gem_object_unreference(®_bo->base);
+ i915_gem_object_put(reg_bo);
out_free:
mutex_unlock(&dev_priv->drm.struct_mutex);
kfree(overlay);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index ac079dc41c23..0d407541778f 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -646,7 +646,7 @@ void intel_fini_pipe_control(struct intel_engine_cs *engine)
return;
i915_gem_object_ggtt_unpin(engine->scratch.obj);
- drm_gem_object_unreference(&engine->scratch.obj->base);
+ i915_gem_object_put(engine->scratch.obj);
engine->scratch.obj = NULL;
}
@@ -677,7 +677,7 @@ int intel_init_pipe_control(struct intel_engine_cs *engine, int size)
return 0;
err_unref:
- drm_gem_object_unreference(&engine->scratch.obj->base);
+ i915_gem_object_put(engine->scratch.obj);
err:
return ret;
}
@@ -1316,7 +1316,7 @@ static void render_ring_cleanup(struct intel_engine_cs *engine)
if (dev_priv->semaphore_obj) {
i915_gem_object_ggtt_unpin(dev_priv->semaphore_obj);
- drm_gem_object_unreference(&dev_priv->semaphore_obj->base);
+ i915_gem_object_put(dev_priv->semaphore_obj);
dev_priv->semaphore_obj = NULL;
}
@@ -1902,7 +1902,7 @@ static void cleanup_status_page(struct intel_engine_cs *engine)
kunmap(sg_page(obj->pages->sgl));
i915_gem_object_ggtt_unpin(obj);
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
engine->status_page.obj = NULL;
}
@@ -1940,7 +1940,7 @@ static int init_status_page(struct intel_engine_cs *engine)
ret = i915_gem_obj_ggtt_pin(obj, 4096, flags);
if (ret) {
err_unref:
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
return ret;
}
@@ -2043,7 +2043,7 @@ err_unpin:
static void intel_destroy_ringbuffer_obj(struct intel_ringbuffer *ringbuf)
{
- drm_gem_object_unreference(&ringbuf->obj->base);
+ i915_gem_object_put(ringbuf->obj);
ringbuf->obj = NULL;
}
@@ -2705,7 +2705,7 @@ static void intel_ring_init_semaphores(struct drm_i915_private *dev_priv,
i915_gem_object_set_cache_level(obj, I915_CACHE_LLC);
ret = i915_gem_obj_ggtt_pin(obj, 0, PIN_NONBLOCK);
if (ret != 0) {
- drm_gem_object_unreference(&obj->base);
+ i915_gem_object_put(obj);
DRM_ERROR("Failed to pin semaphore bo. Disabling semaphores\n");
i915.semaphores = 0;
} else {
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 21/64] drm/i915: Rename drm_gem_object_unreference_unlocked in preparation for lockless free
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (18 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 20/64] drm/i915: Rename drm_gem_object_unreference in preparation for lockless free Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 22/64] drm/i915: Treat ringbuffer writes as write to normal memory Chris Wilson
` (43 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 10 ++++++++++
drivers/gpu/drm/i915/i915_gem.c | 10 +++++-----
drivers/gpu/drm/i915/i915_gem_tiling.c | 2 +-
drivers/gpu/drm/i915/i915_gem_userptr.c | 2 +-
drivers/gpu/drm/i915/intel_display.c | 6 +++---
drivers/gpu/drm/i915/intel_overlay.c | 2 +-
drivers/gpu/drm/i915/intel_pm.c | 2 +-
7 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 845596888e53..5e9323c1c6a7 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2323,6 +2323,16 @@ i915_gem_object_put(struct drm_i915_gem_object *obj)
__deprecated
extern void drm_gem_object_unreference(struct drm_gem_object *);
+__attribute__((nonnull))
+static inline void
+i915_gem_object_put_unlocked(struct drm_i915_gem_object *obj)
+{
+ drm_gem_object_unreference_unlocked(&obj->base);
+}
+
+__deprecated
+extern void drm_gem_object_unreference_unlocked(struct drm_gem_object *);
+
static inline bool
i915_gem_object_has_struct_page(const struct drm_i915_gem_object *obj)
{
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a42cf4fc3e31..5e7cc01b5673 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -410,7 +410,7 @@ i915_gem_create(struct drm_file *file,
ret = drm_gem_handle_create(file, &obj->base, &handle);
/* drop reference from allocate - handle holds it now */
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
if (ret)
return ret;
@@ -1604,7 +1604,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
* pages from.
*/
if (!obj->base.filp) {
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
return -EINVAL;
}
@@ -1616,7 +1616,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
struct vm_area_struct *vma;
if (down_write_killable(&mm->mmap_sem)) {
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
return -EINTR;
}
vma = find_vma(mm, addr);
@@ -1630,7 +1630,7 @@ i915_gem_mmap_ioctl(struct drm_device *dev, void *data,
/* This may race, but that's ok, it only gets set */
WRITE_ONCE(obj->has_wc_mmap, true);
}
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
if (IS_ERR((void *)addr))
return addr;
@@ -3610,7 +3610,7 @@ int i915_gem_get_caching_ioctl(struct drm_device *dev, void *data,
break;
}
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
index 2d9d190bd6ef..e83fc2d0433c 100644
--- a/drivers/gpu/drm/i915/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
@@ -172,7 +172,7 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
if (!i915_tiling_ok(dev,
args->stride, obj->base.size, args->tiling_mode)) {
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
return -EINVAL;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index cd4af22b8c59..ca8b82ab93d6 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -845,7 +845,7 @@ i915_gem_userptr_ioctl(struct drm_device *dev, void *data, struct drm_file *file
ret = drm_gem_handle_create(file, &obj->base, &handle);
/* drop reference from allocate - handle holds it now */
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 9a7a9ad9be9d..a67af8df25ae 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -10529,7 +10529,7 @@ intel_framebuffer_create_for_mode(struct drm_device *dev,
fb = intel_framebuffer_create(dev, &mode_cmd, obj);
if (IS_ERR(fb))
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
return fb;
}
@@ -11843,7 +11843,7 @@ cleanup:
crtc->primary->fb = old_fb;
update_state_fb(crtc->primary);
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
drm_framebuffer_unreference(work->old_fb);
spin_lock_irq(&dev->event_lock);
@@ -15139,7 +15139,7 @@ intel_user_framebuffer_create(struct drm_device *dev,
fb = intel_framebuffer_create(dev, &mode_cmd, obj);
if (IS_ERR(fb))
- drm_gem_object_unreference_unlocked(&obj->base);
+ i915_gem_object_put_unlocked(obj);
return fb;
}
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 30d2f5db7d87..8654a323722e 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -1460,7 +1460,7 @@ void intel_cleanup_overlay(struct drm_i915_private *dev_priv)
* hardware should be off already */
WARN_ON(dev_priv->overlay->active);
- drm_gem_object_unreference_unlocked(&dev_priv->overlay->reg_bo->base);
+ i915_gem_object_put_unlocked(dev_priv->overlay->reg_bo);
kfree(dev_priv->overlay);
}
diff --git a/drivers/gpu/drm/i915/intel_pm.c b/drivers/gpu/drm/i915/intel_pm.c
index 960eaf5051ed..568926adf0e7 100644
--- a/drivers/gpu/drm/i915/intel_pm.c
+++ b/drivers/gpu/drm/i915/intel_pm.c
@@ -5757,7 +5757,7 @@ static void valleyview_cleanup_pctx(struct drm_i915_private *dev_priv)
if (WARN_ON(!dev_priv->vlv_pctx))
return;
- drm_gem_object_unreference_unlocked(&dev_priv->vlv_pctx->base);
+ i915_gem_object_put_unlocked(dev_priv->vlv_pctx);
dev_priv->vlv_pctx = NULL;
}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 22/64] drm/i915: Treat ringbuffer writes as write to normal memory
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (19 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 21/64] drm/i915: Rename drm_gem_object_unreference_unlocked " Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 23/64] drm/i915: Rename ring->virtual_start as ring->vaddr Chris Wilson
` (42 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Ringbuffers are now being written to either through LLC or WC paths, so
treating them as simply iomem is no longer adequate. However, for the
older !llc hardware, the hardware is documentated as treating the TAIL
register update as serialising, so we can relax the barriers when filling
the rings (but even if it were not, it is still an uncached register write
and so serialising anyway.).
For simplicity, let's ignore the iomem annotation.
v2: Remove iomem from ringbuffer->virtual_address
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
drivers/gpu/drm/i915/intel_lrc.h | 6 +++---
drivers/gpu/drm/i915/intel_ringbuffer.h | 28 ++++++++++++++++++++--------
2 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index 2b8255c19dcc..a19dc49e89da 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -78,8 +78,9 @@ int logical_ring_flush_all_caches(struct drm_i915_gem_request *req);
*/
static inline void intel_logical_ring_advance(struct intel_ringbuffer *ringbuf)
{
- ringbuf->tail &= ringbuf->size - 1;
+ __intel_ringbuffer_advance(ringbuf);
}
+
/**
* intel_logical_ring_emit() - write a DWORD to the ringbuffer.
* @ringbuf: Ringbuffer to write to.
@@ -88,8 +89,7 @@ static inline void intel_logical_ring_advance(struct intel_ringbuffer *ringbuf)
static inline void intel_logical_ring_emit(struct intel_ringbuffer *ringbuf,
u32 data)
{
- iowrite32(data, ringbuf->virtual_start + ringbuf->tail);
- ringbuf->tail += 4;
+ __intel_ringbuffer_emit(ringbuf, data);
}
static inline void intel_logical_ring_emit_reg(struct intel_ringbuffer *ringbuf,
i915_reg_t reg)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 3d47e23991cc..7f32edb29396 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -84,7 +84,7 @@ struct intel_ring_hangcheck {
struct intel_ringbuffer {
struct drm_i915_gem_object *obj;
- void __iomem *virtual_start;
+ void *virtual_start;
struct i915_vma *vma;
struct intel_engine_cs *engine;
@@ -452,23 +452,35 @@ int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
-static inline void intel_ring_emit(struct intel_engine_cs *engine,
- u32 data)
+
+static inline void __intel_ringbuffer_emit(struct intel_ringbuffer *rb,
+ u32 data)
+{
+ *(uint32_t *)(rb->virtual_start + rb->tail) = data;
+ rb->tail += 4;
+}
+
+static inline void __intel_ringbuffer_advance(struct intel_ringbuffer *rb)
+{
+ rb->tail &= rb->size - 1;
+}
+
+static inline void intel_ring_emit(struct intel_engine_cs *engine, u32 data)
{
- struct intel_ringbuffer *ringbuf = engine->buffer;
- iowrite32(data, ringbuf->virtual_start + ringbuf->tail);
- ringbuf->tail += 4;
+ __intel_ringbuffer_emit(engine->buffer, data);
}
+
static inline void intel_ring_emit_reg(struct intel_engine_cs *engine,
i915_reg_t reg)
{
intel_ring_emit(engine, i915_mmio_reg_offset(reg));
}
+
static inline void intel_ring_advance(struct intel_engine_cs *engine)
{
- struct intel_ringbuffer *ringbuf = engine->buffer;
- ringbuf->tail &= ringbuf->size - 1;
+ __intel_ringbuffer_advance(engine->buffer);
}
+
int __intel_ring_space(int head, int tail, int size);
void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 23/64] drm/i915: Rename ring->virtual_start as ring->vaddr
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (20 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 22/64] drm/i915: Treat ringbuffer writes as write to normal memory Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 24/64] drm/i915: Convert i915_semaphores_is_enabled over to early sanitize Chris Wilson
` (41 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Just a different colour to better match virtual addresses elsewhere.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_irq.c | 8 ++++----
drivers/gpu/drm/i915/intel_ringbuffer.c | 11 +++++------
drivers/gpu/drm/i915/intel_ringbuffer.h | 4 ++--
3 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c
index a69a5fc75bd6..ddd6bb6ef165 100644
--- a/drivers/gpu/drm/i915/i915_irq.c
+++ b/drivers/gpu/drm/i915/i915_irq.c
@@ -2907,7 +2907,7 @@ semaphore_waits_for(struct intel_engine_cs *engine, u32 *seqno)
head &= engine->buffer->size - 1;
/* This here seems to blow up */
- cmd = ioread32(engine->buffer->virtual_start + head);
+ cmd = ioread32(engine->buffer->vaddr + head);
if (cmd == ipehr)
break;
@@ -2917,11 +2917,11 @@ semaphore_waits_for(struct intel_engine_cs *engine, u32 *seqno)
if (!i)
return NULL;
- *seqno = ioread32(engine->buffer->virtual_start + head + 4) + 1;
+ *seqno = ioread32(engine->buffer->vaddr + head + 4) + 1;
if (INTEL_GEN(dev_priv) >= 8) {
- offset = ioread32(engine->buffer->virtual_start + head + 12);
+ offset = ioread32(engine->buffer->vaddr + head + 12);
offset <<= 32;
- offset = ioread32(engine->buffer->virtual_start + head + 8);
+ offset = ioread32(engine->buffer->vaddr + head + 8);
}
return semaphore_wait_to_signaller_ring(engine, ipehr, offset);
}
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 0d407541778f..15157d3b20e3 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1976,14 +1976,14 @@ static int init_phys_status_page(struct intel_engine_cs *engine)
void intel_unpin_ringbuffer_obj(struct intel_ringbuffer *ringbuf)
{
- GEM_BUG_ON(ringbuf->vma == NULL);
- GEM_BUG_ON(ringbuf->virtual_start == NULL);
+ GEM_BUG_ON(!ringbuf->vma);
+ GEM_BUG_ON(!ringbuf->vaddr);
if (HAS_LLC(ringbuf->obj->base.dev) && !ringbuf->obj->stolen)
i915_gem_object_unpin_map(ringbuf->obj);
else
i915_vma_unpin_iomap(ringbuf->vma);
- ringbuf->virtual_start = NULL;
+ ringbuf->vaddr = NULL;
i915_gem_object_ggtt_unpin(ringbuf->obj);
ringbuf->vma = NULL;
@@ -2032,7 +2032,7 @@ int intel_pin_and_map_ringbuffer_obj(struct drm_i915_private *dev_priv,
}
}
- ringbuf->virtual_start = addr;
+ ringbuf->vaddr = addr;
ringbuf->vma = i915_gem_obj_to_ggtt(obj);
return 0;
@@ -2404,8 +2404,7 @@ int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
GEM_BUG_ON(ringbuf->tail + remain_actual > ringbuf->size);
/* Fill the tail with MI_NOOP */
- memset(ringbuf->virtual_start + ringbuf->tail,
- 0, remain_actual);
+ memset(ringbuf->vaddr + ringbuf->tail, 0, remain_actual);
ringbuf->tail = 0;
ringbuf->space -= remain_actual;
}
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 7f32edb29396..34d79b8fcb7d 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -84,7 +84,7 @@ struct intel_ring_hangcheck {
struct intel_ringbuffer {
struct drm_i915_gem_object *obj;
- void *virtual_start;
+ void *vaddr;
struct i915_vma *vma;
struct intel_engine_cs *engine;
@@ -456,7 +456,7 @@ int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
static inline void __intel_ringbuffer_emit(struct intel_ringbuffer *rb,
u32 data)
{
- *(uint32_t *)(rb->virtual_start + rb->tail) = data;
+ *(uint32_t *)(rb->vaddr + rb->tail) = data;
rb->tail += 4;
}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 24/64] drm/i915: Convert i915_semaphores_is_enabled over to early sanitize
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (21 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 23/64] drm/i915: Rename ring->virtual_start as ring->vaddr Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 25/64] drm/i915: Unify intel_logical_ring_emit and intel_ring_emit Chris Wilson
` (40 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Rather than recomputing whether semaphores are enabled, we can do that
computation once during early initialisation as the i915.semaphores
module parameter is now read-only.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 2 +-
drivers/gpu/drm/i915/i915_drv.c | 25 +++----------------------
drivers/gpu/drm/i915/i915_drv.h | 3 ++-
drivers/gpu/drm/i915/i915_gem.c | 23 ++++++++++++++++++++++-
drivers/gpu/drm/i915/i915_gem_context.c | 2 +-
drivers/gpu/drm/i915/i915_gpu_error.c | 2 +-
drivers/gpu/drm/i915/intel_ringbuffer.c | 6 +++---
7 files changed, 33 insertions(+), 30 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 78819b6e6f8e..72950b9b3e9c 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -3218,7 +3218,7 @@ static int i915_semaphore_status(struct seq_file *m, void *unused)
enum intel_engine_id id;
int j, ret;
- if (!i915_semaphore_is_enabled(dev_priv)) {
+ if (!i915.semaphores) {
seq_puts(m, "Semaphores are disabled\n");
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 2739603d5862..5f2f064b3a48 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -224,27 +224,6 @@ static void intel_detect_pch(struct drm_device *dev)
pci_dev_put(pch);
}
-bool i915_semaphore_is_enabled(struct drm_i915_private *dev_priv)
-{
- if (INTEL_GEN(dev_priv) < 6)
- return false;
-
- if (i915.semaphores >= 0)
- return i915.semaphores;
-
- /* TODO: make semaphores and Execlists play nicely together */
- if (i915.enable_execlists)
- return false;
-
-#ifdef CONFIG_INTEL_IOMMU
- /* Enable semaphores on SNB when IO remapping is off */
- if (IS_GEN6(dev_priv) && intel_iommu_gfx_mapped)
- return false;
-#endif
-
- return true;
-}
-
static int i915_getparam(struct drm_device *dev, void *data,
struct drm_file *file_priv)
{
@@ -320,7 +299,7 @@ static int i915_getparam(struct drm_device *dev, void *data,
value = 1;
break;
case I915_PARAM_HAS_SEMAPHORES:
- value = i915_semaphore_is_enabled(dev_priv);
+ value = i915.semaphores;
break;
case I915_PARAM_HAS_PRIME_VMAP_FLUSH:
value = 1;
@@ -995,6 +974,8 @@ static void intel_sanitize_options(struct drm_i915_private *dev_priv)
i915.enable_ppgtt =
intel_sanitize_enable_ppgtt(dev_priv, i915.enable_ppgtt);
DRM_DEBUG_DRIVER("ppgtt mode: %i\n", i915.enable_ppgtt);
+
+ i915.semaphores = intel_sanitize_semaphores(dev_priv, i915.semaphores);
}
/**
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 5e9323c1c6a7..0991e59a5d37 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2798,6 +2798,8 @@ extern int i915_resume_switcheroo(struct drm_device *dev);
int intel_sanitize_enable_ppgtt(struct drm_i915_private *dev_priv,
int enable_ppgtt);
+bool intel_sanitize_semaphores(struct drm_i915_private *dev_priv, int value);
+
/* i915_drv.c */
void __printf(3, 4)
__i915_printk(struct drm_i915_private *dev_priv, const char *level,
@@ -3623,7 +3625,6 @@ extern void intel_set_rps(struct drm_i915_private *dev_priv, u8 val);
extern void intel_set_memory_cxsr(struct drm_i915_private *dev_priv,
bool enable);
-extern bool i915_semaphore_is_enabled(struct drm_i915_private *dev_priv);
int i915_reg_read_ioctl(struct drm_device *dev, void *data,
struct drm_file *file);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 5e7cc01b5673..337282c9a070 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2851,7 +2851,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (i915_gem_request_completed(from_req))
return 0;
- if (!i915_semaphore_is_enabled(to_i915(obj->base.dev))) {
+ if (!i915.semaphores) {
struct drm_i915_private *i915 = to_i915(obj->base.dev);
ret = __i915_wait_request(from_req,
i915->mm.interruptible,
@@ -4543,6 +4543,27 @@ out:
return ret;
}
+bool intel_sanitize_semaphores(struct drm_i915_private *dev_priv, int value)
+{
+ if (INTEL_INFO(dev_priv)->gen < 6)
+ return false;
+
+ /* TODO: make semaphores and Execlists play nicely together */
+ if (i915.enable_execlists)
+ return false;
+
+ if (value >= 0)
+ return value;
+
+#ifdef CONFIG_INTEL_IOMMU
+ /* Enable semaphores on SNB when IO remapping is off */
+ if (INTEL_INFO(dev_priv)->gen == 6 && intel_iommu_gfx_mapped)
+ return false;
+#endif
+
+ return true;
+}
+
int i915_gem_init(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = to_i915(dev);
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 6ad82ab35145..847325987013 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -556,7 +556,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
u32 flags = hw_flags | MI_MM_SPACE_GTT;
const int num_rings =
/* Use an extended w/a on ivb+ if signalling from other rings */
- i915_semaphore_is_enabled(dev_priv) ?
+ i915.semaphores ?
hweight32(INTEL_INFO(dev_priv)->ring_mask) - 1 :
0;
int len, ret;
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 6daaf4ecd2da..4d39c7284605 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -863,7 +863,7 @@ static void gen8_record_semaphore_state(struct drm_i915_private *dev_priv,
struct intel_engine_cs *to;
enum intel_engine_id id;
- if (!i915_semaphore_is_enabled(dev_priv))
+ if (!i915.semaphores)
return;
if (!error->semaphore_obj)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 15157d3b20e3..1eaf368bcce9 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2692,7 +2692,7 @@ static void intel_ring_init_semaphores(struct drm_i915_private *dev_priv,
struct drm_i915_gem_object *obj;
int ret, i;
- if (!i915_semaphore_is_enabled(dev_priv))
+ if (!i915.semaphores)
return;
if (INTEL_GEN(dev_priv) >= 8 && !dev_priv->semaphore_obj) {
@@ -2713,7 +2713,7 @@ static void intel_ring_init_semaphores(struct drm_i915_private *dev_priv,
}
}
- if (!i915_semaphore_is_enabled(dev_priv))
+ if (!i915.semaphores)
return;
if (INTEL_GEN(dev_priv) >= 8) {
@@ -2857,7 +2857,7 @@ int intel_init_render_ring_buffer(struct drm_device *dev)
engine->init_context = intel_rcs_ctx_init;
engine->add_request = gen8_render_add_request;
engine->flush = gen8_render_ring_flush;
- if (i915_semaphore_is_enabled(dev_priv))
+ if (i915.semaphores)
engine->semaphore.signal = gen8_rcs_signal;
} else if (INTEL_GEN(dev_priv) >= 6) {
engine->init_context = intel_rcs_ctx_init;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 25/64] drm/i915: Unify intel_logical_ring_emit and intel_ring_emit
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (22 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 24/64] drm/i915: Convert i915_semaphores_is_enabled over to early sanitize Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 26/64] drm/i915: Rename request->ringbuf to request->ring Chris Wilson
` (39 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Both perform the same actions with more or less indirection, so just
unify the code.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_context.c | 54 ++--
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 53 ++--
drivers/gpu/drm/i915/i915_gem_gtt.c | 62 ++---
drivers/gpu/drm/i915/intel_display.c | 80 +++---
drivers/gpu/drm/i915/intel_lrc.c | 184 +++++++-------
drivers/gpu/drm/i915/intel_lrc.h | 26 --
drivers/gpu/drm/i915/intel_mocs.c | 38 ++-
drivers/gpu/drm/i915/intel_overlay.c | 50 ++--
drivers/gpu/drm/i915/intel_ringbuffer.c | 387 +++++++++++++++--------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 25 +-
10 files changed, 463 insertions(+), 496 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 847325987013..cdf6528b0687 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -552,7 +552,7 @@ static inline int
mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
{
struct drm_i915_private *dev_priv = req->i915;
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
u32 flags = hw_flags | MI_MM_SPACE_GTT;
const int num_rings =
/* Use an extended w/a on ivb+ if signalling from other rings */
@@ -567,7 +567,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
* itlb_before_ctx_switch.
*/
if (IS_GEN6(dev_priv)) {
- ret = engine->flush(req, I915_GEM_GPU_DOMAINS, 0);
+ ret = req->engine->flush(req, I915_GEM_GPU_DOMAINS, 0);
if (ret)
return ret;
}
@@ -589,64 +589,64 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
/* WaProgramMiArbOnOffAroundMiSetContext:ivb,vlv,hsw,bdw,chv */
if (INTEL_GEN(dev_priv) >= 7) {
- intel_ring_emit(engine, MI_ARB_ON_OFF | MI_ARB_DISABLE);
+ intel_ring_emit(ring, MI_ARB_ON_OFF | MI_ARB_DISABLE);
if (num_rings) {
struct intel_engine_cs *signaller;
- intel_ring_emit(engine,
+ intel_ring_emit(ring,
MI_LOAD_REGISTER_IMM(num_rings));
for_each_engine(signaller, dev_priv) {
- if (signaller == engine)
+ if (signaller == req->engine)
continue;
- intel_ring_emit_reg(engine,
+ intel_ring_emit_reg(ring,
RING_PSMI_CTL(signaller->mmio_base));
- intel_ring_emit(engine,
+ intel_ring_emit(ring,
_MASKED_BIT_ENABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
}
}
}
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_emit(engine, MI_SET_CONTEXT);
- intel_ring_emit(engine,
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_SET_CONTEXT);
+ intel_ring_emit(ring,
i915_gem_obj_ggtt_offset(req->ctx->engine[RCS].state) |
flags);
/*
* w/a: MI_SET_CONTEXT must always be followed by MI_NOOP
* WaMiSetContext_Hang:snb,ivb,vlv
*/
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
if (INTEL_GEN(dev_priv) >= 7) {
if (num_rings) {
struct intel_engine_cs *signaller;
i915_reg_t last_reg = {}; /* keep gcc quiet */
- intel_ring_emit(engine,
+ intel_ring_emit(ring,
MI_LOAD_REGISTER_IMM(num_rings));
for_each_engine(signaller, dev_priv) {
- if (signaller == engine)
+ if (signaller == req->engine)
continue;
last_reg = RING_PSMI_CTL(signaller->mmio_base);
- intel_ring_emit_reg(engine, last_reg);
- intel_ring_emit(engine,
+ intel_ring_emit_reg(ring, last_reg);
+ intel_ring_emit(ring,
_MASKED_BIT_DISABLE(GEN6_PSMI_SLEEP_MSG_DISABLE));
}
/* Insert a delay before the next switch! */
- intel_ring_emit(engine,
+ intel_ring_emit(ring,
MI_STORE_REGISTER_MEM |
MI_SRM_LRM_GLOBAL_GTT);
- intel_ring_emit_reg(engine, last_reg);
- intel_ring_emit(engine, engine->scratch.gtt_offset);
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit_reg(ring, last_reg);
+ intel_ring_emit(ring, req->engine->scratch.gtt_offset);
+ intel_ring_emit(ring, MI_NOOP);
}
- intel_ring_emit(engine, MI_ARB_ON_OFF | MI_ARB_ENABLE);
+ intel_ring_emit(ring, MI_ARB_ON_OFF | MI_ARB_ENABLE);
}
- intel_ring_advance(engine);
+ intel_ring_advance(ring);
return ret;
}
@@ -654,7 +654,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
static int remap_l3(struct drm_i915_gem_request *req, int slice)
{
u32 *remap_info = req->i915->l3_parity.remap_info[slice];
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int i, ret;
if (!remap_info)
@@ -669,13 +669,13 @@ static int remap_l3(struct drm_i915_gem_request *req, int slice)
* here because no other code should access these registers other than
* at initialization time.
*/
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(GEN7_L3LOG_SIZE/4));
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(GEN7_L3LOG_SIZE/4));
for (i = 0; i < GEN7_L3LOG_SIZE/4; i++) {
- intel_ring_emit_reg(engine, GEN7_L3LOG(slice, i));
- intel_ring_emit(engine, remap_info[i]);
+ intel_ring_emit_reg(ring, GEN7_L3LOG(slice, i));
+ intel_ring_emit(ring, remap_info[i]);
}
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index da541d3c52eb..9c9763fec955 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1164,14 +1164,12 @@ i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params)
}
static int
-i915_reset_gen7_sol_offsets(struct drm_device *dev,
- struct drm_i915_gem_request *req)
+i915_reset_gen7_sol_offsets(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
- struct drm_i915_private *dev_priv = to_i915(dev);
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret, i;
- if (!IS_GEN7(dev) || engine != &dev_priv->engine[RCS]) {
+ if (!IS_GEN7(req->i915) || req->engine->id != RCS) {
DRM_DEBUG("sol reset is gen7/rcs only\n");
return -EINVAL;
}
@@ -1181,12 +1179,12 @@ i915_reset_gen7_sol_offsets(struct drm_device *dev,
return ret;
for (i = 0; i < 4; i++) {
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(engine, GEN7_SO_WRITE_OFFSET(i));
- intel_ring_emit(engine, 0);
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(ring, GEN7_SO_WRITE_OFFSET(i));
+ intel_ring_emit(ring, 0);
}
- intel_ring_advance(engine);
+ intel_ring_advance(ring);
return 0;
}
@@ -1249,9 +1247,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
struct drm_i915_gem_execbuffer2 *args,
struct list_head *vmas)
{
- struct drm_device *dev = params->dev;
- struct intel_engine_cs *engine = params->engine;
- struct drm_i915_private *dev_priv = to_i915(dev);
+ struct drm_i915_private *dev_priv = params->request->i915;
u64 exec_start, exec_len;
int instp_mode;
u32 instp_mask;
@@ -1265,34 +1261,31 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
if (ret)
return ret;
- WARN(params->ctx->ppgtt && params->ctx->ppgtt->pd_dirty_rings & (1<<engine->id),
- "%s didn't clear reload\n", engine->name);
-
instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
instp_mask = I915_EXEC_CONSTANTS_MASK;
switch (instp_mode) {
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
+ if (instp_mode != 0 && params->engine->id != RCS) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
if (instp_mode != dev_priv->relative_constants_mode) {
- if (INTEL_INFO(dev)->gen < 4) {
+ if (INTEL_INFO(dev_priv)->gen < 4) {
DRM_DEBUG("no rel constants on pre-gen4\n");
return -EINVAL;
}
- if (INTEL_INFO(dev)->gen > 5 &&
+ if (INTEL_INFO(dev_priv)->gen > 5 &&
instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
return -EINVAL;
}
/* The HW changed the meaning on this bit on gen6 */
- if (INTEL_INFO(dev)->gen >= 6)
+ if (INTEL_INFO(dev_priv)->gen >= 6)
instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
}
break;
@@ -1301,23 +1294,25 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
return -EINVAL;
}
- if (engine == &dev_priv->engine[RCS] &&
+ if (params->engine->id == RCS &&
instp_mode != dev_priv->relative_constants_mode) {
+ struct intel_ringbuffer *ring = params->request->ringbuf;
+
ret = intel_ring_begin(params->request, 4);
if (ret)
return ret;
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(engine, INSTPM);
- intel_ring_emit(engine, instp_mask << 16 | instp_mode);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(ring, INSTPM);
+ intel_ring_emit(ring, instp_mask << 16 | instp_mode);
+ intel_ring_advance(ring);
dev_priv->relative_constants_mode = instp_mode;
}
if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
- ret = i915_reset_gen7_sol_offsets(dev, params->request);
+ ret = i915_reset_gen7_sol_offsets(params->request);
if (ret)
return ret;
}
@@ -1329,9 +1324,9 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
if (exec_len == 0)
exec_len = params->batch_obj->base.size;
- ret = engine->dispatch_execbuffer(params->request,
- exec_start, exec_len,
- params->dispatch_flags);
+ ret = params->engine->dispatch_execbuffer(params->request,
+ exec_start, exec_len,
+ params->dispatch_flags);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 365fe65950e1..2dcc0da272e8 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -669,7 +669,7 @@ static int gen8_write_pdp(struct drm_i915_gem_request *req,
unsigned entry,
dma_addr_t addr)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
BUG_ON(entry >= 4);
@@ -678,13 +678,13 @@ static int gen8_write_pdp(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(engine, GEN8_RING_PDP_UDW(engine, entry));
- intel_ring_emit(engine, upper_32_bits(addr));
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(engine, GEN8_RING_PDP_LDW(engine, entry));
- intel_ring_emit(engine, lower_32_bits(addr));
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(ring, GEN8_RING_PDP_UDW(req->engine, entry));
+ intel_ring_emit(ring, upper_32_bits(addr));
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(ring, GEN8_RING_PDP_LDW(req->engine, entry));
+ intel_ring_emit(ring, lower_32_bits(addr));
+ intel_ring_advance(ring);
return 0;
}
@@ -1660,11 +1660,13 @@ static uint32_t get_pd_offset(struct i915_hw_ppgtt *ppgtt)
static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = engine->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
+ ret = req->engine->flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -1672,13 +1674,13 @@ static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
if (ret)
return ret;
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(2));
- intel_ring_emit_reg(engine, RING_PP_DIR_DCLV(engine));
- intel_ring_emit(engine, PP_DIR_DCLV_2G);
- intel_ring_emit_reg(engine, RING_PP_DIR_BASE(engine));
- intel_ring_emit(engine, get_pd_offset(ppgtt));
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(2));
+ intel_ring_emit_reg(ring, RING_PP_DIR_DCLV(req->engine));
+ intel_ring_emit(ring, PP_DIR_DCLV_2G);
+ intel_ring_emit_reg(ring, RING_PP_DIR_BASE(req->engine));
+ intel_ring_emit(ring, get_pd_offset(ppgtt));
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -1686,11 +1688,13 @@ static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = engine->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
+ ret = req->engine->flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -1698,17 +1702,19 @@ static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
if (ret)
return ret;
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(2));
- intel_ring_emit_reg(engine, RING_PP_DIR_DCLV(engine));
- intel_ring_emit(engine, PP_DIR_DCLV_2G);
- intel_ring_emit_reg(engine, RING_PP_DIR_BASE(engine));
- intel_ring_emit(engine, get_pd_offset(ppgtt));
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(2));
+ intel_ring_emit_reg(ring, RING_PP_DIR_DCLV(req->engine));
+ intel_ring_emit(ring, PP_DIR_DCLV_2G);
+ intel_ring_emit_reg(ring, RING_PP_DIR_BASE(req->engine));
+ intel_ring_emit(ring, get_pd_offset(ppgtt));
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
/* XXX: RCS is the only one to auto invalidate the TLBs? */
- if (engine->id != RCS) {
- ret = engine->flush(req, I915_GEM_GPU_DOMAINS, I915_GEM_GPU_DOMAINS);
+ if (req->engine->id != RCS) {
+ ret = req->engine->flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index a67af8df25ae..61bc2e96e180 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11210,7 +11210,7 @@ static int intel_gen2_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11226,13 +11226,13 @@ static int intel_gen2_queue_flip(struct drm_device *dev,
flip_mask = MI_WAIT_FOR_PLANE_B_FLIP;
else
flip_mask = MI_WAIT_FOR_PLANE_A_FLIP;
- intel_ring_emit(engine, MI_WAIT_FOR_EVENT | flip_mask);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_emit(engine, MI_DISPLAY_FLIP |
+ intel_ring_emit(ring, MI_WAIT_FOR_EVENT | flip_mask);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_DISPLAY_FLIP |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(engine, fb->pitches[0]);
- intel_ring_emit(engine, intel_crtc->flip_work->gtt_offset);
- intel_ring_emit(engine, 0); /* aux display base address, unused */
+ intel_ring_emit(ring, fb->pitches[0]);
+ intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
+ intel_ring_emit(ring, 0); /* aux display base address, unused */
return 0;
}
@@ -11244,7 +11244,7 @@ static int intel_gen3_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11257,13 +11257,13 @@ static int intel_gen3_queue_flip(struct drm_device *dev,
flip_mask = MI_WAIT_FOR_PLANE_B_FLIP;
else
flip_mask = MI_WAIT_FOR_PLANE_A_FLIP;
- intel_ring_emit(engine, MI_WAIT_FOR_EVENT | flip_mask);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_emit(engine, MI_DISPLAY_FLIP_I915 |
+ intel_ring_emit(ring, MI_WAIT_FOR_EVENT | flip_mask);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_DISPLAY_FLIP_I915 |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(engine, fb->pitches[0]);
- intel_ring_emit(engine, intel_crtc->flip_work->gtt_offset);
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, fb->pitches[0]);
+ intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
+ intel_ring_emit(ring, MI_NOOP);
return 0;
}
@@ -11275,7 +11275,7 @@ static int intel_gen4_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11289,10 +11289,10 @@ static int intel_gen4_queue_flip(struct drm_device *dev,
* Display Registers (which do not change across a page-flip)
* so we need only reprogram the base address.
*/
- intel_ring_emit(engine, MI_DISPLAY_FLIP |
+ intel_ring_emit(ring, MI_DISPLAY_FLIP |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(engine, fb->pitches[0]);
- intel_ring_emit(engine, intel_crtc->flip_work->gtt_offset |
+ intel_ring_emit(ring, fb->pitches[0]);
+ intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset |
obj->tiling_mode);
/* XXX Enabling the panel-fitter across page-flip is so far
@@ -11301,7 +11301,7 @@ static int intel_gen4_queue_flip(struct drm_device *dev,
*/
pf = 0;
pipesrc = I915_READ(PIPESRC(intel_crtc->pipe)) & 0x0fff0fff;
- intel_ring_emit(engine, pf | pipesrc);
+ intel_ring_emit(ring, pf | pipesrc);
return 0;
}
@@ -11313,7 +11313,7 @@ static int intel_gen6_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11323,10 +11323,10 @@ static int intel_gen6_queue_flip(struct drm_device *dev,
if (ret)
return ret;
- intel_ring_emit(engine, MI_DISPLAY_FLIP |
+ intel_ring_emit(ring, MI_DISPLAY_FLIP |
MI_DISPLAY_FLIP_PLANE(intel_crtc->plane));
- intel_ring_emit(engine, fb->pitches[0] | obj->tiling_mode);
- intel_ring_emit(engine, intel_crtc->flip_work->gtt_offset);
+ intel_ring_emit(ring, fb->pitches[0] | obj->tiling_mode);
+ intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
/* Contrary to the suggestions in the documentation,
* "Enable Panel Fitter" does not seem to be required when page
@@ -11336,7 +11336,7 @@ static int intel_gen6_queue_flip(struct drm_device *dev,
*/
pf = 0;
pipesrc = I915_READ(PIPESRC(intel_crtc->pipe)) & 0x0fff0fff;
- intel_ring_emit(engine, pf | pipesrc);
+ intel_ring_emit(ring, pf | pipesrc);
return 0;
}
@@ -11348,7 +11348,7 @@ static int intel_gen7_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t plane_bit = 0;
int len, ret;
@@ -11369,7 +11369,7 @@ static int intel_gen7_queue_flip(struct drm_device *dev,
}
len = 4;
- if (engine->id == RCS) {
+ if (req->engine->id == RCS) {
len += 6;
/*
* On Gen 8, SRM is now taking an extra dword to accommodate
@@ -11407,30 +11407,30 @@ static int intel_gen7_queue_flip(struct drm_device *dev,
* for the RCS also doesn't appear to drop events. Setting the DERRMR
* to zero does lead to lockups within MI_DISPLAY_FLIP.
*/
- if (engine->id == RCS) {
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(engine, DERRMR);
- intel_ring_emit(engine, ~(DERRMR_PIPEA_PRI_FLIP_DONE |
+ if (req->engine->id == RCS) {
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(ring, DERRMR);
+ intel_ring_emit(ring, ~(DERRMR_PIPEA_PRI_FLIP_DONE |
DERRMR_PIPEB_PRI_FLIP_DONE |
DERRMR_PIPEC_PRI_FLIP_DONE));
if (IS_GEN8(dev))
- intel_ring_emit(engine, MI_STORE_REGISTER_MEM_GEN8 |
+ intel_ring_emit(ring, MI_STORE_REGISTER_MEM_GEN8 |
MI_SRM_LRM_GLOBAL_GTT);
else
- intel_ring_emit(engine, MI_STORE_REGISTER_MEM |
+ intel_ring_emit(ring, MI_STORE_REGISTER_MEM |
MI_SRM_LRM_GLOBAL_GTT);
- intel_ring_emit_reg(engine, DERRMR);
- intel_ring_emit(engine, engine->scratch.gtt_offset + 256);
+ intel_ring_emit_reg(ring, DERRMR);
+ intel_ring_emit(ring, req->engine->scratch.gtt_offset + 256);
if (IS_GEN8(dev)) {
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, MI_NOOP);
}
}
- intel_ring_emit(engine, MI_DISPLAY_FLIP_I915 | plane_bit);
- intel_ring_emit(engine, (fb->pitches[0] | obj->tiling_mode));
- intel_ring_emit(engine, intel_crtc->flip_work->gtt_offset);
- intel_ring_emit(engine, (MI_NOOP));
+ intel_ring_emit(ring, MI_DISPLAY_FLIP_I915 | plane_bit);
+ intel_ring_emit(ring, (fb->pitches[0] | obj->tiling_mode));
+ intel_ring_emit(ring, intel_crtc->flip_work->gtt_offset);
+ intel_ring_emit(ring, (MI_NOOP));
return 0;
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index edd7b0f9bef3..275b32cce8c3 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -776,7 +776,7 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
struct intel_ringbuffer *ringbuf = request->ringbuf;
struct intel_engine_cs *engine = request->engine;
- intel_logical_ring_advance(ringbuf);
+ intel_ring_advance(ringbuf);
request->tail = ringbuf->tail;
/*
@@ -785,9 +785,9 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
*
* Caller must reserve WA_TAIL_DWORDS for us!
*/
- intel_logical_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_advance(ringbuf);
+ intel_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_advance(ringbuf);
/* We keep the previous context alive until we retire the following
* request. This ensures that any the context object is still pinned
@@ -871,11 +871,11 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
if (ret)
return ret;
- intel_logical_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
- intel_logical_ring_emit_reg(ringbuf, INSTPM);
- intel_logical_ring_emit(ringbuf, instp_mask << 16 | instp_mode);
- intel_logical_ring_advance(ringbuf);
+ intel_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(ringbuf, INSTPM);
+ intel_ring_emit(ringbuf, instp_mask << 16 | instp_mode);
+ intel_ring_advance(ringbuf);
dev_priv->relative_constants_mode = instp_mode;
}
@@ -1048,14 +1048,14 @@ static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
if (ret)
return ret;
- intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(w->count));
+ intel_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(w->count));
for (i = 0; i < w->count; i++) {
- intel_logical_ring_emit_reg(ringbuf, w->reg[i].addr);
- intel_logical_ring_emit(ringbuf, w->reg[i].value);
+ intel_ring_emit_reg(ringbuf, w->reg[i].addr);
+ intel_ring_emit(ringbuf, w->reg[i].value);
}
- intel_logical_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_advance(ringbuf);
+ intel_ring_advance(ringbuf);
engine->gpu_caches_dirty = true;
ret = logical_ring_flush_all_caches(req);
@@ -1545,8 +1545,7 @@ static int gen9_init_render_ring(struct intel_engine_cs *engine)
static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
{
struct i915_hw_ppgtt *ppgtt = req->ctx->ppgtt;
- struct intel_engine_cs *engine = req->engine;
- struct intel_ringbuffer *ringbuf = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ringbuf;
const int num_lri_cmds = GEN8_LEGACY_PDPES * 2;
int i, ret;
@@ -1554,20 +1553,18 @@ static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
if (ret)
return ret;
- intel_logical_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(num_lri_cmds));
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(num_lri_cmds));
for (i = GEN8_LEGACY_PDPES - 1; i >= 0; i--) {
const dma_addr_t pd_daddr = i915_page_dir_dma_addr(ppgtt, i);
- intel_logical_ring_emit_reg(ringbuf,
- GEN8_RING_PDP_UDW(engine, i));
- intel_logical_ring_emit(ringbuf, upper_32_bits(pd_daddr));
- intel_logical_ring_emit_reg(ringbuf,
- GEN8_RING_PDP_LDW(engine, i));
- intel_logical_ring_emit(ringbuf, lower_32_bits(pd_daddr));
+ intel_ring_emit_reg(ring, GEN8_RING_PDP_UDW(req->engine, i));
+ intel_ring_emit(ring, upper_32_bits(pd_daddr));
+ intel_ring_emit_reg(ring, GEN8_RING_PDP_LDW(req->engine, i));
+ intel_ring_emit(ring, lower_32_bits(pd_daddr));
}
- intel_logical_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_advance(ringbuf);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -1575,7 +1572,7 @@ static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
static int gen8_emit_bb_start(struct drm_i915_gem_request *req,
u64 offset, unsigned dispatch_flags)
{
- struct intel_ringbuffer *ringbuf = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ringbuf;
bool ppgtt = !(dispatch_flags & I915_DISPATCH_SECURE);
int ret;
@@ -1602,14 +1599,14 @@ static int gen8_emit_bb_start(struct drm_i915_gem_request *req,
return ret;
/* FIXME(BDW): Address space and security selectors. */
- intel_logical_ring_emit(ringbuf, MI_BATCH_BUFFER_START_GEN8 |
- (ppgtt<<8) |
- (dispatch_flags & I915_DISPATCH_RS ?
- MI_BATCH_RESOURCE_STREAMER : 0));
- intel_logical_ring_emit(ringbuf, lower_32_bits(offset));
- intel_logical_ring_emit(ringbuf, upper_32_bits(offset));
- intel_logical_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_advance(ringbuf);
+ intel_ring_emit(ring, MI_BATCH_BUFFER_START_GEN8 |
+ (ppgtt<<8) |
+ (dispatch_flags & I915_DISPATCH_RS ?
+ MI_BATCH_RESOURCE_STREAMER : 0));
+ intel_ring_emit(ring, lower_32_bits(offset));
+ intel_ring_emit(ring, upper_32_bits(offset));
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -1632,9 +1629,8 @@ static int gen8_emit_flush(struct drm_i915_gem_request *request,
u32 invalidate_domains,
u32 unused)
{
- struct intel_ringbuffer *ringbuf = request->ringbuf;
- struct intel_engine_cs *engine = ringbuf->engine;
- struct drm_i915_private *dev_priv = request->i915;
+ struct intel_ringbuffer *ring = request->ringbuf;
+ struct intel_engine_cs *engine = ring->engine;
uint32_t cmd;
int ret;
@@ -1653,17 +1649,17 @@ static int gen8_emit_flush(struct drm_i915_gem_request *request,
if (invalidate_domains & I915_GEM_GPU_DOMAINS) {
cmd |= MI_INVALIDATE_TLB;
- if (engine == &dev_priv->engine[VCS])
+ if (engine->id == VCS)
cmd |= MI_INVALIDATE_BSD;
}
- intel_logical_ring_emit(ringbuf, cmd);
- intel_logical_ring_emit(ringbuf,
- I915_GEM_HWS_SCRATCH_ADDR |
- MI_FLUSH_DW_USE_GTT);
- intel_logical_ring_emit(ringbuf, 0); /* upper addr */
- intel_logical_ring_emit(ringbuf, 0); /* value */
- intel_logical_ring_advance(ringbuf);
+ intel_ring_emit(ring, cmd);
+ intel_ring_emit(ring,
+ I915_GEM_HWS_SCRATCH_ADDR |
+ MI_FLUSH_DW_USE_GTT);
+ intel_ring_emit(ring, 0); /* upper addr */
+ intel_ring_emit(ring, 0); /* value */
+ intel_ring_advance(ring);
return 0;
}
@@ -1672,8 +1668,8 @@ static int gen8_emit_flush_render(struct drm_i915_gem_request *request,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ringbuf = request->ringbuf;
- struct intel_engine_cs *engine = ringbuf->engine;
+ struct intel_ringbuffer *ring = request->ringbuf;
+ struct intel_engine_cs *engine = request->engine;
u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
bool vf_flush_wa = false, dc_flush_wa = false;
u32 flags = 0;
@@ -1724,40 +1720,40 @@ static int gen8_emit_flush_render(struct drm_i915_gem_request *request,
return ret;
if (vf_flush_wa) {
- intel_logical_ring_emit(ringbuf, GFX_OP_PIPE_CONTROL(6));
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
}
if (dc_flush_wa) {
- intel_logical_ring_emit(ringbuf, GFX_OP_PIPE_CONTROL(6));
- intel_logical_ring_emit(ringbuf, PIPE_CONTROL_DC_FLUSH_ENABLE);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(ring, PIPE_CONTROL_DC_FLUSH_ENABLE);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
}
- intel_logical_ring_emit(ringbuf, GFX_OP_PIPE_CONTROL(6));
- intel_logical_ring_emit(ringbuf, flags);
- intel_logical_ring_emit(ringbuf, scratch_addr);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(ring, flags);
+ intel_ring_emit(ring, scratch_addr);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
if (dc_flush_wa) {
- intel_logical_ring_emit(ringbuf, GFX_OP_PIPE_CONTROL(6));
- intel_logical_ring_emit(ringbuf, PIPE_CONTROL_CS_STALL);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, 0);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(ring, PIPE_CONTROL_CS_STALL);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
}
- intel_logical_ring_advance(ringbuf);
+ intel_ring_advance(ring);
return 0;
}
@@ -1786,7 +1782,7 @@ static void bxt_a_seqno_barrier(struct intel_engine_cs *engine)
static int gen8_emit_request(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ringbuf = request->ringbuf;
+ struct intel_ringbuffer *ring = request->ringbuf;
int ret;
ret = intel_ring_begin(request, 6 + WA_TAIL_DWORDS);
@@ -1796,21 +1792,20 @@ static int gen8_emit_request(struct drm_i915_gem_request *request)
/* w/a: bit 5 needs to be zero for MI_FLUSH_DW address. */
BUILD_BUG_ON(I915_GEM_HWS_INDEX_ADDR & (1 << 5));
- intel_logical_ring_emit(ringbuf,
- (MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW);
- intel_logical_ring_emit(ringbuf,
- intel_hws_seqno_address(request->engine) |
- MI_FLUSH_DW_USE_GTT);
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, request->fence.seqno);
- intel_logical_ring_emit(ringbuf, MI_USER_INTERRUPT);
- intel_logical_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_emit(ring, (MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW);
+ intel_ring_emit(ring,
+ intel_hws_seqno_address(request->engine) |
+ MI_FLUSH_DW_USE_GTT);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, request->fence.seqno);
+ intel_ring_emit(ring, MI_USER_INTERRUPT);
+ intel_ring_emit(ring, MI_NOOP);
return intel_logical_ring_advance_and_submit(request);
}
static int gen8_emit_request_render(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ringbuf = request->ringbuf;
+ struct intel_ringbuffer *ring = request->ringbuf;
int ret;
ret = intel_ring_begin(request, 8 + WA_TAIL_DWORDS);
@@ -1824,19 +1819,18 @@ static int gen8_emit_request_render(struct drm_i915_gem_request *request)
* need a prior CS_STALL, which is emitted by the flush
* following the batch.
*/
- intel_logical_ring_emit(ringbuf, GFX_OP_PIPE_CONTROL(6));
- intel_logical_ring_emit(ringbuf,
- (PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_QW_WRITE));
- intel_logical_ring_emit(ringbuf,
- intel_hws_seqno_address(request->engine));
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, i915_gem_request_get_seqno(request));
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(ring,
+ (PIPE_CONTROL_GLOBAL_GTT_IVB |
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_QW_WRITE));
+ intel_ring_emit(ring, intel_hws_seqno_address(request->engine));
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, i915_gem_request_get_seqno(request));
/* We're thrashing one dword of HWS. */
- intel_logical_ring_emit(ringbuf, 0);
- intel_logical_ring_emit(ringbuf, MI_USER_INTERRUPT);
- intel_logical_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, MI_USER_INTERRUPT);
+ intel_ring_emit(ring, MI_NOOP);
return intel_logical_ring_advance_and_submit(request);
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index a19dc49e89da..63d6a49a0ded 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -70,32 +70,6 @@ void intel_logical_ring_cleanup(struct intel_engine_cs *engine);
int intel_logical_rings_init(struct drm_device *dev);
int logical_ring_flush_all_caches(struct drm_i915_gem_request *req);
-/**
- * intel_logical_ring_advance() - advance the ringbuffer tail
- * @ringbuf: Ringbuffer to advance.
- *
- * The tail is only updated in our logical ringbuffer struct.
- */
-static inline void intel_logical_ring_advance(struct intel_ringbuffer *ringbuf)
-{
- __intel_ringbuffer_advance(ringbuf);
-}
-
-/**
- * intel_logical_ring_emit() - write a DWORD to the ringbuffer.
- * @ringbuf: Ringbuffer to write to.
- * @data: DWORD to write.
- */
-static inline void intel_logical_ring_emit(struct intel_ringbuffer *ringbuf,
- u32 data)
-{
- __intel_ringbuffer_emit(ringbuf, data);
-}
-static inline void intel_logical_ring_emit_reg(struct intel_ringbuffer *ringbuf,
- i915_reg_t reg)
-{
- intel_logical_ring_emit(ringbuf, i915_mmio_reg_offset(reg));
-}
/* Logical Ring Contexts */
diff --git a/drivers/gpu/drm/i915/intel_mocs.c b/drivers/gpu/drm/i915/intel_mocs.c
index 3c1482b8f2f4..28c99846dccd 100644
--- a/drivers/gpu/drm/i915/intel_mocs.c
+++ b/drivers/gpu/drm/i915/intel_mocs.c
@@ -253,14 +253,11 @@ static int emit_mocs_control_table(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_logical_ring_emit(ringbuf,
- MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES));
+ intel_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES));
for (index = 0; index < table->size; index++) {
- intel_logical_ring_emit_reg(ringbuf,
- mocs_register(engine, index));
- intel_logical_ring_emit(ringbuf,
- table->table[index].control_value);
+ intel_ring_emit_reg(ringbuf, mocs_register(engine, index));
+ intel_ring_emit(ringbuf, table->table[index].control_value);
}
/*
@@ -272,14 +269,12 @@ static int emit_mocs_control_table(struct drm_i915_gem_request *req,
* that value to all the used entries.
*/
for (; index < GEN9_NUM_MOCS_ENTRIES; index++) {
- intel_logical_ring_emit_reg(ringbuf,
- mocs_register(engine, index));
- intel_logical_ring_emit(ringbuf,
- table->table[0].control_value);
+ intel_ring_emit_reg(ringbuf, mocs_register(engine, index));
+ intel_ring_emit(ringbuf, table->table[0].control_value);
}
- intel_logical_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_advance(ringbuf);
+ intel_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_advance(ringbuf);
return 0;
}
@@ -317,19 +312,18 @@ static int emit_mocs_l3cc_table(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_logical_ring_emit(ringbuf,
+ intel_ring_emit(ringbuf,
MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES / 2));
for (i = 0; i < table->size/2; i++) {
- intel_logical_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_logical_ring_emit(ringbuf,
- l3cc_combine(table, 2*i, 2*i+1));
+ intel_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
+ intel_ring_emit(ringbuf, l3cc_combine(table, 2*i, 2*i+1));
}
if (table->size & 0x01) {
/* Odd table size - 1 left over */
- intel_logical_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_logical_ring_emit(ringbuf, l3cc_combine(table, 2*i, 0));
+ intel_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
+ intel_ring_emit(ringbuf, l3cc_combine(table, 2*i, 0));
i++;
}
@@ -339,12 +333,12 @@ static int emit_mocs_l3cc_table(struct drm_i915_gem_request *req,
* they are reserved by the hardware.
*/
for (; i < GEN9_NUM_MOCS_ENTRIES / 2; i++) {
- intel_logical_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_logical_ring_emit(ringbuf, l3cc_combine(table, 0, 0));
+ intel_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
+ intel_ring_emit(ringbuf, l3cc_combine(table, 0, 0));
}
- intel_logical_ring_emit(ringbuf, MI_NOOP);
- intel_logical_ring_advance(ringbuf);
+ intel_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_advance(ringbuf);
return 0;
}
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 8654a323722e..92722e614955 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -235,6 +235,7 @@ static int intel_overlay_on(struct intel_overlay *overlay)
struct drm_i915_private *dev_priv = overlay->i915;
struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
+ struct intel_ringbuffer *ring;
int ret;
WARN_ON(overlay->active);
@@ -252,11 +253,12 @@ static int intel_overlay_on(struct intel_overlay *overlay)
overlay->active = true;
- intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_ON);
- intel_ring_emit(engine, overlay->flip_addr | OFC_UPDATE);
- intel_ring_emit(engine, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ ring = req->ringbuf;
+ intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_ON);
+ intel_ring_emit(ring, overlay->flip_addr | OFC_UPDATE);
+ intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return intel_overlay_do_wait_request(overlay, req, NULL);
}
@@ -268,6 +270,7 @@ static int intel_overlay_continue(struct intel_overlay *overlay,
struct drm_i915_private *dev_priv = overlay->i915;
struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
+ struct intel_ringbuffer *ring;
u32 flip_addr = overlay->flip_addr;
u32 tmp;
int ret;
@@ -292,9 +295,10 @@ static int intel_overlay_continue(struct intel_overlay *overlay,
return ret;
}
- intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
- intel_ring_emit(engine, flip_addr);
- intel_ring_advance(engine);
+ ring = req->ringbuf;
+ intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
+ intel_ring_emit(ring, flip_addr);
+ intel_ring_advance(ring);
WARN_ON(overlay->last_flip_req);
i915_gem_request_assign(&overlay->last_flip_req, req);
@@ -336,6 +340,7 @@ static int intel_overlay_off(struct intel_overlay *overlay)
struct drm_i915_private *dev_priv = overlay->i915;
struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
+ struct intel_ringbuffer *ring;
u32 flip_addr = overlay->flip_addr;
int ret;
@@ -357,24 +362,25 @@ static int intel_overlay_off(struct intel_overlay *overlay)
return ret;
}
+ ring = req->ringbuf;
/* wait for overlay to go idle */
- intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
- intel_ring_emit(engine, flip_addr);
- intel_ring_emit(engine, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
+ intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
+ intel_ring_emit(ring, flip_addr);
+ intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
/* turn overlay off */
if (IS_I830(dev_priv)) {
/* Workaround: Don't disable the overlay fully, since otherwise
* it dies on the next OVERLAY_ON cmd. */
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
} else {
- intel_ring_emit(engine, MI_OVERLAY_FLIP | MI_OVERLAY_OFF);
- intel_ring_emit(engine, flip_addr);
- intel_ring_emit(engine,
+ intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_OFF);
+ intel_ring_emit(ring, flip_addr);
+ intel_ring_emit(ring,
MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
}
- intel_ring_advance(engine);
+ intel_ring_advance(ring);
return intel_overlay_do_wait_request(overlay, req, intel_overlay_off_tail);
}
@@ -420,6 +426,7 @@ static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
if (I915_READ(ISR) & I915_OVERLAY_PLANE_FLIP_PENDING_INTERRUPT) {
/* synchronous slowpath */
struct drm_i915_gem_request *req;
+ struct intel_ringbuffer *ring;
req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req))
@@ -431,10 +438,11 @@ static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
return ret;
}
- intel_ring_emit(engine,
+ ring = req->ringbuf;
+ intel_ring_emit(ring,
MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
ret = intel_overlay_do_wait_request(overlay, req,
intel_overlay_release_old_vid_tail);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 1eaf368bcce9..60cd36f64c61 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -58,7 +58,7 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf)
ringbuf->tail, ringbuf->size);
}
-static void __intel_ring_advance(struct intel_engine_cs *engine)
+static void __intel_engine_submit(struct intel_engine_cs *engine)
{
struct intel_ringbuffer *ringbuf = engine->buffer;
ringbuf->tail &= ringbuf->size - 1;
@@ -70,7 +70,7 @@ gen2_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
u32 cmd;
int ret;
@@ -85,9 +85,9 @@ gen2_render_ring_flush(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_ring_emit(engine, cmd);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, cmd);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -97,7 +97,7 @@ gen4_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
u32 cmd;
int ret;
@@ -129,23 +129,20 @@ gen4_render_ring_flush(struct drm_i915_gem_request *req,
* are flushed at any MI_FLUSH.
*/
- cmd = MI_FLUSH | MI_NO_WRITE_FLUSH;
- if ((invalidate_domains|flush_domains) & I915_GEM_DOMAIN_RENDER)
- cmd &= ~MI_NO_WRITE_FLUSH;
- if (invalidate_domains & I915_GEM_DOMAIN_INSTRUCTION)
+ cmd = MI_FLUSH;
+ if (invalidate_domains) {
cmd |= MI_EXE_FLUSH;
-
- if (invalidate_domains & I915_GEM_DOMAIN_COMMAND &&
- (IS_G4X(req->i915) || IS_GEN5(req->i915)))
- cmd |= MI_INVALIDATE_ISP;
+ if (IS_G4X(req->i915) || IS_GEN5(req->i915))
+ cmd |= MI_INVALIDATE_ISP;
+ }
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(engine, cmd);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, cmd);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -190,34 +187,35 @@ gen4_render_ring_flush(struct drm_i915_gem_request *req,
static int
intel_emit_post_sync_nonzero_flush(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
- u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ struct intel_ringbuffer *ring = req->ringbuf;
+ u32 scratch_addr =
+ req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
ret = intel_ring_begin(req, 6);
if (ret)
return ret;
- intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(5));
- intel_ring_emit(engine, PIPE_CONTROL_CS_STALL |
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(5));
+ intel_ring_emit(ring, PIPE_CONTROL_CS_STALL |
PIPE_CONTROL_STALL_AT_SCOREBOARD);
- intel_ring_emit(engine, scratch_addr | PIPE_CONTROL_GLOBAL_GTT); /* address */
- intel_ring_emit(engine, 0); /* low dword */
- intel_ring_emit(engine, 0); /* high dword */
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
+ intel_ring_emit(ring, 0); /* low dword */
+ intel_ring_emit(ring, 0); /* high dword */
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
ret = intel_ring_begin(req, 6);
if (ret)
return ret;
- intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(5));
- intel_ring_emit(engine, PIPE_CONTROL_QW_WRITE);
- intel_ring_emit(engine, scratch_addr | PIPE_CONTROL_GLOBAL_GTT); /* address */
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(5));
+ intel_ring_emit(ring, PIPE_CONTROL_QW_WRITE);
+ intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -226,9 +224,10 @@ static int
gen6_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
+ u32 scratch_addr =
+ req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
u32 flags = 0;
- u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
/* Force SNB workarounds for PIPE_CONTROL flushes */
@@ -266,11 +265,11 @@ gen6_render_ring_flush(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(engine, flags);
- intel_ring_emit(engine, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
- intel_ring_emit(engine, 0);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
+ intel_ring_emit(ring, flags);
+ intel_ring_emit(ring, scratch_addr | PIPE_CONTROL_GLOBAL_GTT);
+ intel_ring_emit(ring, 0);
+ intel_ring_advance(ring);
return 0;
}
@@ -278,19 +277,20 @@ gen6_render_ring_flush(struct drm_i915_gem_request *req,
static int
gen7_render_ring_cs_stall_wa(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 4);
if (ret)
return ret;
- intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(engine, PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_STALL_AT_SCOREBOARD);
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, 0);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
+ intel_ring_emit(ring,
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_STALL_AT_SCOREBOARD);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_advance(ring);
return 0;
}
@@ -299,9 +299,10 @@ static int
gen7_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
+ u32 scratch_addr =
+ req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
u32 flags = 0;
- u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
/*
@@ -350,11 +351,11 @@ gen7_render_ring_flush(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(4));
- intel_ring_emit(engine, flags);
- intel_ring_emit(engine, scratch_addr);
- intel_ring_emit(engine, 0);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(4));
+ intel_ring_emit(ring, flags);
+ intel_ring_emit(ring, scratch_addr);
+ intel_ring_emit(ring, 0);
+ intel_ring_advance(ring);
return 0;
}
@@ -363,20 +364,20 @@ static int
gen8_emit_pipe_control(struct drm_i915_gem_request *req,
u32 flags, u32 scratch_addr)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 6);
if (ret)
return ret;
- intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(engine, flags);
- intel_ring_emit(engine, scratch_addr);
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, 0);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(ring, flags);
+ intel_ring_emit(ring, scratch_addr);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, 0);
+ intel_ring_advance(ring);
return 0;
}
@@ -385,8 +386,8 @@ static int
gen8_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- u32 flags = 0;
u32 scratch_addr = req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
+ u32 flags = 0;
int ret;
flags |= PIPE_CONTROL_CS_STALL;
@@ -684,14 +685,14 @@ err:
static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
struct i915_workarounds *w = &req->i915->workarounds;
int ret, i;
if (w->count == 0)
return 0;
- engine->gpu_caches_dirty = true;
+ req->engine->gpu_caches_dirty = true;
ret = intel_ring_flush_all_caches(req);
if (ret)
return ret;
@@ -700,16 +701,16 @@ static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
if (ret)
return ret;
- intel_ring_emit(engine, MI_LOAD_REGISTER_IMM(w->count));
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(w->count));
for (i = 0; i < w->count; i++) {
- intel_ring_emit_reg(engine, w->reg[i].addr);
- intel_ring_emit(engine, w->reg[i].value);
+ intel_ring_emit_reg(ring, w->reg[i].addr);
+ intel_ring_emit(ring, w->reg[i].value);
}
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_advance(ring);
- engine->gpu_caches_dirty = true;
+ req->engine->gpu_caches_dirty = true;
ret = intel_ring_flush_all_caches(req);
if (ret)
return ret;
@@ -1327,7 +1328,7 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 8
- struct intel_engine_cs *signaller = signaller_req->engine;
+ struct intel_ringbuffer *signaller = signaller_req->ringbuf;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
enum intel_engine_id id;
@@ -1342,20 +1343,23 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
return ret;
for_each_engine_id(waiter, dev_priv, id) {
- u64 gtt_offset = signaller->semaphore.signal_ggtt[id];
+ u64 gtt_offset =
+ signaller_req->engine->semaphore.signal_ggtt[id];
if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID)
continue;
intel_ring_emit(signaller, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(signaller, PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_QW_WRITE |
- PIPE_CONTROL_CS_STALL);
+ intel_ring_emit(signaller,
+ PIPE_CONTROL_GLOBAL_GTT_IVB |
+ PIPE_CONTROL_QW_WRITE |
+ PIPE_CONTROL_CS_STALL);
intel_ring_emit(signaller, lower_32_bits(gtt_offset));
intel_ring_emit(signaller, upper_32_bits(gtt_offset));
intel_ring_emit(signaller, signaller_req->fence.seqno);
intel_ring_emit(signaller, 0);
- intel_ring_emit(signaller, MI_SEMAPHORE_SIGNAL |
- MI_SEMAPHORE_TARGET(waiter->hw_id));
+ intel_ring_emit(signaller,
+ MI_SEMAPHORE_SIGNAL |
+ MI_SEMAPHORE_TARGET(waiter->hw_id));
intel_ring_emit(signaller, 0);
}
@@ -1366,7 +1370,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 6
- struct intel_engine_cs *signaller = signaller_req->engine;
+ struct intel_ringbuffer *signaller = signaller_req->ringbuf;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
enum intel_engine_id id;
@@ -1381,18 +1385,21 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
return ret;
for_each_engine_id(waiter, dev_priv, id) {
- u64 gtt_offset = signaller->semaphore.signal_ggtt[id];
+ u64 gtt_offset =
+ signaller_req->engine->semaphore.signal_ggtt[id];
if (gtt_offset == MI_SEMAPHORE_SYNC_INVALID)
continue;
- intel_ring_emit(signaller, (MI_FLUSH_DW + 1) |
- MI_FLUSH_DW_OP_STOREDW);
- intel_ring_emit(signaller, lower_32_bits(gtt_offset) |
- MI_FLUSH_DW_USE_GTT);
+ intel_ring_emit(signaller,
+ (MI_FLUSH_DW + 1) | MI_FLUSH_DW_OP_STOREDW);
+ intel_ring_emit(signaller,
+ lower_32_bits(gtt_offset) |
+ MI_FLUSH_DW_USE_GTT);
intel_ring_emit(signaller, upper_32_bits(gtt_offset));
intel_ring_emit(signaller, signaller_req->fence.seqno);
- intel_ring_emit(signaller, MI_SEMAPHORE_SIGNAL |
- MI_SEMAPHORE_TARGET(waiter->hw_id));
+ intel_ring_emit(signaller,
+ MI_SEMAPHORE_SIGNAL |
+ MI_SEMAPHORE_TARGET(waiter->hw_id));
intel_ring_emit(signaller, 0);
}
@@ -1402,7 +1409,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
static int gen6_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
- struct intel_engine_cs *signaller = signaller_req->engine;
+ struct intel_ringbuffer *signaller = signaller_req->ringbuf;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *useless;
enum intel_engine_id id;
@@ -1418,7 +1425,8 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
return ret;
for_each_engine_id(useless, dev_priv, id) {
- i915_reg_t mbox_reg = signaller->semaphore.mbox.signal[id];
+ i915_reg_t mbox_reg =
+ signaller_req->engine->semaphore.mbox.signal[id];
if (i915_mmio_reg_valid(mbox_reg)) {
intel_ring_emit(signaller, MI_LOAD_REGISTER_IMM(1));
@@ -1445,23 +1453,22 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
static int
gen6_add_request(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
- if (engine->semaphore.signal)
- ret = engine->semaphore.signal(req, 4);
+ if (req->engine->semaphore.signal)
+ ret = req->engine->semaphore.signal(req, 4);
else
ret = intel_ring_begin(req, 4);
if (ret)
return ret;
- intel_ring_emit(engine, MI_STORE_DWORD_INDEX);
- intel_ring_emit(engine,
- I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
- intel_ring_emit(engine, req->fence.seqno);
- intel_ring_emit(engine, MI_USER_INTERRUPT);
- __intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_STORE_DWORD_INDEX);
+ intel_ring_emit(ring, I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
+ intel_ring_emit(ring, req->fence.seqno);
+ intel_ring_emit(ring, MI_USER_INTERRUPT);
+ __intel_engine_submit(req->engine);
return 0;
}
@@ -1470,6 +1477,7 @@ static int
gen8_render_add_request(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
if (engine->semaphore.signal)
@@ -1479,18 +1487,18 @@ gen8_render_add_request(struct drm_i915_gem_request *req)
if (ret)
return ret;
- intel_ring_emit(engine, GFX_OP_PIPE_CONTROL(6));
- intel_ring_emit(engine, (PIPE_CONTROL_GLOBAL_GTT_IVB |
- PIPE_CONTROL_CS_STALL |
- PIPE_CONTROL_QW_WRITE));
- intel_ring_emit(engine, intel_hws_seqno_address(req->engine));
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, i915_gem_request_get_seqno(req));
+ intel_ring_emit(ring, GFX_OP_PIPE_CONTROL(6));
+ intel_ring_emit(ring, (PIPE_CONTROL_GLOBAL_GTT_IVB |
+ PIPE_CONTROL_CS_STALL |
+ PIPE_CONTROL_QW_WRITE));
+ intel_ring_emit(ring, intel_hws_seqno_address(engine));
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, i915_gem_request_get_seqno(req));
/* We're thrashing one dword of HWS. */
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, MI_USER_INTERRUPT);
- intel_ring_emit(engine, MI_NOOP);
- __intel_ring_advance(engine);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, MI_USER_INTERRUPT);
+ intel_ring_emit(ring, MI_NOOP);
+ __intel_engine_submit(engine);
return 0;
}
@@ -1514,9 +1522,9 @@ gen8_ring_sync(struct drm_i915_gem_request *waiter_req,
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_engine_cs *waiter = waiter_req->engine;
+ struct intel_ringbuffer *waiter = waiter_req->ringbuf;
struct drm_i915_private *dev_priv = waiter_req->i915;
- u64 offset = GEN8_WAIT_OFFSET(waiter, signaller->id);
+ u64 offset = GEN8_WAIT_OFFSET(waiter_req->engine, signaller->id);
struct i915_hw_ppgtt *ppgtt;
int ret;
@@ -1548,11 +1556,11 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_engine_cs *waiter = waiter_req->engine;
+ struct intel_ringbuffer *waiter = waiter_req->ringbuf;
u32 dw1 = MI_SEMAPHORE_MBOX |
MI_SEMAPHORE_COMPARE |
MI_SEMAPHORE_REGISTER;
- u32 wait_mbox = signaller->semaphore.mbox.wait[waiter->id];
+ u32 wait_mbox = signaller->semaphore.mbox.wait[waiter_req->engine->id];
int ret;
/* Throughout all of the GEM code, seqno passed implies our current
@@ -1682,35 +1690,34 @@ bsd_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(engine, MI_FLUSH);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_FLUSH);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
static int
i9xx_add_request(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 4);
if (ret)
return ret;
- intel_ring_emit(engine, MI_STORE_DWORD_INDEX);
- intel_ring_emit(engine,
- I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
- intel_ring_emit(engine, req->fence.seqno);
- intel_ring_emit(engine, MI_USER_INTERRUPT);
- __intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_STORE_DWORD_INDEX);
+ intel_ring_emit(ring, I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
+ intel_ring_emit(ring, req->fence.seqno);
+ intel_ring_emit(ring, MI_USER_INTERRUPT);
+ __intel_engine_submit(req->engine);
return 0;
}
@@ -1777,20 +1784,20 @@ i965_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 length,
unsigned dispatch_flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(engine,
+ intel_ring_emit(ring,
MI_BATCH_BUFFER_START |
MI_BATCH_GTT |
(dispatch_flags & I915_DISPATCH_SECURE ?
0 : MI_BATCH_NON_SECURE_I965));
- intel_ring_emit(engine, offset);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, offset);
+ intel_ring_advance(ring);
return 0;
}
@@ -1804,8 +1811,8 @@ i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *engine = req->engine;
- u32 cs_offset = engine->scratch.gtt_offset;
+ struct intel_ringbuffer *ring = req->ringbuf;
+ u32 cs_offset = req->engine->scratch.gtt_offset;
int ret;
ret = intel_ring_begin(req, 6);
@@ -1813,13 +1820,13 @@ i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
return ret;
/* Evict the invalid PTE TLBs */
- intel_ring_emit(engine, COLOR_BLT_CMD | BLT_WRITE_RGBA);
- intel_ring_emit(engine, BLT_DEPTH_32 | BLT_ROP_COLOR_COPY | 4096);
- intel_ring_emit(engine, I830_TLB_ENTRIES << 16 | 4); /* load each page */
- intel_ring_emit(engine, cs_offset);
- intel_ring_emit(engine, 0xdeadbeef);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, COLOR_BLT_CMD | BLT_WRITE_RGBA);
+ intel_ring_emit(ring, BLT_DEPTH_32 | BLT_ROP_COLOR_COPY | 4096);
+ intel_ring_emit(ring, I830_TLB_ENTRIES << 16 | 4); /* load each page */
+ intel_ring_emit(ring, cs_offset);
+ intel_ring_emit(ring, 0xdeadbeef);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
if ((dispatch_flags & I915_DISPATCH_PINNED) == 0) {
if (len > I830_BATCH_LIMIT)
@@ -1833,17 +1840,17 @@ i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
* stable batch scratch bo area (so that the CS never
* stumbles over its tlb invalidation bug) ...
*/
- intel_ring_emit(engine, SRC_COPY_BLT_CMD | BLT_WRITE_RGBA);
- intel_ring_emit(engine,
+ intel_ring_emit(ring, SRC_COPY_BLT_CMD | BLT_WRITE_RGBA);
+ intel_ring_emit(ring,
BLT_DEPTH_32 | BLT_ROP_SRC_COPY | 4096);
- intel_ring_emit(engine, DIV_ROUND_UP(len, 4096) << 16 | 4096);
- intel_ring_emit(engine, cs_offset);
- intel_ring_emit(engine, 4096);
- intel_ring_emit(engine, offset);
+ intel_ring_emit(ring, DIV_ROUND_UP(len, 4096) << 16 | 4096);
+ intel_ring_emit(ring, cs_offset);
+ intel_ring_emit(ring, 4096);
+ intel_ring_emit(ring, offset);
- intel_ring_emit(engine, MI_FLUSH);
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_FLUSH);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
/* ... and execute it. */
offset = cs_offset;
@@ -1853,10 +1860,10 @@ i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_ring_emit(engine, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
- intel_ring_emit(engine, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE));
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
+ intel_ring_emit(ring, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
+ 0 : MI_BATCH_NON_SECURE));
+ intel_ring_advance(ring);
return 0;
}
@@ -1866,17 +1873,17 @@ i915_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(engine, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
- intel_ring_emit(engine, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
- 0 : MI_BATCH_NON_SECURE));
- intel_ring_advance(engine);
+ intel_ring_emit(ring, MI_BATCH_BUFFER_START | MI_BATCH_GTT);
+ intel_ring_emit(ring, offset | (dispatch_flags & I915_DISPATCH_SECURE ?
+ 0 : MI_BATCH_NON_SECURE));
+ intel_ring_advance(ring);
return 0;
}
@@ -2417,8 +2424,9 @@ int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
/* Align the ring tail to a cacheline boundary */
int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = req->engine;
- int num_dwords = (engine->buffer->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
+ struct intel_ringbuffer *ring = req->ringbuf;
+ int num_dwords =
+ (ring->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
int ret;
if (num_dwords == 0)
@@ -2430,9 +2438,9 @@ int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
return ret;
while (num_dwords--)
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_advance(ring);
return 0;
}
@@ -2523,7 +2531,7 @@ static void gen6_bsd_ring_write_tail(struct intel_engine_cs *engine,
static int gen6_bsd_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
uint32_t cmd;
int ret;
@@ -2551,17 +2559,16 @@ static int gen6_bsd_ring_flush(struct drm_i915_gem_request *req,
if (invalidate & I915_GEM_GPU_DOMAINS)
cmd |= MI_INVALIDATE_TLB | MI_INVALIDATE_BSD;
- intel_ring_emit(engine, cmd);
- intel_ring_emit(engine,
- I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
+ intel_ring_emit(ring, cmd);
+ intel_ring_emit(ring, I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
if (INTEL_GEN(req->i915) >= 8) {
- intel_ring_emit(engine, 0); /* upper addr */
- intel_ring_emit(engine, 0); /* value */
+ intel_ring_emit(ring, 0); /* upper addr */
+ intel_ring_emit(ring, 0); /* value */
} else {
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, MI_NOOP);
}
- intel_ring_advance(engine);
+ intel_ring_advance(ring);
return 0;
}
@@ -2570,8 +2577,8 @@ gen8_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *engine = req->engine;
- bool ppgtt = USES_PPGTT(engine->dev) &&
+ struct intel_ringbuffer *ring = req->ringbuf;
+ bool ppgtt = USES_PPGTT(req->i915) &&
!(dispatch_flags & I915_DISPATCH_SECURE);
int ret;
@@ -2580,13 +2587,13 @@ gen8_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
return ret;
/* FIXME(BDW): Address space and security selectors. */
- intel_ring_emit(engine, MI_BATCH_BUFFER_START_GEN8 | (ppgtt<<8) |
+ intel_ring_emit(ring, MI_BATCH_BUFFER_START_GEN8 | (ppgtt<<8) |
(dispatch_flags & I915_DISPATCH_RS ?
MI_BATCH_RESOURCE_STREAMER : 0));
- intel_ring_emit(engine, lower_32_bits(offset));
- intel_ring_emit(engine, upper_32_bits(offset));
- intel_ring_emit(engine, MI_NOOP);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, lower_32_bits(offset));
+ intel_ring_emit(ring, upper_32_bits(offset));
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -2596,22 +2603,22 @@ hsw_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(engine,
+ intel_ring_emit(ring,
MI_BATCH_BUFFER_START |
(dispatch_flags & I915_DISPATCH_SECURE ?
0 : MI_BATCH_PPGTT_HSW | MI_BATCH_NON_SECURE_HSW) |
(dispatch_flags & I915_DISPATCH_RS ?
MI_BATCH_RESOURCE_STREAMER : 0));
/* bit0-7 is the length on GEN6+ */
- intel_ring_emit(engine, offset);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, offset);
+ intel_ring_advance(ring);
return 0;
}
@@ -2621,20 +2628,20 @@ gen6_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
int ret;
ret = intel_ring_begin(req, 2);
if (ret)
return ret;
- intel_ring_emit(engine,
+ intel_ring_emit(ring,
MI_BATCH_BUFFER_START |
(dispatch_flags & I915_DISPATCH_SECURE ?
0 : MI_BATCH_NON_SECURE_I965));
/* bit0-7 is the length on GEN6+ */
- intel_ring_emit(engine, offset);
- intel_ring_advance(engine);
+ intel_ring_emit(ring, offset);
+ intel_ring_advance(ring);
return 0;
}
@@ -2644,7 +2651,7 @@ gen6_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
static int gen6_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_engine_cs *engine = req->engine;
+ struct intel_ringbuffer *ring = req->ringbuf;
uint32_t cmd;
int ret;
@@ -2671,17 +2678,17 @@ static int gen6_ring_flush(struct drm_i915_gem_request *req,
*/
if (invalidate & I915_GEM_DOMAIN_RENDER)
cmd |= MI_INVALIDATE_TLB;
- intel_ring_emit(engine, cmd);
- intel_ring_emit(engine,
+ intel_ring_emit(ring, cmd);
+ intel_ring_emit(ring,
I915_GEM_HWS_SCRATCH_ADDR | MI_FLUSH_DW_USE_GTT);
if (INTEL_GEN(req->i915) >= 8) {
- intel_ring_emit(engine, 0); /* upper addr */
- intel_ring_emit(engine, 0); /* value */
+ intel_ring_emit(ring, 0); /* upper addr */
+ intel_ring_emit(ring, 0); /* value */
} else {
- intel_ring_emit(engine, 0);
- intel_ring_emit(engine, MI_NOOP);
+ intel_ring_emit(ring, 0);
+ intel_ring_emit(ring, MI_NOOP);
}
- intel_ring_advance(engine);
+ intel_ring_advance(ring);
return 0;
}
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 34d79b8fcb7d..4eac4c066c15 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -453,32 +453,21 @@ int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
-static inline void __intel_ringbuffer_emit(struct intel_ringbuffer *rb,
- u32 data)
+static inline void intel_ring_emit(struct intel_ringbuffer *ring, u32 data)
{
- *(uint32_t *)(rb->vaddr + rb->tail) = data;
- rb->tail += 4;
+ *(uint32_t *)(ring->vaddr + ring->tail) = data;
+ ring->tail += 4;
}
-static inline void __intel_ringbuffer_advance(struct intel_ringbuffer *rb)
-{
- rb->tail &= rb->size - 1;
-}
-
-static inline void intel_ring_emit(struct intel_engine_cs *engine, u32 data)
-{
- __intel_ringbuffer_emit(engine->buffer, data);
-}
-
-static inline void intel_ring_emit_reg(struct intel_engine_cs *engine,
+static inline void intel_ring_emit_reg(struct intel_ringbuffer *ring,
i915_reg_t reg)
{
- intel_ring_emit(engine, i915_mmio_reg_offset(reg));
+ intel_ring_emit(ring, i915_mmio_reg_offset(reg));
}
-static inline void intel_ring_advance(struct intel_engine_cs *engine)
+static inline void intel_ring_advance(struct intel_ringbuffer *ring)
{
- __intel_ringbuffer_advance(engine->buffer);
+ ring->tail &= ring->size - 1;
}
int __intel_ring_space(int head, int tail, int size);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 26/64] drm/i915: Rename request->ringbuf to request->ring
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (23 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 25/64] drm/i915: Unify intel_logical_ring_emit and intel_ring_emit Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 27/64] drm/i915: Rename backpointer from intel_ringbuffer to intel_engine_cs Chris Wilson
` (38 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Now that we have disambuigated ring and engine, we can use the clearer
and more consistent name for the intel_ringbuffer pointer in the
request.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_context.c | 4 +-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 4 +-
drivers/gpu/drm/i915/i915_gem_gtt.c | 6 +-
drivers/gpu/drm/i915/i915_gem_request.c | 16 +++---
drivers/gpu/drm/i915/i915_gem_request.h | 2 +-
drivers/gpu/drm/i915/i915_gpu_error.c | 20 +++----
drivers/gpu/drm/i915/intel_display.c | 10 ++--
drivers/gpu/drm/i915/intel_lrc.c | 57 +++++++++---------
drivers/gpu/drm/i915/intel_mocs.c | 36 ++++++------
drivers/gpu/drm/i915/intel_overlay.c | 8 +--
drivers/gpu/drm/i915/intel_ringbuffer.c | 92 +++++++++++++++---------------
11 files changed, 126 insertions(+), 129 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index cdf6528b0687..271aeba6f3e5 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -552,7 +552,7 @@ static inline int
mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
{
struct drm_i915_private *dev_priv = req->i915;
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
u32 flags = hw_flags | MI_MM_SPACE_GTT;
const int num_rings =
/* Use an extended w/a on ivb+ if signalling from other rings */
@@ -654,7 +654,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
static int remap_l3(struct drm_i915_gem_request *req, int slice)
{
u32 *remap_info = req->i915->l3_parity.remap_info[slice];
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int i, ret;
if (!remap_info)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 9c9763fec955..d77b01fa4332 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1166,7 +1166,7 @@ i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params)
static int
i915_reset_gen7_sol_offsets(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret, i;
if (!IS_GEN7(req->i915) || req->engine->id != RCS) {
@@ -1296,7 +1296,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
if (params->engine->id == RCS &&
instp_mode != dev_priv->relative_constants_mode) {
- struct intel_ringbuffer *ring = params->request->ringbuf;
+ struct intel_ringbuffer *ring = params->request->ring;
ret = intel_ring_begin(params->request, 4);
if (ret)
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 2dcc0da272e8..1d4ff1a7e3e0 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -669,7 +669,7 @@ static int gen8_write_pdp(struct drm_i915_gem_request *req,
unsigned entry,
dma_addr_t addr)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
BUG_ON(entry >= 4);
@@ -1660,7 +1660,7 @@ static uint32_t get_pd_offset(struct i915_hw_ppgtt *ppgtt)
static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
@@ -1688,7 +1688,7 @@ static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index f15924c8a26e..9cab2eb16868 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -170,7 +170,7 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
* Note this requires that we are always called in request
* completion order.
*/
- request->ringbuf->last_retired_head = request->postfix;
+ request->ring->last_retired_head = request->postfix;
i915_gem_request_remove_from_client(request);
@@ -424,7 +424,7 @@ void __i915_add_request(struct drm_i915_gem_request *request,
bool flush_caches)
{
struct intel_engine_cs *engine;
- struct intel_ringbuffer *ringbuf;
+ struct intel_ringbuffer *ring;
u32 request_start;
u32 reserved_tail;
int ret;
@@ -433,14 +433,14 @@ void __i915_add_request(struct drm_i915_gem_request *request,
return;
engine = request->engine;
- ringbuf = request->ringbuf;
+ ring = request->ring;
/*
* To ensure that this call will not fail, space for its emissions
* should already have been reserved in the ring buffer. Let the ring
* know that it is time to use that space up.
*/
- request_start = intel_ring_get_tail(ringbuf);
+ request_start = intel_ring_get_tail(ring);
reserved_tail = request->reserved_space;
request->reserved_space = 0;
@@ -487,21 +487,21 @@ void __i915_add_request(struct drm_i915_gem_request *request,
* GPU processing the request, we never over-estimate the
* position of the head.
*/
- request->postfix = intel_ring_get_tail(ringbuf);
+ request->postfix = intel_ring_get_tail(ring);
if (i915.enable_execlists) {
ret = engine->emit_request(request);
} else {
ret = engine->add_request(request);
- request->tail = intel_ring_get_tail(ringbuf);
+ request->tail = intel_ring_get_tail(ring);
}
/* Not allowed to fail! */
WARN(ret, "emit|add_request failed: %d!\n", ret);
/* Sanity check that the reserved size was large enough. */
- ret = intel_ring_get_tail(ringbuf) - request_start;
+ ret = intel_ring_get_tail(ring) - request_start;
if (ret < 0)
- ret += ringbuf->size;
+ ret += ring->size;
WARN_ONCE(ret > reserved_tail,
"Not enough space reserved (%d bytes) "
"for adding the request (%d bytes)\n",
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index ed3f34dfeecd..f5f651be6790 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -61,7 +61,7 @@ struct drm_i915_gem_request {
*/
struct i915_gem_context *ctx;
struct intel_engine_cs *engine;
- struct intel_ringbuffer *ringbuf;
+ struct intel_ringbuffer *ring;
struct intel_signal_node signaling;
/** GEM sequence number associated with the previous request,
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 4d39c7284605..09997c6adcd2 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -1091,7 +1091,7 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
request = i915_gem_find_active_request(engine);
if (request) {
struct i915_address_space *vm;
- struct intel_ringbuffer *rb;
+ struct intel_ringbuffer *ring;
vm = request->ctx->ppgtt ?
&request->ctx->ppgtt->base : &ggtt->base;
@@ -1108,7 +1108,7 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
if (HAS_BROKEN_CS_TLB(dev_priv))
error->ring[i].wa_batchbuffer =
i915_error_ggtt_object_create(dev_priv,
- engine->scratch.obj);
+ engine->scratch.obj);
if (request->pid) {
struct task_struct *task;
@@ -1125,23 +1125,21 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
error->simulated |=
request->ctx->flags & CONTEXT_NO_ERROR_CAPTURE;
- rb = request->ringbuf;
- error->ring[i].cpu_ring_head = rb->head;
- error->ring[i].cpu_ring_tail = rb->tail;
+ ring = request->ring;
+ error->ring[i].cpu_ring_head = ring->head;
+ error->ring[i].cpu_ring_tail = ring->tail;
error->ring[i].ringbuffer =
i915_error_ggtt_object_create(dev_priv,
- rb->obj);
+ ring->obj);
}
error->ring[i].hws_page =
i915_error_ggtt_object_create(dev_priv,
engine->status_page.obj);
- if (engine->wa_ctx.obj) {
- error->ring[i].wa_ctx =
- i915_error_ggtt_object_create(dev_priv,
- engine->wa_ctx.obj);
- }
+ error->ring[i].wa_ctx =
+ i915_error_ggtt_object_create(dev_priv,
+ engine->wa_ctx.obj);
i915_gem_record_active_context(engine, error, &error->ring[i]);
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 61bc2e96e180..c69b5feed1ac 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11210,7 +11210,7 @@ static int intel_gen2_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11244,7 +11244,7 @@ static int intel_gen3_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11275,7 +11275,7 @@ static int intel_gen4_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11313,7 +11313,7 @@ static int intel_gen6_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11348,7 +11348,7 @@ static int intel_gen7_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t plane_bit = 0;
int len, ret;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 275b32cce8c3..91123eada000 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -717,7 +717,7 @@ int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request
return ret;
}
- request->ringbuf = ce->ringbuf;
+ request->ring = ce->ringbuf;
if (i915.enable_guc_submission) {
/*
@@ -773,11 +773,11 @@ err_unpin:
static int
intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ringbuf = request->ringbuf;
+ struct intel_ringbuffer *ring = request->ring;
struct intel_engine_cs *engine = request->engine;
- intel_ring_advance(ringbuf);
- request->tail = ringbuf->tail;
+ intel_ring_advance(ring);
+ request->tail = ring->tail;
/*
* Here we add two extra NOOPs as padding to avoid
@@ -785,9 +785,9 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
*
* Caller must reserve WA_TAIL_DWORDS for us!
*/
- intel_ring_emit(ringbuf, MI_NOOP);
- intel_ring_emit(ringbuf, MI_NOOP);
- intel_ring_advance(ringbuf);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
/* We keep the previous context alive until we retire the following
* request. This ensures that any the context object is still pinned
@@ -824,7 +824,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_device *dev = params->dev;
struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_ringbuffer *ringbuf = params->ctx->engine[engine->id].ringbuf;
+ struct intel_ringbuffer *ring = params->request->ring;
u64 exec_start;
int instp_mode;
u32 instp_mask;
@@ -836,7 +836,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
case I915_EXEC_CONSTANTS_REL_GENERAL:
case I915_EXEC_CONSTANTS_ABSOLUTE:
case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && engine != &dev_priv->engine[RCS]) {
+ if (instp_mode != 0 && engine->id != RCS) {
DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
return -EINVAL;
}
@@ -865,17 +865,17 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
if (ret)
return ret;
- if (engine == &dev_priv->engine[RCS] &&
+ if (engine->id == RCS &&
instp_mode != dev_priv->relative_constants_mode) {
ret = intel_ring_begin(params->request, 4);
if (ret)
return ret;
- intel_ring_emit(ringbuf, MI_NOOP);
- intel_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ringbuf, INSTPM);
- intel_ring_emit(ringbuf, instp_mask << 16 | instp_mode);
- intel_ring_advance(ringbuf);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
+ intel_ring_emit_reg(ring, INSTPM);
+ intel_ring_emit(ring, instp_mask << 16 | instp_mode);
+ intel_ring_advance(ring);
dev_priv->relative_constants_mode = instp_mode;
}
@@ -1033,7 +1033,7 @@ static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
int ret, i;
struct intel_engine_cs *engine = req->engine;
- struct intel_ringbuffer *ringbuf = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct i915_workarounds *w = &req->i915->workarounds;
if (w->count == 0)
@@ -1048,14 +1048,14 @@ static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
if (ret)
return ret;
- intel_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(w->count));
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(w->count));
for (i = 0; i < w->count; i++) {
- intel_ring_emit_reg(ringbuf, w->reg[i].addr);
- intel_ring_emit(ringbuf, w->reg[i].value);
+ intel_ring_emit_reg(ring, w->reg[i].addr);
+ intel_ring_emit(ring, w->reg[i].value);
}
- intel_ring_emit(ringbuf, MI_NOOP);
+ intel_ring_emit(ring, MI_NOOP);
- intel_ring_advance(ringbuf);
+ intel_ring_advance(ring);
engine->gpu_caches_dirty = true;
ret = logical_ring_flush_all_caches(req);
@@ -1545,7 +1545,7 @@ static int gen9_init_render_ring(struct intel_engine_cs *engine)
static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
{
struct i915_hw_ppgtt *ppgtt = req->ctx->ppgtt;
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
const int num_lri_cmds = GEN8_LEGACY_PDPES * 2;
int i, ret;
@@ -1572,7 +1572,7 @@ static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
static int gen8_emit_bb_start(struct drm_i915_gem_request *req,
u64 offset, unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
bool ppgtt = !(dispatch_flags & I915_DISPATCH_SECURE);
int ret;
@@ -1629,8 +1629,7 @@ static int gen8_emit_flush(struct drm_i915_gem_request *request,
u32 invalidate_domains,
u32 unused)
{
- struct intel_ringbuffer *ring = request->ringbuf;
- struct intel_engine_cs *engine = ring->engine;
+ struct intel_ringbuffer *ring = request->ring;
uint32_t cmd;
int ret;
@@ -1649,7 +1648,7 @@ static int gen8_emit_flush(struct drm_i915_gem_request *request,
if (invalidate_domains & I915_GEM_GPU_DOMAINS) {
cmd |= MI_INVALIDATE_TLB;
- if (engine->id == VCS)
+ if (request->engine->id == VCS)
cmd |= MI_INVALIDATE_BSD;
}
@@ -1668,7 +1667,7 @@ static int gen8_emit_flush_render(struct drm_i915_gem_request *request,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = request->ringbuf;
+ struct intel_ringbuffer *ring = request->ring;
struct intel_engine_cs *engine = request->engine;
u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
bool vf_flush_wa = false, dc_flush_wa = false;
@@ -1782,7 +1781,7 @@ static void bxt_a_seqno_barrier(struct intel_engine_cs *engine)
static int gen8_emit_request(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ring = request->ringbuf;
+ struct intel_ringbuffer *ring = request->ring;
int ret;
ret = intel_ring_begin(request, 6 + WA_TAIL_DWORDS);
@@ -1805,7 +1804,7 @@ static int gen8_emit_request(struct drm_i915_gem_request *request)
static int gen8_emit_request_render(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ring = request->ringbuf;
+ struct intel_ringbuffer *ring = request->ring;
int ret;
ret = intel_ring_begin(request, 8 + WA_TAIL_DWORDS);
diff --git a/drivers/gpu/drm/i915/intel_mocs.c b/drivers/gpu/drm/i915/intel_mocs.c
index 28c99846dccd..3a61d2c289f0 100644
--- a/drivers/gpu/drm/i915/intel_mocs.c
+++ b/drivers/gpu/drm/i915/intel_mocs.c
@@ -241,7 +241,7 @@ int intel_mocs_init_engine(struct intel_engine_cs *engine)
static int emit_mocs_control_table(struct drm_i915_gem_request *req,
const struct drm_i915_mocs_table *table)
{
- struct intel_ringbuffer *ringbuf = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
enum intel_engine_id engine = req->engine->id;
unsigned int index;
int ret;
@@ -253,11 +253,11 @@ static int emit_mocs_control_table(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_ring_emit(ringbuf, MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES));
+ intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES));
for (index = 0; index < table->size; index++) {
- intel_ring_emit_reg(ringbuf, mocs_register(engine, index));
- intel_ring_emit(ringbuf, table->table[index].control_value);
+ intel_ring_emit_reg(ring, mocs_register(engine, index));
+ intel_ring_emit(ring, table->table[index].control_value);
}
/*
@@ -269,12 +269,12 @@ static int emit_mocs_control_table(struct drm_i915_gem_request *req,
* that value to all the used entries.
*/
for (; index < GEN9_NUM_MOCS_ENTRIES; index++) {
- intel_ring_emit_reg(ringbuf, mocs_register(engine, index));
- intel_ring_emit(ringbuf, table->table[0].control_value);
+ intel_ring_emit_reg(ring, mocs_register(engine, index));
+ intel_ring_emit(ring, table->table[0].control_value);
}
- intel_ring_emit(ringbuf, MI_NOOP);
- intel_ring_advance(ringbuf);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
@@ -301,7 +301,7 @@ static inline u32 l3cc_combine(const struct drm_i915_mocs_table *table,
static int emit_mocs_l3cc_table(struct drm_i915_gem_request *req,
const struct drm_i915_mocs_table *table)
{
- struct intel_ringbuffer *ringbuf = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
unsigned int i;
int ret;
@@ -312,18 +312,18 @@ static int emit_mocs_l3cc_table(struct drm_i915_gem_request *req,
if (ret)
return ret;
- intel_ring_emit(ringbuf,
+ intel_ring_emit(ring,
MI_LOAD_REGISTER_IMM(GEN9_NUM_MOCS_ENTRIES / 2));
for (i = 0; i < table->size/2; i++) {
- intel_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_ring_emit(ringbuf, l3cc_combine(table, 2*i, 2*i+1));
+ intel_ring_emit_reg(ring, GEN9_LNCFCMOCS(i));
+ intel_ring_emit(ring, l3cc_combine(table, 2*i, 2*i+1));
}
if (table->size & 0x01) {
/* Odd table size - 1 left over */
- intel_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_ring_emit(ringbuf, l3cc_combine(table, 2*i, 0));
+ intel_ring_emit_reg(ring, GEN9_LNCFCMOCS(i));
+ intel_ring_emit(ring, l3cc_combine(table, 2*i, 0));
i++;
}
@@ -333,12 +333,12 @@ static int emit_mocs_l3cc_table(struct drm_i915_gem_request *req,
* they are reserved by the hardware.
*/
for (; i < GEN9_NUM_MOCS_ENTRIES / 2; i++) {
- intel_ring_emit_reg(ringbuf, GEN9_LNCFCMOCS(i));
- intel_ring_emit(ringbuf, l3cc_combine(table, 0, 0));
+ intel_ring_emit_reg(ring, GEN9_LNCFCMOCS(i));
+ intel_ring_emit(ring, l3cc_combine(table, 0, 0));
}
- intel_ring_emit(ringbuf, MI_NOOP);
- intel_ring_advance(ringbuf);
+ intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
return 0;
}
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 92722e614955..84b8f74bd13c 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -253,7 +253,7 @@ static int intel_overlay_on(struct intel_overlay *overlay)
overlay->active = true;
- ring = req->ringbuf;
+ ring = req->ring;
intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_ON);
intel_ring_emit(ring, overlay->flip_addr | OFC_UPDATE);
intel_ring_emit(ring, MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
@@ -295,7 +295,7 @@ static int intel_overlay_continue(struct intel_overlay *overlay,
return ret;
}
- ring = req->ringbuf;
+ ring = req->ring;
intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
intel_ring_emit(ring, flip_addr);
intel_ring_advance(ring);
@@ -362,7 +362,7 @@ static int intel_overlay_off(struct intel_overlay *overlay)
return ret;
}
- ring = req->ringbuf;
+ ring = req->ring;
/* wait for overlay to go idle */
intel_ring_emit(ring, MI_OVERLAY_FLIP | MI_OVERLAY_CONTINUE);
intel_ring_emit(ring, flip_addr);
@@ -438,7 +438,7 @@ static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
return ret;
}
- ring = req->ringbuf;
+ ring = req->ring;
intel_ring_emit(ring,
MI_WAIT_FOR_EVENT | MI_WAIT_FOR_OVERLAY_FLIP);
intel_ring_emit(ring, MI_NOOP);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 60cd36f64c61..536c9d124383 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -70,7 +70,7 @@ gen2_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
u32 cmd;
int ret;
@@ -97,7 +97,7 @@ gen4_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
u32 cmd;
int ret;
@@ -187,7 +187,7 @@ gen4_render_ring_flush(struct drm_i915_gem_request *req,
static int
intel_emit_post_sync_nonzero_flush(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
u32 scratch_addr =
req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
@@ -224,7 +224,7 @@ static int
gen6_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
u32 scratch_addr =
req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
u32 flags = 0;
@@ -277,7 +277,7 @@ gen6_render_ring_flush(struct drm_i915_gem_request *req,
static int
gen7_render_ring_cs_stall_wa(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 4);
@@ -299,7 +299,7 @@ static int
gen7_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
u32 scratch_addr =
req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
u32 flags = 0;
@@ -364,7 +364,7 @@ static int
gen8_emit_pipe_control(struct drm_i915_gem_request *req,
u32 flags, u32 scratch_addr)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 6);
@@ -685,7 +685,7 @@ err:
static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct i915_workarounds *w = &req->i915->workarounds;
int ret, i;
@@ -1328,7 +1328,7 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 8
- struct intel_ringbuffer *signaller = signaller_req->ringbuf;
+ struct intel_ringbuffer *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
enum intel_engine_id id;
@@ -1370,7 +1370,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 6
- struct intel_ringbuffer *signaller = signaller_req->ringbuf;
+ struct intel_ringbuffer *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
enum intel_engine_id id;
@@ -1409,7 +1409,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
static int gen6_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
- struct intel_ringbuffer *signaller = signaller_req->ringbuf;
+ struct intel_ringbuffer *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *useless;
enum intel_engine_id id;
@@ -1453,7 +1453,7 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
static int
gen6_add_request(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
if (req->engine->semaphore.signal)
@@ -1477,7 +1477,7 @@ static int
gen8_render_add_request(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
if (engine->semaphore.signal)
@@ -1522,7 +1522,7 @@ gen8_ring_sync(struct drm_i915_gem_request *waiter_req,
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_ringbuffer *waiter = waiter_req->ringbuf;
+ struct intel_ringbuffer *waiter = waiter_req->ring;
struct drm_i915_private *dev_priv = waiter_req->i915;
u64 offset = GEN8_WAIT_OFFSET(waiter_req->engine, signaller->id);
struct i915_hw_ppgtt *ppgtt;
@@ -1556,7 +1556,7 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_ringbuffer *waiter = waiter_req->ringbuf;
+ struct intel_ringbuffer *waiter = waiter_req->ring;
u32 dw1 = MI_SEMAPHORE_MBOX |
MI_SEMAPHORE_COMPARE |
MI_SEMAPHORE_REGISTER;
@@ -1690,7 +1690,7 @@ bsd_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -1706,7 +1706,7 @@ bsd_ring_flush(struct drm_i915_gem_request *req,
static int
i9xx_add_request(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 4);
@@ -1784,7 +1784,7 @@ i965_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 length,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -1811,7 +1811,7 @@ i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
u32 cs_offset = req->engine->scratch.gtt_offset;
int ret;
@@ -1873,7 +1873,7 @@ i915_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -2310,7 +2310,7 @@ int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
*/
request->reserved_space += LEGACY_REQUEST_SIZE;
- request->ringbuf = request->engine->buffer;
+ request->ring = request->engine->buffer;
ret = intel_ring_begin(request, 0);
if (ret)
@@ -2322,12 +2322,12 @@ int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
{
- struct intel_ringbuffer *ringbuf = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
struct drm_i915_gem_request *target;
- intel_ring_update_space(ringbuf);
- if (ringbuf->space >= bytes)
+ intel_ring_update_space(ring);
+ if (ring->space >= bytes)
return 0;
/*
@@ -2349,12 +2349,12 @@ static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
* from multiple ringbuffers. Here, we must ignore any that
* aren't from the ringbuffer we're considering.
*/
- if (target->ringbuf != ringbuf)
+ if (target->ring != ring)
continue;
/* Would completion of this request free enough space? */
- space = __intel_ring_space(target->postfix, ringbuf->tail,
- ringbuf->size);
+ space = __intel_ring_space(target->postfix, ring->tail,
+ ring->size);
if (space >= bytes)
break;
}
@@ -2367,9 +2367,9 @@ static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
{
- struct intel_ringbuffer *ringbuf = req->ringbuf;
- int remain_actual = ringbuf->size - ringbuf->tail;
- int remain_usable = ringbuf->effective_size - ringbuf->tail;
+ struct intel_ringbuffer *ring = req->ring;
+ int remain_actual = ring->size - ring->tail;
+ int remain_usable = ring->effective_size - ring->tail;
int bytes = num_dwords * sizeof(u32);
int total_bytes, wait_bytes;
bool need_wrap = false;
@@ -2396,35 +2396,35 @@ int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
wait_bytes = total_bytes;
}
- if (wait_bytes > ringbuf->space) {
+ if (wait_bytes > ring->space) {
int ret = wait_for_space(req, wait_bytes);
if (unlikely(ret))
return ret;
- intel_ring_update_space(ringbuf);
- if (unlikely(ringbuf->space < wait_bytes))
+ intel_ring_update_space(ring);
+ if (unlikely(ring->space < wait_bytes))
return -EAGAIN;
}
if (unlikely(need_wrap)) {
- GEM_BUG_ON(remain_actual > ringbuf->space);
- GEM_BUG_ON(ringbuf->tail + remain_actual > ringbuf->size);
+ GEM_BUG_ON(remain_actual > ring->space);
+ GEM_BUG_ON(ring->tail + remain_actual > ring->size);
/* Fill the tail with MI_NOOP */
- memset(ringbuf->vaddr + ringbuf->tail, 0, remain_actual);
- ringbuf->tail = 0;
- ringbuf->space -= remain_actual;
+ memset(ring->vaddr + ring->tail, 0, remain_actual);
+ ring->tail = 0;
+ ring->space -= remain_actual;
}
- ringbuf->space -= bytes;
- GEM_BUG_ON(ringbuf->space < 0);
+ ring->space -= bytes;
+ GEM_BUG_ON(ring->space < 0);
return 0;
}
/* Align the ring tail to a cacheline boundary */
int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int num_dwords =
(ring->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
int ret;
@@ -2531,7 +2531,7 @@ static void gen6_bsd_ring_write_tail(struct intel_engine_cs *engine,
static int gen6_bsd_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
uint32_t cmd;
int ret;
@@ -2577,7 +2577,7 @@ gen8_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
bool ppgtt = USES_PPGTT(req->i915) &&
!(dispatch_flags & I915_DISPATCH_SECURE);
int ret;
@@ -2603,7 +2603,7 @@ hsw_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -2628,7 +2628,7 @@ gen6_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -2651,7 +2651,7 @@ gen6_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
static int gen6_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_ringbuffer *ring = req->ringbuf;
+ struct intel_ringbuffer *ring = req->ring;
uint32_t cmd;
int ret;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 27/64] drm/i915: Rename backpointer from intel_ringbuffer to intel_engine_cs
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (24 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 26/64] drm/i915: Rename request->ringbuf to request->ring Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 28/64] drm/i915: Rename intel_context[engine].ringbuf Chris Wilson
` (37 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Having ringbuf->ring point to an engine is confusing, so rename it once
again to ring->engine.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_ringbuffer.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 536c9d124383..da5ec5bb5516 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2174,8 +2174,8 @@ static void intel_ring_context_unpin(struct i915_gem_context *ctx,
i915_gem_context_put(ctx);
}
-static int intel_init_ring_buffer(struct drm_device *dev,
- struct intel_engine_cs *engine)
+static int intel_init_engine(struct drm_device *dev,
+ struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_ringbuffer *ringbuf;
@@ -2887,7 +2887,7 @@ int intel_init_render_ring_buffer(struct drm_device *dev)
engine->init_hw = init_render_ring;
engine->cleanup = render_ring_cleanup;
- ret = intel_init_ring_buffer(dev, engine);
+ ret = intel_init_engine(dev, engine);
if (ret)
return ret;
@@ -2936,7 +2936,7 @@ int intel_init_bsd_ring_buffer(struct drm_device *dev)
engine->irq_enable_mask = I915_BSD_USER_INTERRUPT;
}
- return intel_init_ring_buffer(dev, engine);
+ return intel_init_engine(dev, engine);
}
/**
@@ -2959,7 +2959,7 @@ int intel_init_bsd2_ring_buffer(struct drm_device *dev)
engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_VCS2_IRQ_SHIFT;
- return intel_init_ring_buffer(dev, engine);
+ return intel_init_engine(dev, engine);
}
int intel_init_blt_ring_buffer(struct drm_device *dev)
@@ -2982,7 +2982,7 @@ int intel_init_blt_ring_buffer(struct drm_device *dev)
else
engine->irq_enable_mask = GT_BLT_USER_INTERRUPT;
- return intel_init_ring_buffer(dev, engine);
+ return intel_init_engine(dev, engine);
}
int intel_init_vebox_ring_buffer(struct drm_device *dev)
@@ -3009,7 +3009,7 @@ int intel_init_vebox_ring_buffer(struct drm_device *dev)
engine->irq_disable = hsw_vebox_irq_disable;
}
- return intel_init_ring_buffer(dev, engine);
+ return intel_init_engine(dev, engine);
}
int
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 28/64] drm/i915: Rename intel_context[engine].ringbuf
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (25 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 27/64] drm/i915: Rename backpointer from intel_ringbuffer to intel_engine_cs Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 29/64] drm/i915: Rename struct intel_ringbuffer to struct intel_ring Chris Wilson
` (36 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Perform s/ringbuf/ring/ on the context struct for consistency with the
ring/engine split.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 8 ++++----
drivers/gpu/drm/i915/i915_drv.h | 2 +-
drivers/gpu/drm/i915/i915_gem_context.c | 4 ++--
drivers/gpu/drm/i915/i915_guc_submission.c | 2 +-
drivers/gpu/drm/i915/intel_lrc.c | 33 ++++++++++++++----------------
5 files changed, 23 insertions(+), 26 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 72950b9b3e9c..ebc9d10c6ec4 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -425,8 +425,8 @@ static int per_file_ctx_stats(int id, void *ptr, void *data)
for (n = 0; n < ARRAY_SIZE(ctx->engine); n++) {
if (ctx->engine[n].state)
per_file_stats(0, ctx->engine[n].state, data);
- if (ctx->engine[n].ringbuf)
- per_file_stats(0, ctx->engine[n].ringbuf->obj, data);
+ if (ctx->engine[n].ring)
+ per_file_stats(0, ctx->engine[n].ring->obj, data);
}
return 0;
@@ -2064,8 +2064,8 @@ static int i915_context_status(struct seq_file *m, void *unused)
seq_putc(m, ce->initialised ? 'I' : 'i');
if (ce->state)
describe_obj(m, ce->state);
- if (ce->ringbuf)
- describe_ctx_ringbuf(m, ce->ringbuf);
+ if (ce->ring)
+ describe_ctx_ringbuf(m, ce->ring);
seq_putc(m, '\n');
}
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 0991e59a5d37..477074c4a9cf 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -891,7 +891,7 @@ struct i915_gem_context {
struct intel_context {
struct drm_i915_gem_object *state;
- struct intel_ringbuffer *ringbuf;
+ struct intel_ringbuffer *ring;
struct i915_vma *lrc_vma;
uint32_t *lrc_reg_state;
u64 lrc_desc;
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 271aeba6f3e5..79e409a3d2d4 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -173,8 +173,8 @@ void i915_gem_context_free(struct kref *ctx_ref)
continue;
WARN_ON(ce->pin_count);
- if (ce->ringbuf)
- intel_ringbuffer_free(ce->ringbuf);
+ if (ce->ring)
+ intel_ringbuffer_free(ce->ring);
i915_gem_object_put(ce->state);
}
diff --git a/drivers/gpu/drm/i915/i915_guc_submission.c b/drivers/gpu/drm/i915/i915_guc_submission.c
index 01c1c1671811..eccd34832fe6 100644
--- a/drivers/gpu/drm/i915/i915_guc_submission.c
+++ b/drivers/gpu/drm/i915/i915_guc_submission.c
@@ -363,7 +363,7 @@ static void guc_init_ctx_desc(struct intel_guc *guc,
lrc->context_id = (client->ctx_index << GUC_ELC_CTXID_OFFSET) |
(engine->guc_id << GUC_ELC_ENGINE_OFFSET);
- obj = ce->ringbuf->obj;
+ obj = ce->ring->obj;
gfx_addr = i915_gem_obj_ggtt_offset(obj);
lrc->ring_begin = gfx_addr;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 91123eada000..2473c2cbd1a5 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -482,11 +482,8 @@ static void execlists_context_unqueue(struct intel_engine_cs *engine)
* resubmit the request. See gen8_emit_request() for where we
* prepare the padding after the end of the request.
*/
- struct intel_ringbuffer *ringbuf;
-
- ringbuf = req0->ctx->engine[engine->id].ringbuf;
req0->tail += 8;
- req0->tail &= ringbuf->size - 1;
+ req0->tail &= req0->ring->size - 1;
}
execlists_submit_requests(req0, req1);
@@ -717,7 +714,7 @@ int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request
return ret;
}
- request->ring = ce->ringbuf;
+ request->ring = ce->ring;
if (i915.enable_guc_submission) {
/*
@@ -979,14 +976,14 @@ static int intel_lr_context_pin(struct i915_gem_context *ctx,
lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
- ret = intel_pin_and_map_ringbuffer_obj(dev_priv, ce->ringbuf);
+ ret = intel_pin_and_map_ringbuffer_obj(dev_priv, ce->ring);
if (ret)
goto unpin_map;
ce->lrc_vma = i915_gem_obj_to_ggtt(ce->state);
intel_lr_context_descriptor_update(ctx, engine);
- lrc_reg_state[CTX_RING_BUFFER_START+1] = ce->ringbuf->vma->node.start;
+ lrc_reg_state[CTX_RING_BUFFER_START+1] = ce->ring->vma->node.start;
ce->lrc_reg_state = lrc_reg_state;
ce->state->dirty = true;
@@ -1017,7 +1014,7 @@ void intel_lr_context_unpin(struct i915_gem_context *ctx,
if (--ce->pin_count)
return;
- intel_unpin_ringbuffer_obj(ce->ringbuf);
+ intel_unpin_ringbuffer_obj(ce->ring);
i915_gem_object_unpin_map(ce->state);
i915_gem_object_ggtt_unpin(ce->state);
@@ -2470,7 +2467,7 @@ static int execlists_context_deferred_alloc(struct i915_gem_context *ctx,
struct drm_i915_gem_object *ctx_obj;
struct intel_context *ce = &ctx->engine[engine->id];
uint32_t context_size;
- struct intel_ringbuffer *ringbuf;
+ struct intel_ringbuffer *ring;
int ret;
WARN_ON(ce->state);
@@ -2486,29 +2483,29 @@ static int execlists_context_deferred_alloc(struct i915_gem_context *ctx,
return PTR_ERR(ctx_obj);
}
- ringbuf = intel_engine_create_ringbuffer(engine, ctx->ring_size);
- if (IS_ERR(ringbuf)) {
- ret = PTR_ERR(ringbuf);
+ ring = intel_engine_create_ringbuffer(engine, ctx->ring_size);
+ if (IS_ERR(ring)) {
+ ret = PTR_ERR(ring);
goto error_deref_obj;
}
- ret = populate_lr_context(ctx, ctx_obj, engine, ringbuf);
+ ret = populate_lr_context(ctx, ctx_obj, engine, ring);
if (ret) {
DRM_DEBUG_DRIVER("Failed to populate LRC: %d\n", ret);
goto error_ringbuf;
}
- ce->ringbuf = ringbuf;
+ ce->ring = ring;
ce->state = ctx_obj;
ce->initialised = engine->init_context == NULL;
return 0;
error_ringbuf:
- intel_ringbuffer_free(ringbuf);
+ intel_ringbuffer_free(ring);
error_deref_obj:
i915_gem_object_put(ctx_obj);
- ce->ringbuf = NULL;
+ ce->ring = NULL;
ce->state = NULL;
return ret;
}
@@ -2539,7 +2536,7 @@ void intel_lr_context_reset(struct drm_i915_private *dev_priv,
i915_gem_object_unpin_map(ctx_obj);
- ce->ringbuf->head = 0;
- ce->ringbuf->tail = 0;
+ ce->ring->head = 0;
+ ce->ring->tail = 0;
}
}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 29/64] drm/i915: Rename struct intel_ringbuffer to struct intel_ring
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (26 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 28/64] drm/i915: Rename intel_context[engine].ringbuf Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 30/64] drm/i915: Rename residual ringbuf parameters Chris Wilson
` (35 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
The state stored in this struct is not only the information about the
buffer object, but the ring used to communicate with the hardware. Using
buffer here is overly specific and, for me at least, conflates with the
notion of buffer objects themselves.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 11 ++-
drivers/gpu/drm/i915/i915_drv.h | 4 +-
drivers/gpu/drm/i915/i915_gem.c | 24 +++---
drivers/gpu/drm/i915/i915_gem_context.c | 6 +-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 6 +-
drivers/gpu/drm/i915/i915_gem_gtt.c | 6 +-
drivers/gpu/drm/i915/i915_gem_request.c | 6 +-
drivers/gpu/drm/i915/i915_gem_request.h | 2 +-
drivers/gpu/drm/i915/i915_gpu_error.c | 8 +-
drivers/gpu/drm/i915/i915_irq.c | 14 ++--
drivers/gpu/drm/i915/intel_display.c | 10 +--
drivers/gpu/drm/i915/intel_lrc.c | 34 ++++----
drivers/gpu/drm/i915/intel_mocs.c | 4 +-
drivers/gpu/drm/i915/intel_overlay.c | 8 +-
drivers/gpu/drm/i915/intel_ringbuffer.c | 128 ++++++++++++++---------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 51 ++++++------
16 files changed, 160 insertions(+), 162 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index ebc9d10c6ec4..fc337f5a593a 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -1417,7 +1417,7 @@ static int i915_hangcheck_info(struct seq_file *m, void *unused)
intel_runtime_pm_get(dev_priv);
for_each_engine_id(engine, dev_priv, id) {
- acthd[id] = intel_ring_get_active_head(engine);
+ acthd[id] = intel_engine_get_active_head(engine);
seqno[id] = intel_engine_get_seqno(engine);
}
@@ -2015,12 +2015,11 @@ static int i915_gem_framebuffer_info(struct seq_file *m, void *data)
return 0;
}
-static void describe_ctx_ringbuf(struct seq_file *m,
- struct intel_ringbuffer *ringbuf)
+static void describe_ctx_ring(struct seq_file *m, struct intel_ring *ring)
{
seq_printf(m, " (ringbuffer, space: %d, head: %u, tail: %u, last head: %d)",
- ringbuf->space, ringbuf->head, ringbuf->tail,
- ringbuf->last_retired_head);
+ ring->space, ring->head, ring->tail,
+ ring->last_retired_head);
}
static int i915_context_status(struct seq_file *m, void *unused)
@@ -2065,7 +2064,7 @@ static int i915_context_status(struct seq_file *m, void *unused)
if (ce->state)
describe_obj(m, ce->state);
if (ce->ring)
- describe_ctx_ringbuf(m, ce->ring);
+ describe_ctx_ring(m, ce->ring);
seq_putc(m, '\n');
}
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 477074c4a9cf..f59dde1b4156 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -515,7 +515,7 @@ struct drm_i915_error_state {
bool waiting;
int num_waiters;
int hangcheck_score;
- enum intel_ring_hangcheck_action hangcheck_action;
+ enum intel_engine_hangcheck_action hangcheck_action;
int num_requests;
/* our own tracking of ring head and tail */
@@ -891,7 +891,7 @@ struct i915_gem_context {
struct intel_context {
struct drm_i915_gem_object *state;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
struct i915_vma *lrc_vma;
uint32_t *lrc_reg_state;
u64 lrc_desc;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 337282c9a070..5caf4dc14a46 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2486,7 +2486,7 @@ static void i915_gem_reset_engine_status(struct intel_engine_cs *engine)
static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
{
- struct intel_ringbuffer *buffer;
+ struct intel_ring *ring;
while (!list_empty(&engine->active_list)) {
struct drm_i915_gem_object *obj;
@@ -2502,7 +2502,7 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
* (lockless) lookup doesn't try and wait upon the request as we
* reset it.
*/
- intel_ring_init_seqno(engine, engine->last_submitted_seqno);
+ intel_engine_init_seqno(engine, engine->last_submitted_seqno);
/*
* Clear the execlists queue up before freeing the requests, as those
@@ -2541,9 +2541,9 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
* upon reset is less than when we start. Do one more pass over
* all the ringbuffers to reset last_retired_head.
*/
- list_for_each_entry(buffer, &engine->buffers, link) {
- buffer->last_retired_head = buffer->tail;
- intel_ring_update_space(buffer);
+ list_for_each_entry(ring, &engine->buffers, link) {
+ ring->last_retired_head = ring->tail;
+ intel_ring_update_space(ring);
}
}
@@ -2862,7 +2862,7 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
i915_gem_object_retire_request(obj, from_req);
} else {
- int idx = intel_ring_sync_index(from, to);
+ int idx = intel_engine_sync_index(from, to);
u32 seqno = i915_gem_request_get_seqno(from_req);
WARN_ON(!to_req);
@@ -4466,13 +4466,13 @@ int i915_gem_init_engines(struct drm_device *dev)
return 0;
cleanup_vebox_ring:
- intel_cleanup_engine(&dev_priv->engine[VECS]);
+ intel_engine_cleanup(&dev_priv->engine[VECS]);
cleanup_blt_ring:
- intel_cleanup_engine(&dev_priv->engine[BCS]);
+ intel_engine_cleanup(&dev_priv->engine[BCS]);
cleanup_bsd_ring:
- intel_cleanup_engine(&dev_priv->engine[VCS]);
+ intel_engine_cleanup(&dev_priv->engine[VCS]);
cleanup_render_ring:
- intel_cleanup_engine(&dev_priv->engine[RCS]);
+ intel_engine_cleanup(&dev_priv->engine[RCS]);
return ret;
}
@@ -4574,8 +4574,8 @@ int i915_gem_init(struct drm_device *dev)
if (!i915.enable_execlists) {
dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
dev_priv->gt.init_engines = i915_gem_init_engines;
- dev_priv->gt.cleanup_engine = intel_cleanup_engine;
- dev_priv->gt.stop_engine = intel_stop_engine;
+ dev_priv->gt.cleanup_engine = intel_engine_cleanup;
+ dev_priv->gt.stop_engine = intel_engine_stop;
} else {
dev_priv->gt.execbuf_submit = intel_execlists_submission;
dev_priv->gt.init_engines = intel_logical_rings_init;
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 79e409a3d2d4..d0ce01d3684c 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -174,7 +174,7 @@ void i915_gem_context_free(struct kref *ctx_ref)
WARN_ON(ce->pin_count);
if (ce->ring)
- intel_ringbuffer_free(ce->ring);
+ intel_ring_free(ce->ring);
i915_gem_object_put(ce->state);
}
@@ -552,7 +552,7 @@ static inline int
mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
{
struct drm_i915_private *dev_priv = req->i915;
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
u32 flags = hw_flags | MI_MM_SPACE_GTT;
const int num_rings =
/* Use an extended w/a on ivb+ if signalling from other rings */
@@ -654,7 +654,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
static int remap_l3(struct drm_i915_gem_request *req, int slice)
{
u32 *remap_info = req->i915->l3_parity.remap_info[slice];
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int i, ret;
if (!remap_info)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index d77b01fa4332..e05c205f2e19 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -975,7 +975,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
/* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
- return intel_ring_invalidate_all_caches(req);
+ return intel_engine_invalidate_all_caches(req);
}
static bool
@@ -1166,7 +1166,7 @@ i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params)
static int
i915_reset_gen7_sol_offsets(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret, i;
if (!IS_GEN7(req->i915) || req->engine->id != RCS) {
@@ -1296,7 +1296,7 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
if (params->engine->id == RCS &&
instp_mode != dev_priv->relative_constants_mode) {
- struct intel_ringbuffer *ring = params->request->ring;
+ struct intel_ring *ring = params->request->ring;
ret = intel_ring_begin(params->request, 4);
if (ret)
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 1d4ff1a7e3e0..c5357802171b 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -669,7 +669,7 @@ static int gen8_write_pdp(struct drm_i915_gem_request *req,
unsigned entry,
dma_addr_t addr)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
BUG_ON(entry >= 4);
@@ -1660,7 +1660,7 @@ static uint32_t get_pd_offset(struct i915_hw_ppgtt *ppgtt)
static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
@@ -1688,7 +1688,7 @@ static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 9cab2eb16868..4dc51e199269 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -244,7 +244,7 @@ static int i915_gem_init_seqno(struct drm_i915_private *dev_priv, u32 seqno)
/* Finally reset hw state */
for_each_engine(engine, dev_priv)
- intel_ring_init_seqno(engine, seqno);
+ intel_engine_init_seqno(engine, seqno);
return 0;
}
@@ -424,7 +424,7 @@ void __i915_add_request(struct drm_i915_gem_request *request,
bool flush_caches)
{
struct intel_engine_cs *engine;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
u32 request_start;
u32 reserved_tail;
int ret;
@@ -455,7 +455,7 @@ void __i915_add_request(struct drm_i915_gem_request *request,
if (i915.enable_execlists)
ret = logical_ring_flush_all_caches(request);
else
- ret = intel_ring_flush_all_caches(request);
+ ret = intel_engine_flush_all_caches(request);
/* Not allowed to fail! */
WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index f5f651be6790..928fc8baf435 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -61,7 +61,7 @@ struct drm_i915_gem_request {
*/
struct i915_gem_context *ctx;
struct intel_engine_cs *engine;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
struct intel_signal_node signaling;
/** GEM sequence number associated with the previous request,
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 09997c6adcd2..2fbe81d51af1 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -221,7 +221,7 @@ static void print_error_buffers(struct drm_i915_error_state_buf *m,
}
}
-static const char *hangcheck_action_to_str(enum intel_ring_hangcheck_action a)
+static const char *hangcheck_action_to_str(enum intel_engine_hangcheck_action a)
{
switch (a) {
case HANGCHECK_IDLE:
@@ -882,7 +882,7 @@ static void gen8_record_semaphore_state(struct drm_i915_private *dev_priv,
signal_offset = (GEN8_SIGNAL_OFFSET(engine, id) & (PAGE_SIZE - 1))
/ 4;
tmp = error->semaphore_obj->pages[0];
- idx = intel_ring_sync_index(engine, to);
+ idx = intel_engine_sync_index(engine, to);
ering->semaphore_mboxes[idx] = tmp[signal_offset];
ering->semaphore_seqno[idx] = engine->semaphore.sync_seqno[idx];
@@ -983,7 +983,7 @@ static void i915_record_ring_state(struct drm_i915_private *dev_priv,
ering->waiting = intel_engine_has_waiter(engine);
ering->instpm = I915_READ(RING_INSTPM(engine->mmio_base));
- ering->acthd = intel_ring_get_active_head(engine);
+ ering->acthd = intel_engine_get_active_head(engine);
ering->seqno = intel_engine_get_seqno(engine);
ering->last_seqno = engine->last_submitted_seqno;
ering->start = I915_READ_START(engine);
@@ -1091,7 +1091,7 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
request = i915_gem_find_active_request(engine);
if (request) {
struct i915_address_space *vm;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
vm = request->ctx->ppgtt ?
&request->ctx->ppgtt->base : &ggtt->base;
diff --git a/drivers/gpu/drm/i915/i915_irq.c b/drivers/gpu/drm/i915/i915_irq.c
index ddd6bb6ef165..13026e3bd41c 100644
--- a/drivers/gpu/drm/i915/i915_irq.c
+++ b/drivers/gpu/drm/i915/i915_irq.c
@@ -2990,7 +2990,7 @@ static bool subunits_stuck(struct intel_engine_cs *engine)
return stuck;
}
-static enum intel_ring_hangcheck_action
+static enum intel_engine_hangcheck_action
head_stuck(struct intel_engine_cs *engine, u64 acthd)
{
if (acthd != engine->hangcheck.acthd) {
@@ -3008,11 +3008,11 @@ head_stuck(struct intel_engine_cs *engine, u64 acthd)
return HANGCHECK_HUNG;
}
-static enum intel_ring_hangcheck_action
-ring_stuck(struct intel_engine_cs *engine, u64 acthd)
+static enum intel_engine_hangcheck_action
+engine_stuck(struct intel_engine_cs *engine, u64 acthd)
{
struct drm_i915_private *dev_priv = engine->i915;
- enum intel_ring_hangcheck_action ha;
+ enum intel_engine_hangcheck_action ha;
u32 tmp;
ha = head_stuck(engine, acthd);
@@ -3121,7 +3121,7 @@ static void i915_hangcheck_elapsed(struct work_struct *work)
if (engine->irq_seqno_barrier)
engine->irq_seqno_barrier(engine);
- acthd = intel_ring_get_active_head(engine);
+ acthd = intel_engine_get_active_head(engine);
seqno = intel_engine_get_seqno(engine);
/* Reset stuck interrupts between batch advances */
@@ -3151,8 +3151,8 @@ static void i915_hangcheck_elapsed(struct work_struct *work)
* being repeatedly kicked and so responsible
* for stalling the machine.
*/
- engine->hangcheck.action = ring_stuck(engine,
- acthd);
+ engine->hangcheck.action =
+ engine_stuck(engine, acthd);
switch (engine->hangcheck.action) {
case HANGCHECK_IDLE:
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index c69b5feed1ac..d90a3746bce8 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11210,7 +11210,7 @@ static int intel_gen2_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11244,7 +11244,7 @@ static int intel_gen3_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
u32 flip_mask;
int ret;
@@ -11275,7 +11275,7 @@ static int intel_gen4_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11313,7 +11313,7 @@ static int intel_gen6_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct drm_i915_private *dev_priv = to_i915(dev);
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t pf, pipesrc;
@@ -11348,7 +11348,7 @@ static int intel_gen7_queue_flip(struct drm_device *dev,
struct drm_i915_gem_request *req,
uint32_t flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
uint32_t plane_bit = 0;
int len, ret;
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 2473c2cbd1a5..0e8b2cd89aca 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -770,7 +770,7 @@ err_unpin:
static int
intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ring = request->ring;
+ struct intel_ring *ring = request->ring;
struct intel_engine_cs *engine = request->engine;
intel_ring_advance(ring);
@@ -821,7 +821,7 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
struct drm_device *dev = params->dev;
struct intel_engine_cs *engine = params->engine;
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_ringbuffer *ring = params->request->ring;
+ struct intel_ring *ring = params->request->ring;
u64 exec_start;
int instp_mode;
u32 instp_mask;
@@ -976,7 +976,7 @@ static int intel_lr_context_pin(struct i915_gem_context *ctx,
lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
- ret = intel_pin_and_map_ringbuffer_obj(dev_priv, ce->ring);
+ ret = intel_pin_and_map_ring(dev_priv, ce->ring);
if (ret)
goto unpin_map;
@@ -1014,7 +1014,7 @@ void intel_lr_context_unpin(struct i915_gem_context *ctx,
if (--ce->pin_count)
return;
- intel_unpin_ringbuffer_obj(ce->ring);
+ intel_unpin_ring(ce->ring);
i915_gem_object_unpin_map(ce->state);
i915_gem_object_ggtt_unpin(ce->state);
@@ -1030,7 +1030,7 @@ static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
int ret, i;
struct intel_engine_cs *engine = req->engine;
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct i915_workarounds *w = &req->i915->workarounds;
if (w->count == 0)
@@ -1542,7 +1542,7 @@ static int gen9_init_render_ring(struct intel_engine_cs *engine)
static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
{
struct i915_hw_ppgtt *ppgtt = req->ctx->ppgtt;
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
const int num_lri_cmds = GEN8_LEGACY_PDPES * 2;
int i, ret;
@@ -1569,7 +1569,7 @@ static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
static int gen8_emit_bb_start(struct drm_i915_gem_request *req,
u64 offset, unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
bool ppgtt = !(dispatch_flags & I915_DISPATCH_SECURE);
int ret;
@@ -1626,8 +1626,8 @@ static int gen8_emit_flush(struct drm_i915_gem_request *request,
u32 invalidate_domains,
u32 unused)
{
- struct intel_ringbuffer *ring = request->ring;
- uint32_t cmd;
+ struct intel_ring *ring = request->ring;
+ u32 cmd;
int ret;
ret = intel_ring_begin(request, 4);
@@ -1664,7 +1664,7 @@ static int gen8_emit_flush_render(struct drm_i915_gem_request *request,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = request->ring;
+ struct intel_ring *ring = request->ring;
struct intel_engine_cs *engine = request->engine;
u32 scratch_addr = engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
bool vf_flush_wa = false, dc_flush_wa = false;
@@ -1778,7 +1778,7 @@ static void bxt_a_seqno_barrier(struct intel_engine_cs *engine)
static int gen8_emit_request(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ring = request->ring;
+ struct intel_ring *ring = request->ring;
int ret;
ret = intel_ring_begin(request, 6 + WA_TAIL_DWORDS);
@@ -1801,7 +1801,7 @@ static int gen8_emit_request(struct drm_i915_gem_request *request)
static int gen8_emit_request_render(struct drm_i915_gem_request *request)
{
- struct intel_ringbuffer *ring = request->ring;
+ struct intel_ring *ring = request->ring;
int ret;
ret = intel_ring_begin(request, 8 + WA_TAIL_DWORDS);
@@ -2273,7 +2273,7 @@ static int
populate_lr_context(struct i915_gem_context *ctx,
struct drm_i915_gem_object *ctx_obj,
struct intel_engine_cs *engine,
- struct intel_ringbuffer *ringbuf)
+ struct intel_ring *ring)
{
struct drm_i915_private *dev_priv = ctx->i915;
struct i915_hw_ppgtt *ppgtt = ctx->ppgtt;
@@ -2326,7 +2326,7 @@ populate_lr_context(struct i915_gem_context *ctx,
RING_START(engine->mmio_base), 0);
ASSIGN_CTX_REG(reg_state, CTX_RING_BUFFER_CONTROL,
RING_CTL(engine->mmio_base),
- ((ringbuf->size - PAGE_SIZE) & RING_NR_PAGES) | RING_VALID);
+ ((ring->size - PAGE_SIZE) & RING_NR_PAGES) | RING_VALID);
ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_U,
RING_BBADDR_UDW(engine->mmio_base), 0);
ASSIGN_CTX_REG(reg_state, CTX_BB_HEAD_L,
@@ -2467,7 +2467,7 @@ static int execlists_context_deferred_alloc(struct i915_gem_context *ctx,
struct drm_i915_gem_object *ctx_obj;
struct intel_context *ce = &ctx->engine[engine->id];
uint32_t context_size;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
int ret;
WARN_ON(ce->state);
@@ -2483,7 +2483,7 @@ static int execlists_context_deferred_alloc(struct i915_gem_context *ctx,
return PTR_ERR(ctx_obj);
}
- ring = intel_engine_create_ringbuffer(engine, ctx->ring_size);
+ ring = intel_engine_create_ring(engine, ctx->ring_size);
if (IS_ERR(ring)) {
ret = PTR_ERR(ring);
goto error_deref_obj;
@@ -2502,7 +2502,7 @@ static int execlists_context_deferred_alloc(struct i915_gem_context *ctx,
return 0;
error_ringbuf:
- intel_ringbuffer_free(ring);
+ intel_ring_free(ring);
error_deref_obj:
i915_gem_object_put(ctx_obj);
ce->ring = NULL;
diff --git a/drivers/gpu/drm/i915/intel_mocs.c b/drivers/gpu/drm/i915/intel_mocs.c
index 3a61d2c289f0..967afebaf112 100644
--- a/drivers/gpu/drm/i915/intel_mocs.c
+++ b/drivers/gpu/drm/i915/intel_mocs.c
@@ -241,7 +241,7 @@ int intel_mocs_init_engine(struct intel_engine_cs *engine)
static int emit_mocs_control_table(struct drm_i915_gem_request *req,
const struct drm_i915_mocs_table *table)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
enum intel_engine_id engine = req->engine->id;
unsigned int index;
int ret;
@@ -301,7 +301,7 @@ static inline u32 l3cc_combine(const struct drm_i915_mocs_table *table,
static int emit_mocs_l3cc_table(struct drm_i915_gem_request *req,
const struct drm_i915_mocs_table *table)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
unsigned int i;
int ret;
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 84b8f74bd13c..a5071e281088 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -235,7 +235,7 @@ static int intel_overlay_on(struct intel_overlay *overlay)
struct drm_i915_private *dev_priv = overlay->i915;
struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
int ret;
WARN_ON(overlay->active);
@@ -270,7 +270,7 @@ static int intel_overlay_continue(struct intel_overlay *overlay,
struct drm_i915_private *dev_priv = overlay->i915;
struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
u32 flip_addr = overlay->flip_addr;
u32 tmp;
int ret;
@@ -340,7 +340,7 @@ static int intel_overlay_off(struct intel_overlay *overlay)
struct drm_i915_private *dev_priv = overlay->i915;
struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
u32 flip_addr = overlay->flip_addr;
int ret;
@@ -426,7 +426,7 @@ static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
if (I915_READ(ISR) & I915_OVERLAY_PLANE_FLIP_PENDING_INTERRUPT) {
/* synchronous slowpath */
struct drm_i915_gem_request *req;
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
req = i915_gem_request_alloc(engine, NULL);
if (IS_ERR(req))
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index da5ec5bb5516..9ac56de18bd4 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -47,7 +47,7 @@ int __intel_ring_space(int head, int tail, int size)
return space - I915_RING_FREE_SPACE;
}
-void intel_ring_update_space(struct intel_ringbuffer *ringbuf)
+void intel_ring_update_space(struct intel_ring *ringbuf)
{
if (ringbuf->last_retired_head != -1) {
ringbuf->head = ringbuf->last_retired_head;
@@ -60,9 +60,10 @@ void intel_ring_update_space(struct intel_ringbuffer *ringbuf)
static void __intel_engine_submit(struct intel_engine_cs *engine)
{
- struct intel_ringbuffer *ringbuf = engine->buffer;
- ringbuf->tail &= ringbuf->size - 1;
- engine->write_tail(engine, ringbuf->tail);
+ struct intel_ring *ring = engine->buffer;
+
+ ring->tail &= ring->size - 1;
+ engine->write_tail(engine, ring->tail);
}
static int
@@ -70,7 +71,7 @@ gen2_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
u32 cmd;
int ret;
@@ -97,7 +98,7 @@ gen4_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
u32 cmd;
int ret;
@@ -187,7 +188,7 @@ gen4_render_ring_flush(struct drm_i915_gem_request *req,
static int
intel_emit_post_sync_nonzero_flush(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
u32 scratch_addr =
req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
int ret;
@@ -224,7 +225,7 @@ static int
gen6_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
u32 scratch_addr =
req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
u32 flags = 0;
@@ -277,7 +278,7 @@ gen6_render_ring_flush(struct drm_i915_gem_request *req,
static int
gen7_render_ring_cs_stall_wa(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 4);
@@ -299,7 +300,7 @@ static int
gen7_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains, u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
u32 scratch_addr =
req->engine->scratch.gtt_offset + 2 * CACHELINE_BYTES;
u32 flags = 0;
@@ -364,7 +365,7 @@ static int
gen8_emit_pipe_control(struct drm_i915_gem_request *req,
u32 flags, u32 scratch_addr)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 6);
@@ -427,7 +428,7 @@ static void ring_write_tail(struct intel_engine_cs *engine,
I915_WRITE_TAIL(engine, value);
}
-u64 intel_ring_get_active_head(struct intel_engine_cs *engine)
+u64 intel_engine_get_active_head(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
u64 acthd;
@@ -558,8 +559,8 @@ void intel_engine_init_hangcheck(struct intel_engine_cs *engine)
static int init_ring_common(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
- struct intel_ringbuffer *ringbuf = engine->buffer;
- struct drm_i915_gem_object *obj = ringbuf->obj;
+ struct intel_ring *ring = engine->buffer;
+ struct drm_i915_gem_object *obj = ring->obj;
int ret = 0;
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
@@ -609,7 +610,7 @@ static int init_ring_common(struct intel_engine_cs *engine)
(void)I915_READ_HEAD(engine);
I915_WRITE_CTL(engine,
- ((ringbuf->size - PAGE_SIZE) & RING_NR_PAGES)
+ ((ring->size - PAGE_SIZE) & RING_NR_PAGES)
| RING_VALID);
/* If the head is still not zero, the ring is dead */
@@ -628,10 +629,10 @@ static int init_ring_common(struct intel_engine_cs *engine)
goto out;
}
- ringbuf->last_retired_head = -1;
- ringbuf->head = I915_READ_HEAD(engine);
- ringbuf->tail = I915_READ_TAIL(engine) & TAIL_ADDR;
- intel_ring_update_space(ringbuf);
+ ring->last_retired_head = -1;
+ ring->head = I915_READ_HEAD(engine);
+ ring->tail = I915_READ_TAIL(engine) & TAIL_ADDR;
+ intel_ring_update_space(ring);
intel_engine_init_hangcheck(engine);
@@ -685,7 +686,7 @@ err:
static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct i915_workarounds *w = &req->i915->workarounds;
int ret, i;
@@ -693,7 +694,7 @@ static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
return 0;
req->engine->gpu_caches_dirty = true;
- ret = intel_ring_flush_all_caches(req);
+ ret = intel_engine_flush_all_caches(req);
if (ret)
return ret;
@@ -711,7 +712,7 @@ static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
intel_ring_advance(ring);
req->engine->gpu_caches_dirty = true;
- ret = intel_ring_flush_all_caches(req);
+ ret = intel_engine_flush_all_caches(req);
if (ret)
return ret;
@@ -1328,7 +1329,7 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 8
- struct intel_ringbuffer *signaller = signaller_req->ring;
+ struct intel_ring *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
enum intel_engine_id id;
@@ -1370,7 +1371,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
#define MBOX_UPDATE_DWORDS 6
- struct intel_ringbuffer *signaller = signaller_req->ring;
+ struct intel_ring *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
enum intel_engine_id id;
@@ -1409,7 +1410,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
static int gen6_signal(struct drm_i915_gem_request *signaller_req,
unsigned int num_dwords)
{
- struct intel_ringbuffer *signaller = signaller_req->ring;
+ struct intel_ring *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *useless;
enum intel_engine_id id;
@@ -1453,7 +1454,7 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
static int
gen6_add_request(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
if (req->engine->semaphore.signal)
@@ -1477,7 +1478,7 @@ static int
gen8_render_add_request(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
if (engine->semaphore.signal)
@@ -1522,7 +1523,7 @@ gen8_ring_sync(struct drm_i915_gem_request *waiter_req,
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_ringbuffer *waiter = waiter_req->ring;
+ struct intel_ring *waiter = waiter_req->ring;
struct drm_i915_private *dev_priv = waiter_req->i915;
u64 offset = GEN8_WAIT_OFFSET(waiter_req->engine, signaller->id);
struct i915_hw_ppgtt *ppgtt;
@@ -1556,7 +1557,7 @@ gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
struct intel_engine_cs *signaller,
u32 seqno)
{
- struct intel_ringbuffer *waiter = waiter_req->ring;
+ struct intel_ring *waiter = waiter_req->ring;
u32 dw1 = MI_SEMAPHORE_MBOX |
MI_SEMAPHORE_COMPARE |
MI_SEMAPHORE_REGISTER;
@@ -1690,7 +1691,7 @@ bsd_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
u32 flush_domains)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -1706,7 +1707,7 @@ bsd_ring_flush(struct drm_i915_gem_request *req,
static int
i9xx_add_request(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 4);
@@ -1784,7 +1785,7 @@ i965_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 length,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -1811,7 +1812,7 @@ i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
u32 cs_offset = req->engine->scratch.gtt_offset;
int ret;
@@ -1873,7 +1874,7 @@ i915_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -1981,7 +1982,7 @@ static int init_phys_status_page(struct intel_engine_cs *engine)
return 0;
}
-void intel_unpin_ringbuffer_obj(struct intel_ringbuffer *ringbuf)
+void intel_unpin_ring(struct intel_ring *ringbuf)
{
GEM_BUG_ON(!ringbuf->vma);
GEM_BUG_ON(!ringbuf->vaddr);
@@ -1996,8 +1997,8 @@ void intel_unpin_ringbuffer_obj(struct intel_ringbuffer *ringbuf)
ringbuf->vma = NULL;
}
-int intel_pin_and_map_ringbuffer_obj(struct drm_i915_private *dev_priv,
- struct intel_ringbuffer *ringbuf)
+int intel_pin_and_map_ring(struct drm_i915_private *dev_priv,
+ struct intel_ring *ringbuf)
{
struct drm_i915_gem_object *obj = ringbuf->obj;
/* Ring wraparound at offset 0 sometimes hangs. No idea why. */
@@ -2048,14 +2049,14 @@ err_unpin:
return ret;
}
-static void intel_destroy_ringbuffer_obj(struct intel_ringbuffer *ringbuf)
+static void intel_destroy_ringbuffer_obj(struct intel_ring *ringbuf)
{
i915_gem_object_put(ringbuf->obj);
ringbuf->obj = NULL;
}
static int intel_alloc_ringbuffer_obj(struct drm_device *dev,
- struct intel_ringbuffer *ringbuf)
+ struct intel_ring *ringbuf)
{
struct drm_i915_gem_object *obj;
@@ -2075,10 +2076,10 @@ static int intel_alloc_ringbuffer_obj(struct drm_device *dev,
return 0;
}
-struct intel_ringbuffer *
-intel_engine_create_ringbuffer(struct intel_engine_cs *engine, int size)
+struct intel_ring *
+intel_engine_create_ring(struct intel_engine_cs *engine, int size)
{
- struct intel_ringbuffer *ring;
+ struct intel_ring *ring;
int ret;
ring = kzalloc(sizeof(*ring), GFP_KERNEL);
@@ -2116,7 +2117,7 @@ intel_engine_create_ringbuffer(struct intel_engine_cs *engine, int size)
}
void
-intel_ringbuffer_free(struct intel_ringbuffer *ring)
+intel_ring_free(struct intel_ring *ring)
{
intel_destroy_ringbuffer_obj(ring);
list_del(&ring->link);
@@ -2178,7 +2179,7 @@ static int intel_init_engine(struct drm_device *dev,
struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_ringbuffer *ringbuf;
+ struct intel_ring *ringbuf;
int ret;
WARN_ON(engine->buffer);
@@ -2208,7 +2209,7 @@ static int intel_init_engine(struct drm_device *dev,
if (ret)
goto error;
- ringbuf = intel_engine_create_ringbuffer(engine, 32 * PAGE_SIZE);
+ ringbuf = intel_engine_create_ring(engine, 32 * PAGE_SIZE);
if (IS_ERR(ringbuf)) {
ret = PTR_ERR(ringbuf);
goto error;
@@ -2226,7 +2227,7 @@ static int intel_init_engine(struct drm_device *dev,
goto error;
}
- ret = intel_pin_and_map_ringbuffer_obj(dev_priv, ringbuf);
+ ret = intel_pin_and_map_ring(dev_priv, ringbuf);
if (ret) {
DRM_ERROR("Failed to pin and map ringbuffer %s: %d\n",
engine->name, ret);
@@ -2241,11 +2242,11 @@ static int intel_init_engine(struct drm_device *dev,
return 0;
error:
- intel_cleanup_engine(engine);
+ intel_engine_cleanup(engine);
return ret;
}
-void intel_cleanup_engine(struct intel_engine_cs *engine)
+void intel_engine_cleanup(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv;
@@ -2255,11 +2256,11 @@ void intel_cleanup_engine(struct intel_engine_cs *engine)
dev_priv = engine->i915;
if (engine->buffer) {
- intel_stop_engine(engine);
+ intel_engine_stop(engine);
WARN_ON(!IS_GEN2(dev_priv) && (I915_READ_MODE(engine) & MODE_IDLE) == 0);
- intel_unpin_ringbuffer_obj(engine->buffer);
- intel_ringbuffer_free(engine->buffer);
+ intel_unpin_ring(engine->buffer);
+ intel_ring_free(engine->buffer);
engine->buffer = NULL;
}
@@ -2322,7 +2323,7 @@ int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request)
static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
struct drm_i915_gem_request *target;
@@ -2367,7 +2368,7 @@ static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int remain_actual = ring->size - ring->tail;
int remain_usable = ring->effective_size - ring->tail;
int bytes = num_dwords * sizeof(u32);
@@ -2424,7 +2425,7 @@ int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
/* Align the ring tail to a cacheline boundary */
int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int num_dwords =
(ring->tail & (CACHELINE_BYTES - 1)) / sizeof(uint32_t);
int ret;
@@ -2445,7 +2446,7 @@ int intel_ring_cacheline_align(struct drm_i915_gem_request *req)
return 0;
}
-void intel_ring_init_seqno(struct intel_engine_cs *engine, u32 seqno)
+void intel_engine_init_seqno(struct intel_engine_cs *engine, u32 seqno)
{
struct drm_i915_private *dev_priv = engine->i915;
@@ -2531,7 +2532,7 @@ static void gen6_bsd_ring_write_tail(struct intel_engine_cs *engine,
static int gen6_bsd_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
uint32_t cmd;
int ret;
@@ -2577,7 +2578,7 @@ gen8_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
bool ppgtt = USES_PPGTT(req->i915) &&
!(dispatch_flags & I915_DISPATCH_SECURE);
int ret;
@@ -2603,7 +2604,7 @@ hsw_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -2628,7 +2629,7 @@ gen6_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
u64 offset, u32 len,
unsigned dispatch_flags)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
int ret;
ret = intel_ring_begin(req, 2);
@@ -2651,7 +2652,7 @@ gen6_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
static int gen6_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate, u32 flush)
{
- struct intel_ringbuffer *ring = req->ring;
+ struct intel_ring *ring = req->ring;
uint32_t cmd;
int ret;
@@ -3013,7 +3014,7 @@ int intel_init_vebox_ring_buffer(struct drm_device *dev)
}
int
-intel_ring_flush_all_caches(struct drm_i915_gem_request *req)
+intel_engine_flush_all_caches(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
int ret;
@@ -3032,7 +3033,7 @@ intel_ring_flush_all_caches(struct drm_i915_gem_request *req)
}
int
-intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req)
+intel_engine_invalidate_all_caches(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
uint32_t flush_domains;
@@ -3052,8 +3053,7 @@ intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req)
return 0;
}
-void
-intel_stop_engine(struct intel_engine_cs *engine)
+void intel_engine_stop(struct intel_engine_cs *engine)
{
int ret;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 4eac4c066c15..590e9d006a95 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -62,7 +62,7 @@ struct intel_hw_status_page {
(i915_gem_obj_ggtt_offset(dev_priv->semaphore_obj) + \
GEN8_SEMAPHORE_OFFSET(from, (__ring)->id))
-enum intel_ring_hangcheck_action {
+enum intel_engine_hangcheck_action {
HANGCHECK_IDLE = 0,
HANGCHECK_WAIT,
HANGCHECK_ACTIVE,
@@ -72,17 +72,17 @@ enum intel_ring_hangcheck_action {
#define HANGCHECK_SCORE_RING_HUNG 31
-struct intel_ring_hangcheck {
+struct intel_engine_hangcheck {
u64 acthd;
unsigned long user_interrupts;
u32 seqno;
int score;
- enum intel_ring_hangcheck_action action;
+ enum intel_engine_hangcheck_action action;
int deadlock;
u32 instdone[I915_NUM_INSTDONE_REG];
};
-struct intel_ringbuffer {
+struct intel_ring {
struct drm_i915_gem_object *obj;
void *vaddr;
struct i915_vma *vma;
@@ -148,7 +148,7 @@ struct intel_engine_cs {
unsigned int guc_id; /* XXX same as hw_id? */
u64 fence_context;
u32 mmio_base;
- struct intel_ringbuffer *buffer;
+ struct intel_ring *buffer;
struct list_head buffers;
/* Rather than have every client wait upon all user interrupts,
@@ -328,7 +328,7 @@ struct intel_engine_cs {
struct i915_gem_context *last_context;
- struct intel_ring_hangcheck hangcheck;
+ struct intel_engine_hangcheck hangcheck;
struct {
struct drm_i915_gem_object *obj;
@@ -375,8 +375,8 @@ intel_engine_flag(const struct intel_engine_cs *engine)
}
static inline u32
-intel_ring_sync_index(struct intel_engine_cs *engine,
- struct intel_engine_cs *other)
+intel_engine_sync_index(struct intel_engine_cs *engine,
+ struct intel_engine_cs *other)
{
int idx;
@@ -438,45 +438,44 @@ intel_write_status_page(struct intel_engine_cs *engine,
#define I915_GEM_HWS_SCRATCH_INDEX 0x40
#define I915_GEM_HWS_SCRATCH_ADDR (I915_GEM_HWS_SCRATCH_INDEX << MI_STORE_DWORD_INDEX_SHIFT)
-struct intel_ringbuffer *
-intel_engine_create_ringbuffer(struct intel_engine_cs *engine, int size);
-int intel_pin_and_map_ringbuffer_obj(struct drm_i915_private *dev_priv,
- struct intel_ringbuffer *ringbuf);
-void intel_unpin_ringbuffer_obj(struct intel_ringbuffer *ringbuf);
-void intel_ringbuffer_free(struct intel_ringbuffer *ring);
+struct intel_ring *
+intel_engine_create_ring(struct intel_engine_cs *engine, int size);
+int intel_pin_and_map_ring(struct drm_i915_private *dev_priv,
+ struct intel_ring *ring);
+void intel_unpin_ring(struct intel_ring *ring);
+void intel_ring_free(struct intel_ring *ring);
-void intel_stop_engine(struct intel_engine_cs *engine);
-void intel_cleanup_engine(struct intel_engine_cs *engine);
+void intel_engine_stop(struct intel_engine_cs *engine);
+void intel_engine_cleanup(struct intel_engine_cs *engine);
int intel_ring_alloc_request_extras(struct drm_i915_gem_request *request);
int __must_check intel_ring_begin(struct drm_i915_gem_request *req, int n);
int __must_check intel_ring_cacheline_align(struct drm_i915_gem_request *req);
-static inline void intel_ring_emit(struct intel_ringbuffer *ring, u32 data)
+static inline void intel_ring_emit(struct intel_ring *ring, u32 data)
{
*(uint32_t *)(ring->vaddr + ring->tail) = data;
ring->tail += 4;
}
-static inline void intel_ring_emit_reg(struct intel_ringbuffer *ring,
- i915_reg_t reg)
+static inline void intel_ring_emit_reg(struct intel_ring *ring, i915_reg_t reg)
{
intel_ring_emit(ring, i915_mmio_reg_offset(reg));
}
-static inline void intel_ring_advance(struct intel_ringbuffer *ring)
+static inline void intel_ring_advance(struct intel_ring *ring)
{
ring->tail &= ring->size - 1;
}
int __intel_ring_space(int head, int tail, int size);
-void intel_ring_update_space(struct intel_ringbuffer *ringbuf);
+void intel_ring_update_space(struct intel_ring *ringbuf);
int __must_check intel_engine_idle(struct intel_engine_cs *engine);
-void intel_ring_init_seqno(struct intel_engine_cs *engine, u32 seqno);
-int intel_ring_flush_all_caches(struct drm_i915_gem_request *req);
-int intel_ring_invalidate_all_caches(struct drm_i915_gem_request *req);
+void intel_engine_init_seqno(struct intel_engine_cs *engine, u32 seqno);
+int intel_engine_flush_all_caches(struct drm_i915_gem_request *req);
+int intel_engine_invalidate_all_caches(struct drm_i915_gem_request *req);
int intel_init_pipe_control(struct intel_engine_cs *engine, int size);
void intel_fini_pipe_control(struct intel_engine_cs *engine);
@@ -487,7 +486,7 @@ int intel_init_bsd2_ring_buffer(struct drm_device *dev);
int intel_init_blt_ring_buffer(struct drm_device *dev);
int intel_init_vebox_ring_buffer(struct drm_device *dev);
-u64 intel_ring_get_active_head(struct intel_engine_cs *engine);
+u64 intel_engine_get_active_head(struct intel_engine_cs *engine);
static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine)
{
return intel_read_status_page(engine, I915_GEM_HWS_INDEX);
@@ -495,7 +494,7 @@ static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine)
int init_workarounds_ring(struct intel_engine_cs *engine);
-static inline u32 intel_ring_get_tail(struct intel_ringbuffer *ringbuf)
+static inline u32 intel_ring_get_tail(struct intel_ring *ringbuf)
{
return ringbuf->tail;
}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 30/64] drm/i915: Rename residual ringbuf parameters
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (27 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 29/64] drm/i915: Rename struct intel_ringbuffer to struct intel_ring Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 31/64] drm/i915: Rename intel_pin_and_map_ring() Chris Wilson
` (34 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Now that we have a clear ring/engine split and a struct intel_ring, we
no longer need the stopgap ringbuf names.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_ringbuffer.c | 66 ++++++++++++++++-----------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 6 +--
2 files changed, 36 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 9ac56de18bd4..214d51f291e5 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -47,15 +47,15 @@ int __intel_ring_space(int head, int tail, int size)
return space - I915_RING_FREE_SPACE;
}
-void intel_ring_update_space(struct intel_ring *ringbuf)
+void intel_ring_update_space(struct intel_ring *ring)
{
- if (ringbuf->last_retired_head != -1) {
- ringbuf->head = ringbuf->last_retired_head;
- ringbuf->last_retired_head = -1;
+ if (ring->last_retired_head != -1) {
+ ring->head = ring->last_retired_head;
+ ring->last_retired_head = -1;
}
- ringbuf->space = __intel_ring_space(ringbuf->head & HEAD_ADDR,
- ringbuf->tail, ringbuf->size);
+ ring->space = __intel_ring_space(ring->head & HEAD_ADDR,
+ ring->tail, ring->size);
}
static void __intel_engine_submit(struct intel_engine_cs *engine)
@@ -1982,25 +1982,25 @@ static int init_phys_status_page(struct intel_engine_cs *engine)
return 0;
}
-void intel_unpin_ring(struct intel_ring *ringbuf)
+void intel_unpin_ring(struct intel_ring *ring)
{
- GEM_BUG_ON(!ringbuf->vma);
- GEM_BUG_ON(!ringbuf->vaddr);
+ GEM_BUG_ON(!ring->vma);
+ GEM_BUG_ON(!ring->vaddr);
- if (HAS_LLC(ringbuf->obj->base.dev) && !ringbuf->obj->stolen)
- i915_gem_object_unpin_map(ringbuf->obj);
+ if (HAS_LLC(ring->obj->base.dev) && !ring->obj->stolen)
+ i915_gem_object_unpin_map(ring->obj);
else
- i915_vma_unpin_iomap(ringbuf->vma);
- ringbuf->vaddr = NULL;
+ i915_vma_unpin_iomap(ring->vma);
+ ring->vaddr = NULL;
- i915_gem_object_ggtt_unpin(ringbuf->obj);
- ringbuf->vma = NULL;
+ i915_gem_object_ggtt_unpin(ring->obj);
+ ring->vma = NULL;
}
int intel_pin_and_map_ring(struct drm_i915_private *dev_priv,
- struct intel_ring *ringbuf)
+ struct intel_ring *ring)
{
- struct drm_i915_gem_object *obj = ringbuf->obj;
+ struct drm_i915_gem_object *obj = ring->obj;
/* Ring wraparound at offset 0 sometimes hangs. No idea why. */
unsigned flags = PIN_OFFSET_BIAS | 4096;
void *addr;
@@ -2040,8 +2040,8 @@ int intel_pin_and_map_ring(struct drm_i915_private *dev_priv,
}
}
- ringbuf->vaddr = addr;
- ringbuf->vma = i915_gem_obj_to_ggtt(obj);
+ ring->vaddr = addr;
+ ring->vma = i915_gem_obj_to_ggtt(obj);
return 0;
err_unpin:
@@ -2049,29 +2049,29 @@ err_unpin:
return ret;
}
-static void intel_destroy_ringbuffer_obj(struct intel_ring *ringbuf)
+static void intel_destroy_ringbuffer_obj(struct intel_ring *ring)
{
- i915_gem_object_put(ringbuf->obj);
- ringbuf->obj = NULL;
+ i915_gem_object_put(ring->obj);
+ ring->obj = NULL;
}
static int intel_alloc_ringbuffer_obj(struct drm_device *dev,
- struct intel_ring *ringbuf)
+ struct intel_ring *ring)
{
struct drm_i915_gem_object *obj;
obj = NULL;
if (!HAS_LLC(dev))
- obj = i915_gem_object_create_stolen(dev, ringbuf->size);
+ obj = i915_gem_object_create_stolen(dev, ring->size);
if (obj == NULL)
- obj = i915_gem_object_create(dev, ringbuf->size);
+ obj = i915_gem_object_create(dev, ring->size);
if (IS_ERR(obj))
return PTR_ERR(obj);
/* mark ring buffers as read-only from GPU side by default */
obj->gt_ro = 1;
- ringbuf->obj = obj;
+ ring->obj = obj;
return 0;
}
@@ -2179,7 +2179,7 @@ static int intel_init_engine(struct drm_device *dev,
struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_ring *ringbuf;
+ struct intel_ring *ring;
int ret;
WARN_ON(engine->buffer);
@@ -2209,12 +2209,12 @@ static int intel_init_engine(struct drm_device *dev,
if (ret)
goto error;
- ringbuf = intel_engine_create_ring(engine, 32 * PAGE_SIZE);
- if (IS_ERR(ringbuf)) {
- ret = PTR_ERR(ringbuf);
+ ring = intel_engine_create_ring(engine, 32 * PAGE_SIZE);
+ if (IS_ERR(ring)) {
+ ret = PTR_ERR(ring);
goto error;
}
- engine->buffer = ringbuf;
+ engine->buffer = ring;
if (I915_NEED_GFX_HWS(dev_priv)) {
ret = init_status_page(engine);
@@ -2227,11 +2227,11 @@ static int intel_init_engine(struct drm_device *dev,
goto error;
}
- ret = intel_pin_and_map_ring(dev_priv, ringbuf);
+ ret = intel_pin_and_map_ring(dev_priv, ring);
if (ret) {
DRM_ERROR("Failed to pin and map ringbuffer %s: %d\n",
engine->name, ret);
- intel_destroy_ringbuffer_obj(ringbuf);
+ intel_destroy_ringbuffer_obj(ring);
goto error;
}
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 590e9d006a95..14fc0a38ed58 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -470,7 +470,7 @@ static inline void intel_ring_advance(struct intel_ring *ring)
}
int __intel_ring_space(int head, int tail, int size);
-void intel_ring_update_space(struct intel_ring *ringbuf);
+void intel_ring_update_space(struct intel_ring *ring);
int __must_check intel_engine_idle(struct intel_engine_cs *engine);
void intel_engine_init_seqno(struct intel_engine_cs *engine, u32 seqno);
@@ -494,9 +494,9 @@ static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine)
int init_workarounds_ring(struct intel_engine_cs *engine);
-static inline u32 intel_ring_get_tail(struct intel_ring *ringbuf)
+static inline u32 intel_ring_get_tail(struct intel_ring *ring)
{
- return ringbuf->tail;
+ return ring->tail;
}
/*
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 31/64] drm/i915: Rename intel_pin_and_map_ring()
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (28 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 30/64] drm/i915: Rename residual ringbuf parameters Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 32/64] drm/i915: Remove obsolete engine->gpu_caches_dirty Chris Wilson
` (33 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
For more consistent oop-naming, we would use intel_ring_verb, so pick
intel_ring_pin() and intel_ring_unpin().
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_lrc.c | 4 ++--
drivers/gpu/drm/i915/intel_ringbuffer.c | 38 ++++++++++++++++-----------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 5 ++---
3 files changed, 23 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 0e8b2cd89aca..d5fd5fff7eca 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -976,7 +976,7 @@ static int intel_lr_context_pin(struct i915_gem_context *ctx,
lrc_reg_state = vaddr + LRC_STATE_PN * PAGE_SIZE;
- ret = intel_pin_and_map_ring(dev_priv, ce->ring);
+ ret = intel_ring_pin(ce->ring);
if (ret)
goto unpin_map;
@@ -1014,7 +1014,7 @@ void intel_lr_context_unpin(struct i915_gem_context *ctx,
if (--ce->pin_count)
return;
- intel_unpin_ring(ce->ring);
+ intel_ring_unpin(ce->ring);
i915_gem_object_unpin_map(ce->state);
i915_gem_object_ggtt_unpin(ce->state);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 214d51f291e5..8d931552e9e0 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1982,24 +1982,9 @@ static int init_phys_status_page(struct intel_engine_cs *engine)
return 0;
}
-void intel_unpin_ring(struct intel_ring *ring)
-{
- GEM_BUG_ON(!ring->vma);
- GEM_BUG_ON(!ring->vaddr);
-
- if (HAS_LLC(ring->obj->base.dev) && !ring->obj->stolen)
- i915_gem_object_unpin_map(ring->obj);
- else
- i915_vma_unpin_iomap(ring->vma);
- ring->vaddr = NULL;
-
- i915_gem_object_ggtt_unpin(ring->obj);
- ring->vma = NULL;
-}
-
-int intel_pin_and_map_ring(struct drm_i915_private *dev_priv,
- struct intel_ring *ring)
+int intel_ring_pin(struct intel_ring *ring)
{
+ struct drm_i915_private *dev_priv = ring->engine->i915;
struct drm_i915_gem_object *obj = ring->obj;
/* Ring wraparound at offset 0 sometimes hangs. No idea why. */
unsigned flags = PIN_OFFSET_BIAS | 4096;
@@ -2049,6 +2034,21 @@ err_unpin:
return ret;
}
+void intel_ring_unpin(struct intel_ring *ring)
+{
+ GEM_BUG_ON(!ring->vma);
+ GEM_BUG_ON(!ring->vaddr);
+
+ if (HAS_LLC(ring->engine->i915) && !ring->obj->stolen)
+ i915_gem_object_unpin_map(ring->obj);
+ else
+ i915_vma_unpin_iomap(ring->vma);
+ ring->vaddr = NULL;
+
+ i915_gem_object_ggtt_unpin(ring->obj);
+ ring->vma = NULL;
+}
+
static void intel_destroy_ringbuffer_obj(struct intel_ring *ring)
{
i915_gem_object_put(ring->obj);
@@ -2227,7 +2227,7 @@ static int intel_init_engine(struct drm_device *dev,
goto error;
}
- ret = intel_pin_and_map_ring(dev_priv, ring);
+ ret = intel_ring_pin(ring);
if (ret) {
DRM_ERROR("Failed to pin and map ringbuffer %s: %d\n",
engine->name, ret);
@@ -2259,7 +2259,7 @@ void intel_engine_cleanup(struct intel_engine_cs *engine)
intel_engine_stop(engine);
WARN_ON(!IS_GEN2(dev_priv) && (I915_READ_MODE(engine) & MODE_IDLE) == 0);
- intel_unpin_ring(engine->buffer);
+ intel_ring_unpin(engine->buffer);
intel_ring_free(engine->buffer);
engine->buffer = NULL;
}
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 14fc0a38ed58..dbc17c4ac98b 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -440,9 +440,8 @@ intel_write_status_page(struct intel_engine_cs *engine,
struct intel_ring *
intel_engine_create_ring(struct intel_engine_cs *engine, int size);
-int intel_pin_and_map_ring(struct drm_i915_private *dev_priv,
- struct intel_ring *ring);
-void intel_unpin_ring(struct intel_ring *ring);
+int intel_ring_pin(struct intel_ring *ring);
+void intel_ring_unpin(struct intel_ring *ring);
void intel_ring_free(struct intel_ring *ring);
void intel_engine_stop(struct intel_engine_cs *engine);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 32/64] drm/i915: Remove obsolete engine->gpu_caches_dirty
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (29 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 31/64] drm/i915: Rename intel_pin_and_map_ring() Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 33/64] drm/i915: Simplify request_alloc by returning the allocated request Chris Wilson
` (32 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Space for flushing the GPU cache prior to completing the request is
preallocated and so cannot fail.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_context.c | 2 +-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 9 +---
drivers/gpu/drm/i915/i915_gem_gtt.c | 18 ++++----
drivers/gpu/drm/i915/i915_gem_request.c | 7 ++-
drivers/gpu/drm/i915/intel_lrc.c | 47 +++----------------
drivers/gpu/drm/i915/intel_lrc.h | 2 -
drivers/gpu/drm/i915/intel_ringbuffer.c | 72 +++++++-----------------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 7 ---
8 files changed, 39 insertions(+), 125 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index d0ce01d3684c..32051a68246f 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -567,7 +567,7 @@ mi_set_context(struct drm_i915_gem_request *req, u32 hw_flags)
* itlb_before_ctx_switch.
*/
if (IS_GEN6(dev_priv)) {
- ret = req->engine->flush(req, I915_GEM_GPU_DOMAINS, 0);
+ ret = req->engine->emit_flush(req, I915_GEM_GPU_DOMAINS, 0);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index e05c205f2e19..3b5d03a063eb 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -972,10 +972,8 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
if (flush_domains & I915_GEM_DOMAIN_GTT)
wmb();
- /* Unconditionally invalidate gpu caches and ensure that we do flush
- * any residual writes from the previous batch.
- */
- return intel_engine_invalidate_all_caches(req);
+ /* Unconditionally invalidate gpu caches and TLBs. */
+ return req->engine->emit_flush(req, I915_GEM_GPU_DOMAINS, 0);
}
static bool
@@ -1156,9 +1154,6 @@ i915_gem_execbuffer_move_to_active(struct list_head *vmas,
static void
i915_gem_execbuffer_retire_commands(struct i915_execbuffer_params *params)
{
- /* Unconditionally force add_request to emit a full flush. */
- params->engine->gpu_caches_dirty = true;
-
/* Add a breadcrumb for the completion of the batch buffer */
__i915_add_request(params->request, params->batch_obj, true);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index c5357802171b..a3d9c723dd02 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -1664,9 +1664,9 @@ static int hsw_mm_switch(struct i915_hw_ppgtt *ppgtt,
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = req->engine->flush(req,
- I915_GEM_GPU_DOMAINS,
- I915_GEM_GPU_DOMAINS);
+ ret = req->engine->emit_flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -1692,9 +1692,9 @@ static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
int ret;
/* NB: TLBs must be flushed and invalidated before a switch */
- ret = req->engine->flush(req,
- I915_GEM_GPU_DOMAINS,
- I915_GEM_GPU_DOMAINS);
+ ret = req->engine->emit_flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -1712,9 +1712,9 @@ static int gen7_mm_switch(struct i915_hw_ppgtt *ppgtt,
/* XXX: RCS is the only one to auto invalidate the TLBs? */
if (req->engine->id != RCS) {
- ret = req->engine->flush(req,
- I915_GEM_GPU_DOMAINS,
- I915_GEM_GPU_DOMAINS);
+ ret = req->engine->emit_flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 4dc51e199269..f9fca8c71f24 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -452,10 +452,9 @@ void __i915_add_request(struct drm_i915_gem_request *request,
* what.
*/
if (flush_caches) {
- if (i915.enable_execlists)
- ret = logical_ring_flush_all_caches(request);
- else
- ret = intel_engine_flush_all_caches(request);
+ ret = request->engine->emit_flush(request,
+ 0, I915_GEM_GPU_DOMAINS);
+
/* Not allowed to fail! */
WARN(ret, "*_ring_flush_all_caches failed: %d!\n", ret);
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index d5fd5fff7eca..025d819614be 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -645,24 +645,6 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
spin_unlock_bh(&engine->execlist_lock);
}
-static int logical_ring_invalidate_all_caches(struct drm_i915_gem_request *req)
-{
- struct intel_engine_cs *engine = req->engine;
- uint32_t flush_domains;
- int ret;
-
- flush_domains = 0;
- if (engine->gpu_caches_dirty)
- flush_domains = I915_GEM_GPU_DOMAINS;
-
- ret = engine->emit_flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
- if (ret)
- return ret;
-
- engine->gpu_caches_dirty = false;
- return 0;
-}
-
static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
struct list_head *vmas)
{
@@ -693,7 +675,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
/* Unconditionally invalidate gpu caches and ensure that we do flush
* any residual writes from the previous batch.
*/
- return logical_ring_invalidate_all_caches(req);
+ return req->engine->emit_flush(req, I915_GEM_GPU_DOMAINS, 0);
}
int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
@@ -933,22 +915,6 @@ void intel_logical_ring_stop(struct intel_engine_cs *engine)
I915_WRITE_MODE(engine, _MASKED_BIT_DISABLE(STOP_RING));
}
-int logical_ring_flush_all_caches(struct drm_i915_gem_request *req)
-{
- struct intel_engine_cs *engine = req->engine;
- int ret;
-
- if (!engine->gpu_caches_dirty)
- return 0;
-
- ret = engine->emit_flush(req, 0, I915_GEM_GPU_DOMAINS);
- if (ret)
- return ret;
-
- engine->gpu_caches_dirty = false;
- return 0;
-}
-
static int intel_lr_context_pin(struct i915_gem_context *ctx,
struct intel_engine_cs *engine)
{
@@ -1029,15 +995,15 @@ void intel_lr_context_unpin(struct i915_gem_context *ctx,
static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
{
int ret, i;
- struct intel_engine_cs *engine = req->engine;
struct intel_ring *ring = req->ring;
struct i915_workarounds *w = &req->i915->workarounds;
if (w->count == 0)
return 0;
- engine->gpu_caches_dirty = true;
- ret = logical_ring_flush_all_caches(req);
+ ret = req->engine->emit_flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -1054,8 +1020,9 @@ static int intel_logical_ring_workarounds_emit(struct drm_i915_gem_request *req)
intel_ring_advance(ring);
- engine->gpu_caches_dirty = true;
- ret = logical_ring_flush_all_caches(req);
+ ret = req->engine->emit_flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index 63d6a49a0ded..5dae5bc5046e 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -69,8 +69,6 @@ void intel_logical_ring_stop(struct intel_engine_cs *engine);
void intel_logical_ring_cleanup(struct intel_engine_cs *engine);
int intel_logical_rings_init(struct drm_device *dev);
-int logical_ring_flush_all_caches(struct drm_i915_gem_request *req);
-
/* Logical Ring Contexts */
/* One extra page is added before LRC for GuC as shared data */
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 8d931552e9e0..5cf9d3e3d12a 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -693,8 +693,9 @@ static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
if (w->count == 0)
return 0;
- req->engine->gpu_caches_dirty = true;
- ret = intel_engine_flush_all_caches(req);
+ ret = req->engine->emit_flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -711,8 +712,9 @@ static int intel_ring_workarounds_emit(struct drm_i915_gem_request *req)
intel_ring_advance(ring);
- req->engine->gpu_caches_dirty = true;
- ret = intel_engine_flush_all_caches(req);
+ ret = req->engine->emit_flush(req,
+ I915_GEM_GPU_DOMAINS,
+ I915_GEM_GPU_DOMAINS);
if (ret)
return ret;
@@ -2864,21 +2866,21 @@ int intel_init_render_ring_buffer(struct drm_device *dev)
if (INTEL_GEN(dev_priv) >= 8) {
engine->init_context = intel_rcs_ctx_init;
engine->add_request = gen8_render_add_request;
- engine->flush = gen8_render_ring_flush;
+ engine->emit_flush = gen8_render_ring_flush;
if (i915.semaphores)
engine->semaphore.signal = gen8_rcs_signal;
} else if (INTEL_GEN(dev_priv) >= 6) {
engine->init_context = intel_rcs_ctx_init;
- engine->flush = gen7_render_ring_flush;
+ engine->emit_flush = gen7_render_ring_flush;
if (IS_GEN6(dev_priv))
- engine->flush = gen6_render_ring_flush;
+ engine->emit_flush = gen6_render_ring_flush;
} else if (IS_GEN5(dev_priv)) {
- engine->flush = gen4_render_ring_flush;
+ engine->emit_flush = gen4_render_ring_flush;
} else {
if (INTEL_GEN(dev_priv) < 4)
- engine->flush = gen2_render_ring_flush;
+ engine->emit_flush = gen2_render_ring_flush;
else
- engine->flush = gen4_render_ring_flush;
+ engine->emit_flush = gen4_render_ring_flush;
engine->irq_enable_mask = I915_USER_INTERRUPT;
}
@@ -2922,7 +2924,7 @@ int intel_init_bsd_ring_buffer(struct drm_device *dev)
/* gen6 bsd needs a special wa for tail updates */
if (IS_GEN6(dev_priv))
engine->write_tail = gen6_bsd_ring_write_tail;
- engine->flush = gen6_bsd_ring_flush;
+ engine->emit_flush = gen6_bsd_ring_flush;
if (INTEL_GEN(dev_priv) >= 8)
engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_VCS1_IRQ_SHIFT;
@@ -2930,7 +2932,7 @@ int intel_init_bsd_ring_buffer(struct drm_device *dev)
engine->irq_enable_mask = GT_BSD_USER_INTERRUPT;
} else {
engine->mmio_base = BSD_RING_BASE;
- engine->flush = bsd_ring_flush;
+ engine->emit_flush = bsd_ring_flush;
if (IS_GEN5(dev_priv))
engine->irq_enable_mask = ILK_BSD_USER_INTERRUPT;
else
@@ -2956,7 +2958,7 @@ int intel_init_bsd2_ring_buffer(struct drm_device *dev)
intel_ring_default_vfuncs(dev_priv, engine);
- engine->flush = gen6_bsd_ring_flush;
+ engine->emit_flush = gen6_bsd_ring_flush;
engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_VCS2_IRQ_SHIFT;
@@ -2976,7 +2978,7 @@ int intel_init_blt_ring_buffer(struct drm_device *dev)
intel_ring_default_vfuncs(dev_priv, engine);
- engine->flush = gen6_ring_flush;
+ engine->emit_flush = gen6_ring_flush;
if (INTEL_GEN(dev_priv) >= 8)
engine->irq_enable_mask =
GT_RENDER_USER_INTERRUPT << GEN8_BCS_IRQ_SHIFT;
@@ -2999,7 +3001,7 @@ int intel_init_vebox_ring_buffer(struct drm_device *dev)
intel_ring_default_vfuncs(dev_priv, engine);
- engine->flush = gen6_ring_flush;
+ engine->emit_flush = gen6_ring_flush;
if (INTEL_GEN(dev_priv) >= 8) {
engine->irq_enable_mask =
@@ -3013,46 +3015,6 @@ int intel_init_vebox_ring_buffer(struct drm_device *dev)
return intel_init_engine(dev, engine);
}
-int
-intel_engine_flush_all_caches(struct drm_i915_gem_request *req)
-{
- struct intel_engine_cs *engine = req->engine;
- int ret;
-
- if (!engine->gpu_caches_dirty)
- return 0;
-
- ret = engine->flush(req, 0, I915_GEM_GPU_DOMAINS);
- if (ret)
- return ret;
-
- trace_i915_gem_ring_flush(req, 0, I915_GEM_GPU_DOMAINS);
-
- engine->gpu_caches_dirty = false;
- return 0;
-}
-
-int
-intel_engine_invalidate_all_caches(struct drm_i915_gem_request *req)
-{
- struct intel_engine_cs *engine = req->engine;
- uint32_t flush_domains;
- int ret;
-
- flush_domains = 0;
- if (engine->gpu_caches_dirty)
- flush_domains = I915_GEM_GPU_DOMAINS;
-
- ret = engine->flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
- if (ret)
- return ret;
-
- trace_i915_gem_ring_flush(req, I915_GEM_GPU_DOMAINS, flush_domains);
-
- engine->gpu_caches_dirty = false;
- return 0;
-}
-
void intel_engine_stop(struct intel_engine_cs *engine)
{
int ret;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index dbc17c4ac98b..79faad9462d2 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -205,9 +205,6 @@ struct intel_engine_cs {
void (*write_tail)(struct intel_engine_cs *ring,
u32 value);
- int __must_check (*flush)(struct drm_i915_gem_request *req,
- u32 invalidate_domains,
- u32 flush_domains);
int (*add_request)(struct drm_i915_gem_request *req);
/* Some chipsets are not quite as coherent as advertised and need
* an expensive kick to force a true read of the up-to-date seqno.
@@ -324,8 +321,6 @@ struct intel_engine_cs {
*/
u32 last_submitted_seqno;
- bool gpu_caches_dirty;
-
struct i915_gem_context *last_context;
struct intel_engine_hangcheck hangcheck;
@@ -473,8 +468,6 @@ void intel_ring_update_space(struct intel_ring *ring);
int __must_check intel_engine_idle(struct intel_engine_cs *engine);
void intel_engine_init_seqno(struct intel_engine_cs *engine, u32 seqno);
-int intel_engine_flush_all_caches(struct drm_i915_gem_request *req);
-int intel_engine_invalidate_all_caches(struct drm_i915_gem_request *req);
int intel_init_pipe_control(struct intel_engine_cs *engine, int size);
void intel_fini_pipe_control(struct intel_engine_cs *engine);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 33/64] drm/i915: Simplify request_alloc by returning the allocated request
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (30 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 32/64] drm/i915: Remove obsolete engine->gpu_caches_dirty Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 34/64] drm/i915: Unify legacy/execlists emission of MI_BATCHBUFFER_START Chris Wilson
` (31 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
If is simpler and leads to more readable code through the callstack if
the allocation returns the allocated struct through the return value.
The importance of this is that it no longer looks like we accidentally
allocate requests as side-effect of calling certain functions.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 3 +-
drivers/gpu/drm/i915/i915_gem.c | 75 ++++++++----------------------
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 12 ++---
drivers/gpu/drm/i915/i915_gem_request.c | 58 ++++++++---------------
drivers/gpu/drm/i915/i915_trace.h | 13 +++---
drivers/gpu/drm/i915/intel_display.c | 36 ++++++--------
drivers/gpu/drm/i915/intel_lrc.c | 2 +-
drivers/gpu/drm/i915/intel_overlay.c | 20 ++++----
8 files changed, 79 insertions(+), 140 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index f59dde1b4156..4a5ce22dcaf0 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3151,8 +3151,7 @@ static inline void i915_gem_object_unpin_map(struct drm_i915_gem_object *obj)
int __must_check i915_mutex_lock_interruptible(struct drm_device *dev);
int i915_gem_object_sync(struct drm_i915_gem_object *obj,
- struct intel_engine_cs *to,
- struct drm_i915_gem_request **to_req);
+ struct drm_i915_gem_request *to);
void i915_vma_move_to_active(struct i915_vma *vma,
struct drm_i915_gem_request *req);
int i915_gem_dumb_create(struct drm_file *file_priv,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 5caf4dc14a46..7c21a4f4028d 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2837,51 +2837,35 @@ out:
static int
__i915_gem_object_sync(struct drm_i915_gem_object *obj,
- struct intel_engine_cs *to,
- struct drm_i915_gem_request *from_req,
- struct drm_i915_gem_request **to_req)
+ struct drm_i915_gem_request *to,
+ struct drm_i915_gem_request *from)
{
- struct intel_engine_cs *from;
int ret;
- from = i915_gem_request_get_engine(from_req);
- if (to == from)
+ if (to->engine == from->engine)
return 0;
- if (i915_gem_request_completed(from_req))
+ if (i915_gem_request_completed(from))
return 0;
if (!i915.semaphores) {
- struct drm_i915_private *i915 = to_i915(obj->base.dev);
- ret = __i915_wait_request(from_req,
- i915->mm.interruptible,
+ ret = __i915_wait_request(from,
+ from->i915->mm.interruptible,
NULL,
NO_WAITBOOST);
if (ret)
return ret;
- i915_gem_object_retire_request(obj, from_req);
+ i915_gem_object_retire_request(obj, from);
} else {
- int idx = intel_engine_sync_index(from, to);
- u32 seqno = i915_gem_request_get_seqno(from_req);
+ int idx = intel_engine_sync_index(from->engine, to->engine);
+ u32 seqno = i915_gem_request_get_seqno(from);
- WARN_ON(!to_req);
-
- if (seqno <= from->semaphore.sync_seqno[idx])
+ if (seqno <= from->engine->semaphore.sync_seqno[idx])
return 0;
- if (*to_req == NULL) {
- struct drm_i915_gem_request *req;
-
- req = i915_gem_request_alloc(to, NULL);
- if (IS_ERR(req))
- return PTR_ERR(req);
-
- *to_req = req;
- }
-
- trace_i915_gem_ring_sync_to(*to_req, from, from_req);
- ret = to->semaphore.sync_to(*to_req, from, seqno);
+ trace_i915_gem_ring_sync_to(to, from);
+ ret = to->engine->semaphore.sync_to(to, from->engine, seqno);
if (ret)
return ret;
@@ -2889,8 +2873,8 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
* might have just caused seqno wrap under
* the radar.
*/
- from->semaphore.sync_seqno[idx] =
- i915_gem_request_get_seqno(obj->last_read_req[from->id]);
+ from->engine->semaphore.sync_seqno[idx] =
+ i915_gem_request_get_seqno(obj->last_read_req[from->engine->id]);
}
return 0;
@@ -2900,17 +2884,12 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
* i915_gem_object_sync - sync an object to a ring.
*
* @obj: object which may be in use on another ring.
- * @to: ring we wish to use the object on. May be NULL.
- * @to_req: request we wish to use the object for. See below.
- * This will be allocated and returned if a request is
- * required but not passed in.
+ * @to: request we are wishing to use
*
* This code is meant to abstract object synchronization with the GPU.
- * Calling with NULL implies synchronizing the object with the CPU
- * rather than a particular GPU ring. Conceptually we serialise writes
- * between engines inside the GPU. We only allow one engine to write
- * into a buffer at any time, but multiple readers. To ensure each has
- * a coherent view of memory, we must:
+ * Conceptually we serialise writes between engines inside the GPU.
+ * We only allow one engine to write into a buffer at any time, but
+ * multiple readers. To ensure each has a coherent view of memory, we must:
*
* - If there is an outstanding write request to the object, the new
* request must wait for it to complete (either CPU or in hw, requests
@@ -2919,22 +2898,11 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
* - If we are a write request (pending_write_domain is set), the new
* request must wait for outstanding read requests to complete.
*
- * For CPU synchronisation (NULL to) no request is required. For syncing with
- * rings to_req must be non-NULL. However, a request does not have to be
- * pre-allocated. If *to_req is NULL and sync commands will be emitted then a
- * request will be allocated automatically and returned through *to_req. Note
- * that it is not guaranteed that commands will be emitted (because the system
- * might already be idle). Hence there is no need to create a request that
- * might never have any work submitted. Note further that if a request is
- * returned in *to_req, it is the responsibility of the caller to submit
- * that request (after potentially adding more work to it).
- *
* Returns 0 if successful, else propagates up the lower layer error.
*/
int
i915_gem_object_sync(struct drm_i915_gem_object *obj,
- struct intel_engine_cs *to,
- struct drm_i915_gem_request **to_req)
+ struct drm_i915_gem_request *to)
{
const bool readonly = obj->base.pending_write_domain == 0;
struct drm_i915_gem_request *req[I915_NUM_ENGINES];
@@ -2943,9 +2911,6 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (!obj->active)
return 0;
- if (to == NULL)
- return i915_gem_object_wait_rendering(obj, readonly);
-
n = 0;
if (readonly) {
if (obj->last_write_req)
@@ -2956,7 +2921,7 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
req[n++] = obj->last_read_req[i];
}
for (i = 0; i < n; i++) {
- ret = __i915_gem_object_sync(obj, to, req[i], to_req);
+ ret = __i915_gem_object_sync(obj, to, req[i]);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 3b5d03a063eb..5a8d58a177eb 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -955,7 +955,7 @@ i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->engine, &req);
+ ret = i915_gem_object_sync(obj, req);
if (ret)
return ret;
}
@@ -1439,7 +1439,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
{
struct drm_i915_private *dev_priv = to_i915(dev);
struct i915_ggtt *ggtt = &dev_priv->ggtt;
- struct drm_i915_gem_request *req = NULL;
struct eb_vmas *eb;
struct drm_i915_gem_object *batch_obj;
struct drm_i915_gem_exec_object2 shadow_exec_entry;
@@ -1627,13 +1626,13 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->batch_obj_vm_offset = i915_gem_obj_offset(batch_obj, vm);
/* Allocate a request for this batch buffer nice and early. */
- req = i915_gem_request_alloc(engine, ctx);
- if (IS_ERR(req)) {
- ret = PTR_ERR(req);
+ params->request = i915_gem_request_alloc(engine, ctx);
+ if (IS_ERR(params->request)) {
+ ret = PTR_ERR(params->request);
goto err_batch_unpin;
}
- ret = i915_gem_request_add_to_client(req, file);
+ ret = i915_gem_request_add_to_client(params->request, file);
if (ret)
goto err_request;
@@ -1649,7 +1648,6 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->dispatch_flags = dispatch_flags;
params->batch_obj = batch_obj;
params->ctx = ctx;
- params->request = req;
ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
err_request:
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index f9fca8c71f24..3c28bd11fa9d 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -292,10 +292,21 @@ static int i915_gem_get_seqno(struct drm_i915_private *dev_priv, u32 *seqno)
return 0;
}
-static inline int
-__i915_gem_request_alloc(struct intel_engine_cs *engine,
- struct i915_gem_context *ctx,
- struct drm_i915_gem_request **req_out)
+/**
+ * i915_gem_request_alloc - allocate a request structure
+ *
+ * @engine: engine that we wish to issue the request on.
+ * @ctx: context that the request will be associated with.
+ * This can be NULL if the request is not directly related to
+ * any specific user context, in which case this function will
+ * choose an appropriate context to use.
+ *
+ * Returns a pointer to the allocated request if successful,
+ * or an error code if not.
+ */
+struct drm_i915_gem_request *
+i915_gem_request_alloc(struct intel_engine_cs *engine,
+ struct i915_gem_context *ctx)
{
struct drm_i915_private *dev_priv = engine->i915;
unsigned int reset_counter = i915_reset_counter(&dev_priv->gpu_error);
@@ -303,18 +314,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
u32 seqno;
int ret;
- if (!req_out)
- return -EINVAL;
-
- *req_out = NULL;
-
/* ABI: Before userspace accesses the GPU (e.g. execbuffer), report
* EIO if the GPU is already wedged, or EAGAIN to drop the struct_mutex
* and restart.
*/
ret = i915_gem_check_wedge(reset_counter, dev_priv->mm.interruptible);
if (ret)
- return ret;
+ return ERR_PTR(ret);
if (!list_empty(&engine->request_list)) {
req = list_first_entry(&engine->request_list,
@@ -325,7 +331,7 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
if (!req)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
ret = i915_gem_get_seqno(dev_priv, &seqno);
if (ret)
@@ -358,39 +364,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
if (ret)
goto err_ctx;
- *req_out = req;
- return 0;
+ return req;
err_ctx:
i915_gem_context_put(ctx);
err:
kmem_cache_free(dev_priv->requests, req);
- return ret;
-}
-
-/**
- * i915_gem_request_alloc - allocate a request structure
- *
- * @engine: engine that we wish to issue the request on.
- * @ctx: context that the request will be associated with.
- * This can be NULL if the request is not directly related to
- * any specific user context, in which case this function will
- * choose an appropriate context to use.
- *
- * Returns a pointer to the allocated request if successful,
- * or an error code if not.
- */
-struct drm_i915_gem_request *
-i915_gem_request_alloc(struct intel_engine_cs *engine,
- struct i915_gem_context *ctx)
-{
- struct drm_i915_gem_request *req;
- int err;
-
- if (!ctx)
- ctx = engine->i915->kernel_context;
- err = __i915_gem_request_alloc(engine, ctx, &req);
- return err ? ERR_PTR(err) : req;
+ return ERR_PTR(ret);
}
static void i915_gem_mark_busy(const struct intel_engine_cs *engine)
diff --git a/drivers/gpu/drm/i915/i915_trace.h b/drivers/gpu/drm/i915/i915_trace.h
index 007112d1e049..9e43c0aa6e3b 100644
--- a/drivers/gpu/drm/i915/i915_trace.h
+++ b/drivers/gpu/drm/i915/i915_trace.h
@@ -449,10 +449,9 @@ TRACE_EVENT(i915_gem_evict_vm,
);
TRACE_EVENT(i915_gem_ring_sync_to,
- TP_PROTO(struct drm_i915_gem_request *to_req,
- struct intel_engine_cs *from,
- struct drm_i915_gem_request *req),
- TP_ARGS(to_req, from, req),
+ TP_PROTO(struct drm_i915_gem_request *to,
+ struct drm_i915_gem_request *from),
+ TP_ARGS(to, from),
TP_STRUCT__entry(
__field(u32, dev)
@@ -463,9 +462,9 @@ TRACE_EVENT(i915_gem_ring_sync_to,
TP_fast_assign(
__entry->dev = from->i915->drm.primary->index;
- __entry->sync_from = from->id;
- __entry->sync_to = to_req->engine->id;
- __entry->seqno = req->fence.seqno;
+ __entry->sync_from = from->engine->id;
+ __entry->sync_to = to->engine->id;
+ __entry->seqno = from->fence.seqno;
),
TP_printk("dev=%u, sync-from=%u, sync-to=%u, seqno=%u",
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index d90a3746bce8..5f4bdc3d4972 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11670,7 +11670,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
struct intel_flip_work *work;
struct intel_engine_cs *engine;
bool mmio_flip;
- struct drm_i915_gem_request *request = NULL;
+ struct drm_i915_gem_request *request;
int ret;
/*
@@ -11777,22 +11777,6 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
mmio_flip = use_mmio_flip(engine, obj);
- /* When using CS flips, we want to emit semaphores between rings.
- * However, when using mmio flips we will create a task to do the
- * synchronisation, so all we want here is to pin the framebuffer
- * into the display plane and skip any waits.
- */
- if (!mmio_flip) {
- ret = i915_gem_object_sync(obj, engine, &request);
- if (!ret && !request) {
- request = i915_gem_request_alloc(engine, NULL);
- ret = PTR_ERR_OR_ZERO(request);
- }
-
- if (ret)
- goto cleanup_pending;
- }
-
ret = intel_pin_and_fence_fb_obj(fb, primary->state->rotation);
if (ret)
goto cleanup_pending;
@@ -11810,14 +11794,24 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
schedule_work(&work->mmio_work);
} else {
- i915_gem_request_assign(&work->flip_queued_req, request);
+ request = i915_gem_request_alloc(engine, engine->last_context);
+ if (IS_ERR(request)) {
+ ret = PTR_ERR(request);
+ goto cleanup_unpin;
+ }
+
+ ret = i915_gem_object_sync(obj, request);
+ if (ret)
+ goto cleanup_request;
+
ret = dev_priv->display.queue_flip(dev, crtc, fb, obj, request,
page_flip_flags);
if (ret)
- goto cleanup_unpin;
+ goto cleanup_request;
intel_mark_page_flip_active(intel_crtc, work);
+ work->flip_queued_req = i915_gem_request_get(request);
i915_add_request_no_flush(request);
}
@@ -11832,11 +11826,11 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
return 0;
+cleanup_request:
+ i915_add_request_no_flush(request);
cleanup_unpin:
intel_unpin_fb_obj(fb, crtc->primary->state->rotation);
cleanup_pending:
- if (!IS_ERR_OR_NULL(request))
- i915_add_request_no_flush(request);
atomic_dec(&intel_crtc->unpin_work_count);
mutex_unlock(&dev->struct_mutex);
cleanup:
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 025d819614be..a11fd33ddbc8 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -658,7 +658,7 @@ static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
struct drm_i915_gem_object *obj = vma->obj;
if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req->engine, &req);
+ ret = i915_gem_object_sync(obj, req);
if (ret)
return ret;
}
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index a5071e281088..356a1f6f95aa 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -229,11 +229,18 @@ static int intel_overlay_do_wait_request(struct intel_overlay *overlay,
return 0;
}
+static struct drm_i915_gem_request *alloc_request(struct intel_overlay *overlay)
+{
+ struct drm_i915_private *dev_priv = overlay->i915;
+ struct intel_engine_cs *engine = &dev_priv->engine[RCS];
+
+ return i915_gem_request_alloc(engine, dev_priv->kernel_context);
+}
+
/* overlay needs to be disable in OCMD reg */
static int intel_overlay_on(struct intel_overlay *overlay)
{
struct drm_i915_private *dev_priv = overlay->i915;
- struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
struct intel_ring *ring;
int ret;
@@ -241,7 +248,7 @@ static int intel_overlay_on(struct intel_overlay *overlay)
WARN_ON(overlay->active);
WARN_ON(IS_I830(dev_priv) && !(dev_priv->quirks & QUIRK_PIPEA_FORCE));
- req = i915_gem_request_alloc(engine, NULL);
+ req = alloc_request(overlay);
if (IS_ERR(req))
return PTR_ERR(req);
@@ -268,7 +275,6 @@ static int intel_overlay_continue(struct intel_overlay *overlay,
bool load_polyphase_filter)
{
struct drm_i915_private *dev_priv = overlay->i915;
- struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
struct intel_ring *ring;
u32 flip_addr = overlay->flip_addr;
@@ -285,7 +291,7 @@ static int intel_overlay_continue(struct intel_overlay *overlay,
if (tmp & (1 << 17))
DRM_DEBUG("overlay underrun, DOVSTA: %x\n", tmp);
- req = i915_gem_request_alloc(engine, NULL);
+ req = alloc_request(overlay);
if (IS_ERR(req))
return PTR_ERR(req);
@@ -338,7 +344,6 @@ static void intel_overlay_off_tail(struct intel_overlay *overlay)
static int intel_overlay_off(struct intel_overlay *overlay)
{
struct drm_i915_private *dev_priv = overlay->i915;
- struct intel_engine_cs *engine = &dev_priv->engine[RCS];
struct drm_i915_gem_request *req;
struct intel_ring *ring;
u32 flip_addr = overlay->flip_addr;
@@ -352,7 +357,7 @@ static int intel_overlay_off(struct intel_overlay *overlay)
* of the hw. Do it in both cases */
flip_addr |= OFC_UPDATE;
- req = i915_gem_request_alloc(engine, NULL);
+ req = alloc_request(overlay);
if (IS_ERR(req))
return PTR_ERR(req);
@@ -412,7 +417,6 @@ static int intel_overlay_recover_from_interrupt(struct intel_overlay *overlay)
static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
{
struct drm_i915_private *dev_priv = overlay->i915;
- struct intel_engine_cs *engine = &dev_priv->engine[RCS];
int ret;
lockdep_assert_held(&dev_priv->drm.struct_mutex);
@@ -428,7 +432,7 @@ static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
struct drm_i915_gem_request *req;
struct intel_ring *ring;
- req = i915_gem_request_alloc(engine, NULL);
+ req = alloc_request(overlay);
if (IS_ERR(req))
return PTR_ERR(req);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 34/64] drm/i915: Unify legacy/execlists emission of MI_BATCHBUFFER_START
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (31 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 33/64] drm/i915: Simplify request_alloc by returning the allocated request Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 35/64] drm/i915: Convert engine->write_tail to operate on a request Chris Wilson
` (30 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Both the ->dispatch_execbuffer and ->emit_bb_start callbacks do exactly
the same thing, add MI_BATCHBUFFER_START to the request's ringbuffer -
we need only one vfunc.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 6 ++--
drivers/gpu/drm/i915/i915_gem_render_state.c | 16 +++++-----
drivers/gpu/drm/i915/intel_lrc.c | 15 ++++++---
drivers/gpu/drm/i915/intel_ringbuffer.c | 48 ++++++++++++++--------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 12 +++----
5 files changed, 50 insertions(+), 47 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 5a8d58a177eb..287e4e7b3e51 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1319,9 +1319,9 @@ i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
if (exec_len == 0)
exec_len = params->batch_obj->base.size;
- ret = params->engine->dispatch_execbuffer(params->request,
- exec_start, exec_len,
- params->dispatch_flags);
+ ret = params->engine->emit_bb_start(params->request,
+ exec_start, exec_len,
+ params->dispatch_flags);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_gem_render_state.c b/drivers/gpu/drm/i915/i915_gem_render_state.c
index b2be4676a5cf..2ba759f3ab6f 100644
--- a/drivers/gpu/drm/i915/i915_gem_render_state.c
+++ b/drivers/gpu/drm/i915/i915_gem_render_state.c
@@ -234,18 +234,18 @@ int i915_gem_render_state_init(struct drm_i915_gem_request *req)
if (so.rodata == NULL)
return 0;
- ret = req->engine->dispatch_execbuffer(req, so.ggtt_offset,
- so.rodata->batch_items * 4,
- I915_DISPATCH_SECURE);
+ ret = req->engine->emit_bb_start(req, so.ggtt_offset,
+ so.rodata->batch_items * 4,
+ I915_DISPATCH_SECURE);
if (ret)
goto out;
if (so.aux_batch_size > 8) {
- ret = req->engine->dispatch_execbuffer(req,
- (so.ggtt_offset +
- so.aux_batch_offset),
- so.aux_batch_size,
- I915_DISPATCH_SECURE);
+ ret = req->engine->emit_bb_start(req,
+ (so.ggtt_offset +
+ so.aux_batch_offset),
+ so.aux_batch_size,
+ I915_DISPATCH_SECURE);
if (ret)
goto out;
}
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index a11fd33ddbc8..3b98148d1cf8 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -862,7 +862,9 @@ int intel_execlists_submission(struct i915_execbuffer_params *params,
exec_start = params->batch_obj_vm_offset +
args->batch_start_offset;
- ret = engine->emit_bb_start(params->request, exec_start, params->dispatch_flags);
+ ret = engine->emit_bb_start(params->request,
+ exec_start, args->batch_len,
+ params->dispatch_flags);
if (ret)
return ret;
@@ -1534,7 +1536,8 @@ static int intel_logical_ring_emit_pdps(struct drm_i915_gem_request *req)
}
static int gen8_emit_bb_start(struct drm_i915_gem_request *req,
- u64 offset, unsigned dispatch_flags)
+ u64 offset, u32 len,
+ unsigned int dispatch_flags)
{
struct intel_ring *ring = req->ring;
bool ppgtt = !(dispatch_flags & I915_DISPATCH_SECURE);
@@ -1810,13 +1813,15 @@ static int intel_lr_context_render_state_init(struct drm_i915_gem_request *req)
return 0;
ret = req->engine->emit_bb_start(req, so.ggtt_offset,
- I915_DISPATCH_SECURE);
+ so.rodata->batch_items * 4,
+ I915_DISPATCH_SECURE);
if (ret)
goto out;
ret = req->engine->emit_bb_start(req,
- (so.ggtt_offset + so.aux_batch_offset),
- I915_DISPATCH_SECURE);
+ (so.ggtt_offset + so.aux_batch_offset),
+ so.aux_batch_size,
+ I915_DISPATCH_SECURE);
if (ret)
goto out;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 5cf9d3e3d12a..9af35f255dec 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1783,9 +1783,9 @@ gen8_irq_disable(struct intel_engine_cs *engine)
}
static int
-i965_dispatch_execbuffer(struct drm_i915_gem_request *req,
- u64 offset, u32 length,
- unsigned dispatch_flags)
+i965_emit_bb_start(struct drm_i915_gem_request *req,
+ u64 offset, u32 length,
+ unsigned int dispatch_flags)
{
struct intel_ring *ring = req->ring;
int ret;
@@ -1810,9 +1810,9 @@ i965_dispatch_execbuffer(struct drm_i915_gem_request *req,
#define I830_TLB_ENTRIES (2)
#define I830_WA_SIZE max(I830_TLB_ENTRIES*4096, I830_BATCH_LIMIT)
static int
-i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
- u64 offset, u32 len,
- unsigned dispatch_flags)
+i830_emit_bb_start(struct drm_i915_gem_request *req,
+ u64 offset, u32 len,
+ unsigned int dispatch_flags)
{
struct intel_ring *ring = req->ring;
u32 cs_offset = req->engine->scratch.gtt_offset;
@@ -1872,9 +1872,9 @@ i830_dispatch_execbuffer(struct drm_i915_gem_request *req,
}
static int
-i915_dispatch_execbuffer(struct drm_i915_gem_request *req,
- u64 offset, u32 len,
- unsigned dispatch_flags)
+i915_emit_bb_start(struct drm_i915_gem_request *req,
+ u64 offset, u32 len,
+ unsigned int dispatch_flags)
{
struct intel_ring *ring = req->ring;
int ret;
@@ -2576,9 +2576,9 @@ static int gen6_bsd_ring_flush(struct drm_i915_gem_request *req,
}
static int
-gen8_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
- u64 offset, u32 len,
- unsigned dispatch_flags)
+gen8_emit_bb_start(struct drm_i915_gem_request *req,
+ u64 offset, u32 len,
+ unsigned int dispatch_flags)
{
struct intel_ring *ring = req->ring;
bool ppgtt = USES_PPGTT(req->i915) &&
@@ -2602,9 +2602,9 @@ gen8_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
}
static int
-hsw_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
- u64 offset, u32 len,
- unsigned dispatch_flags)
+hsw_emit_bb_start(struct drm_i915_gem_request *req,
+ u64 offset, u32 len,
+ unsigned int dispatch_flags)
{
struct intel_ring *ring = req->ring;
int ret;
@@ -2627,9 +2627,9 @@ hsw_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
}
static int
-gen6_ring_dispatch_execbuffer(struct drm_i915_gem_request *req,
- u64 offset, u32 len,
- unsigned dispatch_flags)
+gen6_emit_bb_start(struct drm_i915_gem_request *req,
+ u64 offset, u32 len,
+ unsigned int dispatch_flags)
{
struct intel_ring *ring = req->ring;
int ret;
@@ -2831,15 +2831,15 @@ static void intel_ring_default_vfuncs(struct drm_i915_private *dev_priv,
engine->add_request = gen6_add_request;
if (INTEL_GEN(dev_priv) >= 8)
- engine->dispatch_execbuffer = gen8_ring_dispatch_execbuffer;
+ engine->emit_bb_start = gen8_emit_bb_start;
else if (INTEL_GEN(dev_priv) >= 6)
- engine->dispatch_execbuffer = gen6_ring_dispatch_execbuffer;
+ engine->emit_bb_start = gen6_emit_bb_start;
else if (INTEL_GEN(dev_priv) >= 4)
- engine->dispatch_execbuffer = i965_dispatch_execbuffer;
+ engine->emit_bb_start = i965_emit_bb_start;
else if (IS_I830(dev_priv) || IS_845G(dev_priv))
- engine->dispatch_execbuffer = i830_dispatch_execbuffer;
+ engine->emit_bb_start = i830_emit_bb_start;
else
- engine->dispatch_execbuffer = i915_dispatch_execbuffer;
+ engine->emit_bb_start = i915_emit_bb_start;
intel_ring_init_irq(dev_priv, engine);
intel_ring_init_semaphores(dev_priv, engine);
@@ -2885,7 +2885,7 @@ int intel_init_render_ring_buffer(struct drm_device *dev)
}
if (IS_HASWELL(dev_priv))
- engine->dispatch_execbuffer = hsw_ring_dispatch_execbuffer;
+ engine->emit_bb_start = hsw_emit_bb_start;
engine->init_hw = init_render_ring;
engine->cleanup = render_ring_cleanup;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 79faad9462d2..f6cecd47a3ef 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -213,12 +213,6 @@ struct intel_engine_cs {
* monotonic, even if not coherent.
*/
void (*irq_seqno_barrier)(struct intel_engine_cs *ring);
- int (*dispatch_execbuffer)(struct drm_i915_gem_request *req,
- u64 offset, u32 length,
- unsigned dispatch_flags);
-#define I915_DISPATCH_SECURE 0x1
-#define I915_DISPATCH_PINNED 0x2
-#define I915_DISPATCH_RS 0x4
void (*cleanup)(struct intel_engine_cs *ring);
/* GEN8 signal/wait table - never trust comments!
@@ -294,7 +288,11 @@ struct intel_engine_cs {
u32 invalidate_domains,
u32 flush_domains);
int (*emit_bb_start)(struct drm_i915_gem_request *req,
- u64 offset, unsigned dispatch_flags);
+ u64 offset, u32 length,
+ unsigned int dispatch_flags);
+#define I915_DISPATCH_SECURE 0x1
+#define I915_DISPATCH_PINNED 0x2
+#define I915_DISPATCH_RS 0x4
/**
* List of objects currently involved in rendering from the
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 35/64] drm/i915: Convert engine->write_tail to operate on a request
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (32 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 34/64] drm/i915: Unify legacy/execlists emission of MI_BATCHBUFFER_START Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 36/64] drm/i915: Unify request submission Chris Wilson
` (29 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
If we rewrite the I915_WRITE_TAIL specialisation for the legacy
ringbuffer as submitting the request onto the ringbuffer, we can unify
the vfunc with both execlists and GuC in the next patch.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_request.c | 7 ++---
drivers/gpu/drm/i915/intel_ringbuffer.c | 52 ++++++++++++++++-----------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 3 +-
3 files changed, 29 insertions(+), 33 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 3c28bd11fa9d..18b9278d16f3 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -468,13 +468,10 @@ void __i915_add_request(struct drm_i915_gem_request *request,
*/
request->postfix = intel_ring_get_tail(ring);
- if (i915.enable_execlists) {
+ if (i915.enable_execlists)
ret = engine->emit_request(request);
- } else {
+ else
ret = engine->add_request(request);
-
- request->tail = intel_ring_get_tail(ring);
- }
/* Not allowed to fail! */
WARN(ret, "emit|add_request failed: %d!\n", ret);
/* Sanity check that the reserved size was large enough. */
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 9af35f255dec..63166a7cf70d 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -58,14 +58,6 @@ void intel_ring_update_space(struct intel_ring *ring)
ring->tail, ring->size);
}
-static void __intel_engine_submit(struct intel_engine_cs *engine)
-{
- struct intel_ring *ring = engine->buffer;
-
- ring->tail &= ring->size - 1;
- engine->write_tail(engine, ring->tail);
-}
-
static int
gen2_render_ring_flush(struct drm_i915_gem_request *req,
u32 invalidate_domains,
@@ -421,13 +413,6 @@ gen8_render_ring_flush(struct drm_i915_gem_request *req,
return gen8_emit_pipe_control(req, flags, scratch_addr);
}
-static void ring_write_tail(struct intel_engine_cs *engine,
- u32 value)
-{
- struct drm_i915_private *dev_priv = engine->i915;
- I915_WRITE_TAIL(engine, value);
-}
-
u64 intel_engine_get_active_head(struct intel_engine_cs *engine)
{
struct drm_i915_private *dev_priv = engine->i915;
@@ -541,7 +526,7 @@ static bool stop_ring(struct intel_engine_cs *engine)
I915_WRITE_CTL(engine, 0);
I915_WRITE_HEAD(engine, 0);
- engine->write_tail(engine, 0);
+ I915_WRITE_TAIL(engine, 0);
if (!IS_GEN2(dev_priv)) {
(void)I915_READ_CTL(engine);
@@ -1471,7 +1456,11 @@ gen6_add_request(struct drm_i915_gem_request *req)
intel_ring_emit(ring, I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
intel_ring_emit(ring, req->fence.seqno);
intel_ring_emit(ring, MI_USER_INTERRUPT);
- __intel_engine_submit(req->engine);
+ intel_ring_advance(ring);
+
+ req->tail = intel_ring_get_tail(ring);
+
+ req->engine->submit_request(req);
return 0;
}
@@ -1501,7 +1490,8 @@ gen8_render_add_request(struct drm_i915_gem_request *req)
intel_ring_emit(ring, 0);
intel_ring_emit(ring, MI_USER_INTERRUPT);
intel_ring_emit(ring, MI_NOOP);
- __intel_engine_submit(engine);
+
+ req->engine->submit_request(req);
return 0;
}
@@ -1720,11 +1710,22 @@ i9xx_add_request(struct drm_i915_gem_request *req)
intel_ring_emit(ring, I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
intel_ring_emit(ring, req->fence.seqno);
intel_ring_emit(ring, MI_USER_INTERRUPT);
- __intel_engine_submit(req->engine);
+ intel_ring_advance(ring);
+
+ req->tail = intel_ring_get_tail(ring);
+
+ req->engine->submit_request(req);
return 0;
}
+static void i9xx_submit_request(struct drm_i915_gem_request *request)
+{
+ struct drm_i915_private *dev_priv = request->i915;
+
+ I915_WRITE_TAIL(request->engine, request->tail);
+}
+
static void
gen6_irq_enable(struct intel_engine_cs *engine)
{
@@ -2492,10 +2493,9 @@ void intel_engine_init_seqno(struct intel_engine_cs *engine, u32 seqno)
rcu_read_unlock();
}
-static void gen6_bsd_ring_write_tail(struct intel_engine_cs *engine,
- u32 value)
+static void gen6_bsd_submit_request(struct drm_i915_gem_request *request)
{
- struct drm_i915_private *dev_priv = engine->i915;
+ struct drm_i915_private *dev_priv = request->i915;
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
@@ -2519,8 +2519,8 @@ static void gen6_bsd_ring_write_tail(struct intel_engine_cs *engine,
DRM_ERROR("timed out waiting for the BSD ring to wake up\n");
/* Now that the ring is fully powered up, update the tail */
- I915_WRITE_FW(RING_TAIL(engine->mmio_base), value);
- POSTING_READ_FW(RING_TAIL(engine->mmio_base));
+ I915_WRITE_FW(RING_TAIL(request->engine->mmio_base), request->tail);
+ POSTING_READ_FW(RING_TAIL(request->engine->mmio_base));
/* Let the ring send IDLE messages to the GT again,
* and so let it sleep to conserve power when idle.
@@ -2824,7 +2824,7 @@ static void intel_ring_default_vfuncs(struct drm_i915_private *dev_priv,
struct intel_engine_cs *engine)
{
engine->init_hw = init_ring_common;
- engine->write_tail = ring_write_tail;
+ engine->submit_request = i9xx_submit_request;
engine->add_request = i9xx_add_request;
if (INTEL_GEN(dev_priv) >= 6)
@@ -2923,7 +2923,7 @@ int intel_init_bsd_ring_buffer(struct drm_device *dev)
engine->mmio_base = GEN6_BSD_RING_BASE;
/* gen6 bsd needs a special wa for tail updates */
if (IS_GEN6(dev_priv))
- engine->write_tail = gen6_bsd_ring_write_tail;
+ engine->submit_request = gen6_bsd_submit_request;
engine->emit_flush = gen6_bsd_ring_flush;
if (INTEL_GEN(dev_priv) >= 8)
engine->irq_enable_mask =
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index f6cecd47a3ef..a4dbac23ce9d 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -203,8 +203,6 @@ struct intel_engine_cs {
int (*init_context)(struct drm_i915_gem_request *req);
- void (*write_tail)(struct intel_engine_cs *ring,
- u32 value);
int (*add_request)(struct drm_i915_gem_request *req);
/* Some chipsets are not quite as coherent as advertised and need
* an expensive kick to force a true read of the up-to-date seqno.
@@ -293,6 +291,7 @@ struct intel_engine_cs {
#define I915_DISPATCH_SECURE 0x1
#define I915_DISPATCH_PINNED 0x2
#define I915_DISPATCH_RS 0x4
+ void (*submit_request)(struct drm_i915_gem_request *req);
/**
* List of objects currently involved in rendering from the
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 36/64] drm/i915: Unify request submission
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (33 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 35/64] drm/i915: Convert engine->write_tail to operate on a request Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 37/64] drm/i915: Stop passing caller's num_dwords to engine->semaphore.signal() Chris Wilson
` (28 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Move request submission from emit_request into its own common vfunc
from i915_add_request().
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_request.c | 7 +++----
drivers/gpu/drm/i915/i915_guc_submission.c | 9 ++++++---
drivers/gpu/drm/i915/intel_guc.h | 1 -
drivers/gpu/drm/i915/intel_lrc.c | 10 +++-------
drivers/gpu/drm/i915/intel_ringbuffer.c | 26 ++++++++++----------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 23 +++++++++++------------
6 files changed, 33 insertions(+), 43 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 18b9278d16f3..bb5a15eedfa4 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -468,12 +468,10 @@ void __i915_add_request(struct drm_i915_gem_request *request,
*/
request->postfix = intel_ring_get_tail(ring);
- if (i915.enable_execlists)
- ret = engine->emit_request(request);
- else
- ret = engine->add_request(request);
/* Not allowed to fail! */
+ ret = engine->emit_request(request);
WARN(ret, "emit|add_request failed: %d!\n", ret);
+
/* Sanity check that the reserved size was large enough. */
ret = intel_ring_get_tail(ring) - request_start;
if (ret < 0)
@@ -484,6 +482,7 @@ void __i915_add_request(struct drm_i915_gem_request *request,
reserved_tail, ret);
i915_gem_mark_busy(engine);
+ engine->submit_request(request);
}
static unsigned long local_clock_us(unsigned int *cpu)
diff --git a/drivers/gpu/drm/i915/i915_guc_submission.c b/drivers/gpu/drm/i915/i915_guc_submission.c
index eccd34832fe6..32d0e1890950 100644
--- a/drivers/gpu/drm/i915/i915_guc_submission.c
+++ b/drivers/gpu/drm/i915/i915_guc_submission.c
@@ -585,7 +585,7 @@ static int guc_ring_doorbell(struct i915_guc_client *gc)
* The only error here arises if the doorbell hardware isn't functioning
* as expected, which really shouln't happen.
*/
-int i915_guc_submit(struct drm_i915_gem_request *rq)
+static void i915_guc_submit(struct drm_i915_gem_request *rq)
{
unsigned int engine_id = rq->engine->id;
struct intel_guc *guc = &rq->i915->guc;
@@ -602,8 +602,6 @@ int i915_guc_submit(struct drm_i915_gem_request *rq)
guc->submissions[engine_id] += 1;
guc->last_seqno[engine_id] = rq->fence.seqno;
-
- return b_ret;
}
/*
@@ -992,6 +990,7 @@ int i915_guc_submission_enable(struct drm_i915_private *dev_priv)
{
struct intel_guc *guc = &dev_priv->guc;
struct i915_guc_client *client;
+ struct intel_engine_cs *engine;
/* client for execbuf submission */
client = guc_client_alloc(dev_priv,
@@ -1006,6 +1005,10 @@ int i915_guc_submission_enable(struct drm_i915_private *dev_priv)
host2guc_sample_forcewake(guc, client);
guc_init_doorbell_hw(guc);
+ /* Take over from manual control of ELSP (execlists) */
+ for_each_engine(engine, dev_priv)
+ engine->submit_request = i915_guc_submit;
+
return 0;
}
diff --git a/drivers/gpu/drm/i915/intel_guc.h b/drivers/gpu/drm/i915/intel_guc.h
index 3e3e743740c0..623cf26cd784 100644
--- a/drivers/gpu/drm/i915/intel_guc.h
+++ b/drivers/gpu/drm/i915/intel_guc.h
@@ -160,7 +160,6 @@ extern int intel_guc_resume(struct drm_device *dev);
int i915_guc_submission_init(struct drm_i915_private *dev_priv);
int i915_guc_submission_enable(struct drm_i915_private *dev_priv);
int i915_guc_wq_check_space(struct drm_i915_gem_request *rq);
-int i915_guc_submit(struct drm_i915_gem_request *rq);
void i915_guc_submission_disable(struct drm_i915_private *dev_priv);
void i915_guc_submission_fini(struct drm_i915_private *dev_priv);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index 3b98148d1cf8..a8332135eea1 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -776,12 +776,6 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
*/
request->previous_context = engine->last_context;
engine->last_context = request->ctx;
-
- if (i915.enable_guc_submission)
- i915_guc_submit(request);
- else
- execlists_context_queue(request);
-
return 0;
}
@@ -1905,8 +1899,10 @@ logical_ring_default_vfuncs(struct intel_engine_cs *engine)
{
/* Default vfuncs which can be overriden by each engine. */
engine->init_hw = gen8_init_common_ring;
- engine->emit_request = gen8_emit_request;
engine->emit_flush = gen8_emit_flush;
+ engine->emit_request = gen8_emit_request;
+ engine->submit_request = execlists_context_queue;
+
engine->irq_enable = gen8_logical_ring_enable_irq;
engine->irq_disable = gen8_logical_ring_disable_irq;
engine->emit_bb_start = gen8_emit_bb_start;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 63166a7cf70d..47f67ccab0a6 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1431,15 +1431,14 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
}
/**
- * gen6_add_request - Update the semaphore mailbox registers
+ * gen6_emit_request - Update the semaphore mailbox registers
*
* @request - request to write to the ring
*
* Update the mailbox registers in the *other* rings with the current seqno.
* This acts like a signal in the canonical semaphore.
*/
-static int
-gen6_add_request(struct drm_i915_gem_request *req)
+static int gen6_emit_request(struct drm_i915_gem_request *req)
{
struct intel_ring *ring = req->ring;
int ret;
@@ -1460,13 +1459,10 @@ gen6_add_request(struct drm_i915_gem_request *req)
req->tail = intel_ring_get_tail(ring);
- req->engine->submit_request(req);
-
return 0;
}
-static int
-gen8_render_add_request(struct drm_i915_gem_request *req)
+static int gen8_render_emit_request(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
struct intel_ring *ring = req->ring;
@@ -1490,8 +1486,9 @@ gen8_render_add_request(struct drm_i915_gem_request *req)
intel_ring_emit(ring, 0);
intel_ring_emit(ring, MI_USER_INTERRUPT);
intel_ring_emit(ring, MI_NOOP);
+ intel_ring_advance(ring);
- req->engine->submit_request(req);
+ req->tail = intel_ring_get_tail(ring);
return 0;
}
@@ -1696,8 +1693,7 @@ bsd_ring_flush(struct drm_i915_gem_request *req,
return 0;
}
-static int
-i9xx_add_request(struct drm_i915_gem_request *req)
+static int i9xx_emit_request(struct drm_i915_gem_request *req)
{
struct intel_ring *ring = req->ring;
int ret;
@@ -1714,8 +1710,6 @@ i9xx_add_request(struct drm_i915_gem_request *req)
req->tail = intel_ring_get_tail(ring);
- req->engine->submit_request(req);
-
return 0;
}
@@ -2824,11 +2818,11 @@ static void intel_ring_default_vfuncs(struct drm_i915_private *dev_priv,
struct intel_engine_cs *engine)
{
engine->init_hw = init_ring_common;
- engine->submit_request = i9xx_submit_request;
- engine->add_request = i9xx_add_request;
+ engine->emit_request = i9xx_emit_request;
if (INTEL_GEN(dev_priv) >= 6)
- engine->add_request = gen6_add_request;
+ engine->emit_request = gen6_emit_request;
+ engine->submit_request = i9xx_submit_request;
if (INTEL_GEN(dev_priv) >= 8)
engine->emit_bb_start = gen8_emit_bb_start;
@@ -2865,7 +2859,7 @@ int intel_init_render_ring_buffer(struct drm_device *dev)
if (INTEL_GEN(dev_priv) >= 8) {
engine->init_context = intel_rcs_ctx_init;
- engine->add_request = gen8_render_add_request;
+ engine->emit_request = gen8_render_emit_request;
engine->emit_flush = gen8_render_ring_flush;
if (i915.semaphores)
engine->semaphore.signal = gen8_rcs_signal;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index a4dbac23ce9d..f6fdfb8c1776 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -203,7 +203,17 @@ struct intel_engine_cs {
int (*init_context)(struct drm_i915_gem_request *req);
- int (*add_request)(struct drm_i915_gem_request *req);
+ int (*emit_flush)(struct drm_i915_gem_request *request,
+ u32 invalidate_domains,
+ u32 flush_domains);
+ int (*emit_bb_start)(struct drm_i915_gem_request *req,
+ u64 offset, u32 length,
+ unsigned int dispatch_flags);
+#define I915_DISPATCH_SECURE 0x1
+#define I915_DISPATCH_PINNED 0x2
+#define I915_DISPATCH_RS 0x4
+ int (*emit_request)(struct drm_i915_gem_request *req);
+ void (*submit_request)(struct drm_i915_gem_request *req);
/* Some chipsets are not quite as coherent as advertised and need
* an expensive kick to force a true read of the up-to-date seqno.
* However, the up-to-date seqno is not always required and the last
@@ -281,17 +291,6 @@ struct intel_engine_cs {
unsigned int idle_lite_restore_wa;
bool disable_lite_restore_wa;
u32 ctx_desc_template;
- int (*emit_request)(struct drm_i915_gem_request *request);
- int (*emit_flush)(struct drm_i915_gem_request *request,
- u32 invalidate_domains,
- u32 flush_domains);
- int (*emit_bb_start)(struct drm_i915_gem_request *req,
- u64 offset, u32 length,
- unsigned int dispatch_flags);
-#define I915_DISPATCH_SECURE 0x1
-#define I915_DISPATCH_PINNED 0x2
-#define I915_DISPATCH_RS 0x4
- void (*submit_request)(struct drm_i915_gem_request *req);
/**
* List of objects currently involved in rendering from the
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 37/64] drm/i915: Stop passing caller's num_dwords to engine->semaphore.signal()
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (34 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 36/64] drm/i915: Unify request submission Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 38/64] drm/i915: Reuse legacy breadcrumbs + tail emission Chris Wilson
` (27 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Rather than pass in the num_dwords that the caller wishes to use after
the signal command packet, split the breadcrumb emission into two phases
and have both the signal and breadcrumb individiually acquire space on
the ring. This makes the interface simpler for the reader, and will
simplify for patches.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_ringbuffer.c | 51 ++++++++++++++-------------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 4 +--
2 files changed, 23 insertions(+), 32 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 47f67ccab0a6..7f5e0b34273f 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1312,10 +1312,8 @@ static void render_ring_cleanup(struct intel_engine_cs *engine)
intel_fini_pipe_control(engine);
}
-static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
- unsigned int num_dwords)
+static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req)
{
-#define MBOX_UPDATE_DWORDS 8
struct intel_ring *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
@@ -1323,10 +1321,7 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
int ret, num_rings;
num_rings = hweight32(INTEL_INFO(dev_priv)->ring_mask);
- num_dwords += (num_rings-1) * MBOX_UPDATE_DWORDS;
-#undef MBOX_UPDATE_DWORDS
-
- ret = intel_ring_begin(signaller_req, num_dwords);
+ ret = intel_ring_begin(signaller_req, (num_rings-1) * 8);
if (ret)
return ret;
@@ -1350,14 +1345,13 @@ static int gen8_rcs_signal(struct drm_i915_gem_request *signaller_req,
MI_SEMAPHORE_TARGET(waiter->hw_id));
intel_ring_emit(signaller, 0);
}
+ intel_ring_advance(signaller);
return 0;
}
-static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
- unsigned int num_dwords)
+static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req)
{
-#define MBOX_UPDATE_DWORDS 6
struct intel_ring *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
struct intel_engine_cs *waiter;
@@ -1365,10 +1359,7 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
int ret, num_rings;
num_rings = hweight32(INTEL_INFO(dev_priv)->ring_mask);
- num_dwords += (num_rings-1) * MBOX_UPDATE_DWORDS;
-#undef MBOX_UPDATE_DWORDS
-
- ret = intel_ring_begin(signaller_req, num_dwords);
+ ret = intel_ring_begin(signaller_req, (num_rings-1) * 6);
if (ret)
return ret;
@@ -1390,12 +1381,12 @@ static int gen8_xcs_signal(struct drm_i915_gem_request *signaller_req,
MI_SEMAPHORE_TARGET(waiter->hw_id));
intel_ring_emit(signaller, 0);
}
+ intel_ring_advance(signaller);
return 0;
}
-static int gen6_signal(struct drm_i915_gem_request *signaller_req,
- unsigned int num_dwords)
+static int gen6_signal(struct drm_i915_gem_request *signaller_req)
{
struct intel_ring *signaller = signaller_req->ring;
struct drm_i915_private *dev_priv = signaller_req->i915;
@@ -1403,12 +1394,8 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
enum intel_engine_id id;
int ret, num_rings;
-#define MBOX_UPDATE_DWORDS 3
num_rings = hweight32(INTEL_INFO(dev_priv)->ring_mask);
- num_dwords += round_up((num_rings-1) * MBOX_UPDATE_DWORDS, 2);
-#undef MBOX_UPDATE_DWORDS
-
- ret = intel_ring_begin(signaller_req, num_dwords);
+ ret = intel_ring_begin(signaller_req, round_up((num_rings-1) * 3, 2));
if (ret)
return ret;
@@ -1426,6 +1413,7 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req,
/* If num_dwords was rounded, make sure the tail pointer is correct */
if (num_rings % 2 == 0)
intel_ring_emit(signaller, MI_NOOP);
+ intel_ring_advance(signaller);
return 0;
}
@@ -1443,11 +1431,13 @@ static int gen6_emit_request(struct drm_i915_gem_request *req)
struct intel_ring *ring = req->ring;
int ret;
- if (req->engine->semaphore.signal)
- ret = req->engine->semaphore.signal(req, 4);
- else
- ret = intel_ring_begin(req, 4);
+ if (req->engine->semaphore.signal) {
+ ret = req->engine->semaphore.signal(req);
+ if (ret)
+ return ret;
+ }
+ ret = intel_ring_begin(req, 4);
if (ret)
return ret;
@@ -1468,10 +1458,13 @@ static int gen8_render_emit_request(struct drm_i915_gem_request *req)
struct intel_ring *ring = req->ring;
int ret;
- if (engine->semaphore.signal)
- ret = engine->semaphore.signal(req, 8);
- else
- ret = intel_ring_begin(req, 8);
+ if (engine->semaphore.signal) {
+ ret = engine->semaphore.signal(req);
+ if (ret)
+ return ret;
+ }
+
+ ret = intel_ring_begin(req, 8);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index f6fdfb8c1776..ce620d426e8c 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -277,9 +277,7 @@ struct intel_engine_cs {
int (*sync_to)(struct drm_i915_gem_request *to_req,
struct intel_engine_cs *from,
u32 seqno);
- int (*signal)(struct drm_i915_gem_request *signaller_req,
- /* num_dwords needed by caller */
- unsigned int num_dwords);
+ int (*signal)(struct drm_i915_gem_request *signaller_req);
} semaphore;
/* Execlists */
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 38/64] drm/i915: Reuse legacy breadcrumbs + tail emission
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (35 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 37/64] drm/i915: Stop passing caller's num_dwords to engine->semaphore.signal() Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 39/64] drm/i915/ringbuffer: Specialise SNB+ request emission for semaphores Chris Wilson
` (26 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
As GEN6+ is now a simple variant on the basic breadcrumbs + tail write,
reuse the common code.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_ringbuffer.c | 71 ++++++++++++++-------------------
1 file changed, 29 insertions(+), 42 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 7f5e0b34273f..0f613239d044 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1418,25 +1418,18 @@ static int gen6_signal(struct drm_i915_gem_request *signaller_req)
return 0;
}
-/**
- * gen6_emit_request - Update the semaphore mailbox registers
- *
- * @request - request to write to the ring
- *
- * Update the mailbox registers in the *other* rings with the current seqno.
- * This acts like a signal in the canonical semaphore.
- */
-static int gen6_emit_request(struct drm_i915_gem_request *req)
+static void i9xx_submit_request(struct drm_i915_gem_request *request)
+{
+ struct drm_i915_private *dev_priv = request->i915;
+
+ I915_WRITE_TAIL(request->engine, request->tail);
+}
+
+static int i9xx_emit_request(struct drm_i915_gem_request *req)
{
struct intel_ring *ring = req->ring;
int ret;
- if (req->engine->semaphore.signal) {
- ret = req->engine->semaphore.signal(req);
- if (ret)
- return ret;
- }
-
ret = intel_ring_begin(req, 4);
if (ret)
return ret;
@@ -1452,6 +1445,27 @@ static int gen6_emit_request(struct drm_i915_gem_request *req)
return 0;
}
+/**
+ * gen6_emit_request - Update the semaphore mailbox registers
+ *
+ * @request - request to write to the ring
+ *
+ * Update the mailbox registers in the *other* rings with the current seqno.
+ * This acts like a signal in the canonical semaphore.
+ */
+static int gen6_emit_request(struct drm_i915_gem_request *req)
+{
+ if (req->engine->semaphore.signal) {
+ int ret;
+
+ ret = req->engine->semaphore.signal(req);
+ if (ret)
+ return ret;
+ }
+
+ return i9xx_emit_request(req);
+}
+
static int gen8_render_emit_request(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
@@ -1686,33 +1700,6 @@ bsd_ring_flush(struct drm_i915_gem_request *req,
return 0;
}
-static int i9xx_emit_request(struct drm_i915_gem_request *req)
-{
- struct intel_ring *ring = req->ring;
- int ret;
-
- ret = intel_ring_begin(req, 4);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, MI_STORE_DWORD_INDEX);
- intel_ring_emit(ring, I915_GEM_HWS_INDEX << MI_STORE_DWORD_INDEX_SHIFT);
- intel_ring_emit(ring, req->fence.seqno);
- intel_ring_emit(ring, MI_USER_INTERRUPT);
- intel_ring_advance(ring);
-
- req->tail = intel_ring_get_tail(ring);
-
- return 0;
-}
-
-static void i9xx_submit_request(struct drm_i915_gem_request *request)
-{
- struct drm_i915_private *dev_priv = request->i915;
-
- I915_WRITE_TAIL(request->engine, request->tail);
-}
-
static void
gen6_irq_enable(struct intel_engine_cs *engine)
{
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 39/64] drm/i915/ringbuffer: Specialise SNB+ request emission for semaphores
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (36 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 38/64] drm/i915: Reuse legacy breadcrumbs + tail emission Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 40/64] drm/i915: Remove duplicate golden render state init from execlists Chris Wilson
` (25 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
As gen6_emit_request() only differs from i9xx_emit_request() when
semaphores are enabled, only use the specialised vfunc in that scenario.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_ringbuffer.c | 18 ++++++++----------
1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 0f613239d044..dd5e13f0e7e9 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1446,22 +1446,20 @@ static int i9xx_emit_request(struct drm_i915_gem_request *req)
}
/**
- * gen6_emit_request - Update the semaphore mailbox registers
+ * gen6_sema_emit_request - Update the semaphore mailbox registers
*
* @request - request to write to the ring
*
* Update the mailbox registers in the *other* rings with the current seqno.
* This acts like a signal in the canonical semaphore.
*/
-static int gen6_emit_request(struct drm_i915_gem_request *req)
+static int gen6_sema_emit_request(struct drm_i915_gem_request *req)
{
- if (req->engine->semaphore.signal) {
- int ret;
+ int ret;
- ret = req->engine->semaphore.signal(req);
- if (ret)
- return ret;
- }
+ ret = req->engine->semaphore.signal(req);
+ if (ret)
+ return ret;
return i9xx_emit_request(req);
}
@@ -2700,6 +2698,8 @@ static void intel_ring_init_semaphores(struct drm_i915_private *dev_priv,
if (!i915.semaphores)
return;
+ engine->emit_request = gen6_sema_emit_request;
+
if (INTEL_GEN(dev_priv) >= 8) {
u64 offset = i915_gem_obj_ggtt_offset(dev_priv->semaphore_obj);
@@ -2800,8 +2800,6 @@ static void intel_ring_default_vfuncs(struct drm_i915_private *dev_priv,
engine->init_hw = init_ring_common;
engine->emit_request = i9xx_emit_request;
- if (INTEL_GEN(dev_priv) >= 6)
- engine->emit_request = gen6_emit_request;
engine->submit_request = i9xx_submit_request;
if (INTEL_GEN(dev_priv) >= 8)
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 40/64] drm/i915: Remove duplicate golden render state init from execlists
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (37 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 39/64] drm/i915/ringbuffer: Specialise SNB+ request emission for semaphores Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 41/64] drm/i915: Unify legacy/execlists submit_execbuf callbacks Chris Wilson
` (24 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Now that we use the same vfuncs for emitting the batch buffer in both
execlists and legacy, the golden render state initialisation is
identical between both.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_render_state.c | 23 +++++++++++++------
drivers/gpu/drm/i915/i915_gem_render_state.h | 18 ---------------
drivers/gpu/drm/i915/intel_lrc.c | 34 +---------------------------
drivers/gpu/drm/i915/intel_renderstate.h | 16 +++++++++----
4 files changed, 28 insertions(+), 63 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_render_state.c b/drivers/gpu/drm/i915/i915_gem_render_state.c
index 2ba759f3ab6f..dd84793d75d9 100644
--- a/drivers/gpu/drm/i915/i915_gem_render_state.c
+++ b/drivers/gpu/drm/i915/i915_gem_render_state.c
@@ -28,6 +28,15 @@
#include "i915_drv.h"
#include "intel_renderstate.h"
+struct render_state {
+ const struct intel_renderstate_rodata *rodata;
+ struct drm_i915_gem_object *obj;
+ u64 ggtt_offset;
+ int gen;
+ u32 aux_batch_size;
+ u32 aux_batch_offset;
+};
+
static const struct intel_renderstate_rodata *
render_state_get_rodata(const int gen)
{
@@ -51,6 +60,7 @@ static int render_state_init(struct render_state *so,
int ret;
so->gen = INTEL_GEN(dev_priv);
+ so->ggtt_offset = 0;
so->rodata = render_state_get_rodata(so->gen);
if (so->rodata == NULL)
return 0;
@@ -192,14 +202,14 @@ err_out:
#undef OUT_BATCH
-void i915_gem_render_state_fini(struct render_state *so)
+static void render_state_fini(struct render_state *so)
{
i915_gem_object_ggtt_unpin(so->obj);
i915_gem_object_put(so->obj);
}
-int i915_gem_render_state_prepare(struct intel_engine_cs *engine,
- struct render_state *so)
+static int render_state_prepare(struct intel_engine_cs *engine,
+ struct render_state *so)
{
int ret;
@@ -215,7 +225,7 @@ int i915_gem_render_state_prepare(struct intel_engine_cs *engine,
ret = render_state_setup(so);
if (ret) {
- i915_gem_render_state_fini(so);
+ render_state_fini(so);
return ret;
}
@@ -227,7 +237,7 @@ int i915_gem_render_state_init(struct drm_i915_gem_request *req)
struct render_state so;
int ret;
- ret = i915_gem_render_state_prepare(req->engine, &so);
+ ret = render_state_prepare(req->engine, &so);
if (ret)
return ret;
@@ -251,8 +261,7 @@ int i915_gem_render_state_init(struct drm_i915_gem_request *req)
}
i915_vma_move_to_active(i915_gem_obj_to_ggtt(so.obj), req);
-
out:
- i915_gem_render_state_fini(&so);
+ render_state_fini(&so);
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_render_state.h b/drivers/gpu/drm/i915/i915_gem_render_state.h
index 6aaa3a10a630..c44fca8599bb 100644
--- a/drivers/gpu/drm/i915/i915_gem_render_state.h
+++ b/drivers/gpu/drm/i915/i915_gem_render_state.h
@@ -26,24 +26,6 @@
#include <linux/types.h>
-struct intel_renderstate_rodata {
- const u32 *reloc;
- const u32 *batch;
- const u32 batch_items;
-};
-
-struct render_state {
- const struct intel_renderstate_rodata *rodata;
- struct drm_i915_gem_object *obj;
- u64 ggtt_offset;
- int gen;
- u32 aux_batch_size;
- u32 aux_batch_offset;
-};
-
int i915_gem_render_state_init(struct drm_i915_gem_request *req);
-void i915_gem_render_state_fini(struct render_state *so);
-int i915_gem_render_state_prepare(struct intel_engine_cs *engine,
- struct render_state *so);
#endif /* _I915_GEM_RENDER_STATE_H_ */
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index a8332135eea1..dd9acef414b6 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1794,38 +1794,6 @@ static int gen8_emit_request_render(struct drm_i915_gem_request *request)
return intel_logical_ring_advance_and_submit(request);
}
-static int intel_lr_context_render_state_init(struct drm_i915_gem_request *req)
-{
- struct render_state so;
- int ret;
-
- ret = i915_gem_render_state_prepare(req->engine, &so);
- if (ret)
- return ret;
-
- if (so.rodata == NULL)
- return 0;
-
- ret = req->engine->emit_bb_start(req, so.ggtt_offset,
- so.rodata->batch_items * 4,
- I915_DISPATCH_SECURE);
- if (ret)
- goto out;
-
- ret = req->engine->emit_bb_start(req,
- (so.ggtt_offset + so.aux_batch_offset),
- so.aux_batch_size,
- I915_DISPATCH_SECURE);
- if (ret)
- goto out;
-
- i915_vma_move_to_active(i915_gem_obj_to_ggtt(so.obj), req);
-
-out:
- i915_gem_render_state_fini(&so);
- return ret;
-}
-
static int gen8_init_rcs_context(struct drm_i915_gem_request *req)
{
int ret;
@@ -1842,7 +1810,7 @@ static int gen8_init_rcs_context(struct drm_i915_gem_request *req)
if (ret)
DRM_ERROR("MOCS failed to program: expect performance issues.\n");
- return intel_lr_context_render_state_init(req);
+ return i915_gem_render_state_init(req);
}
/**
diff --git a/drivers/gpu/drm/i915/intel_renderstate.h b/drivers/gpu/drm/i915/intel_renderstate.h
index 5bd69852752c..08f6fea05a2c 100644
--- a/drivers/gpu/drm/i915/intel_renderstate.h
+++ b/drivers/gpu/drm/i915/intel_renderstate.h
@@ -24,12 +24,13 @@
#ifndef _INTEL_RENDERSTATE_H
#define _INTEL_RENDERSTATE_H
-#include "i915_drv.h"
+#include <linux/types.h>
-extern const struct intel_renderstate_rodata gen6_null_state;
-extern const struct intel_renderstate_rodata gen7_null_state;
-extern const struct intel_renderstate_rodata gen8_null_state;
-extern const struct intel_renderstate_rodata gen9_null_state;
+struct intel_renderstate_rodata {
+ const u32 *reloc;
+ const u32 *batch;
+ const u32 batch_items;
+};
#define RO_RENDERSTATE(_g) \
const struct intel_renderstate_rodata gen ## _g ## _null_state = { \
@@ -38,4 +39,9 @@ extern const struct intel_renderstate_rodata gen9_null_state;
.batch_items = sizeof(gen ## _g ## _null_state_batch)/4, \
}
+extern const struct intel_renderstate_rodata gen6_null_state;
+extern const struct intel_renderstate_rodata gen7_null_state;
+extern const struct intel_renderstate_rodata gen8_null_state;
+extern const struct intel_renderstate_rodata gen9_null_state;
+
#endif /* INTEL_RENDERSTATE_H */
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 41/64] drm/i915: Unify legacy/execlists submit_execbuf callbacks
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (38 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 40/64] drm/i915: Remove duplicate golden render state init from execlists Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 42/64] drm/i915: Simplify calling engine->sync_to Chris Wilson
` (23 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Now that emitting requests is identical between legacy and execlists, we
can use the same function to build up the ring for submitting to either
engine. (With the exception of i915_switch_contexts(), but in time that
will also be handled gracefully.)
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 20 -----
drivers/gpu/drm/i915/i915_gem.c | 2 -
drivers/gpu/drm/i915/i915_gem_context.c | 3 +-
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 24 ++++--
drivers/gpu/drm/i915/intel_lrc.c | 123 -----------------------------
drivers/gpu/drm/i915/intel_lrc.h | 4 -
6 files changed, 20 insertions(+), 156 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 4a5ce22dcaf0..bddb5ecec902 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1700,18 +1700,6 @@ struct i915_virtual_gpu {
bool active;
};
-struct i915_execbuffer_params {
- struct drm_device *dev;
- struct drm_file *file;
- uint32_t dispatch_flags;
- uint32_t args_batch_start_offset;
- uint64_t batch_obj_vm_offset;
- struct intel_engine_cs *engine;
- struct drm_i915_gem_object *batch_obj;
- struct i915_gem_context *ctx;
- struct drm_i915_gem_request *request;
-};
-
/* used in computing the new watermarks state */
struct intel_wm_config {
unsigned int num_pipes_active;
@@ -2011,9 +1999,6 @@ struct drm_i915_private {
/* Abstract the submission mechanism (legacy ringbuffer or execlists) away */
struct {
- int (*execbuf_submit)(struct i915_execbuffer_params *params,
- struct drm_i915_gem_execbuffer2 *args,
- struct list_head *vmas);
int (*init_engines)(struct drm_device *dev);
void (*cleanup_engine)(struct intel_engine_cs *engine);
void (*stop_engine)(struct intel_engine_cs *engine);
@@ -2973,11 +2958,6 @@ int i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
int i915_gem_sw_finish_ioctl(struct drm_device *dev, void *data,
struct drm_file *file_priv);
-void i915_gem_execbuffer_move_to_active(struct list_head *vmas,
- struct drm_i915_gem_request *req);
-int i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
- struct drm_i915_gem_execbuffer2 *args,
- struct list_head *vmas);
int i915_gem_execbuffer(struct drm_device *dev, void *data,
struct drm_file *file_priv);
int i915_gem_execbuffer2(struct drm_device *dev, void *data,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 7c21a4f4028d..7a4841292dcc 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -4537,12 +4537,10 @@ int i915_gem_init(struct drm_device *dev)
mutex_lock(&dev->struct_mutex);
if (!i915.enable_execlists) {
- dev_priv->gt.execbuf_submit = i915_gem_ringbuffer_submission;
dev_priv->gt.init_engines = i915_gem_init_engines;
dev_priv->gt.cleanup_engine = intel_engine_cleanup;
dev_priv->gt.stop_engine = intel_engine_stop;
} else {
- dev_priv->gt.execbuf_submit = intel_execlists_submission;
dev_priv->gt.init_engines = intel_logical_rings_init;
dev_priv->gt.cleanup_engine = intel_logical_ring_cleanup;
dev_priv->gt.stop_engine = intel_logical_ring_stop;
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 32051a68246f..ed071fb1b713 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -893,8 +893,9 @@ int i915_switch_context(struct drm_i915_gem_request *req)
{
struct intel_engine_cs *engine = req->engine;
- WARN_ON(i915.enable_execlists);
lockdep_assert_held(&req->i915->drm.struct_mutex);
+ if (i915.enable_execlists)
+ return 0;
if (!req->ctx->engine[engine->id].state) {
struct i915_gem_context *to = req->ctx;
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 287e4e7b3e51..5bfb2e2f3e67 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -45,6 +45,18 @@
#define BATCH_OFFSET_BIAS (256*1024)
+struct i915_execbuffer_params {
+ struct drm_device *dev;
+ struct drm_file *file;
+ u32 dispatch_flags;
+ u32 args_batch_start_offset;
+ u32 batch_obj_vm_offset;
+ struct intel_engine_cs *engine;
+ struct drm_i915_gem_object *batch_obj;
+ struct i915_gem_context *ctx;
+ struct drm_i915_gem_request *request;
+};
+
struct eb_vmas {
struct list_head vmas;
int and;
@@ -1109,7 +1121,7 @@ static void eb_export_fence(struct drm_i915_gem_object *obj,
mutex_unlock(&resv->lock.base);
}
-void
+static void
i915_gem_execbuffer_move_to_active(struct list_head *vmas,
struct drm_i915_gem_request *req)
{
@@ -1237,10 +1249,10 @@ err:
return ERR_PTR(ret);
}
-int
-i915_gem_ringbuffer_submission(struct i915_execbuffer_params *params,
- struct drm_i915_gem_execbuffer2 *args,
- struct list_head *vmas)
+static int
+execbuf_submit(struct i915_execbuffer_params *params,
+ struct drm_i915_gem_execbuffer2 *args,
+ struct list_head *vmas)
{
struct drm_i915_private *dev_priv = params->request->i915;
u64 exec_start, exec_len;
@@ -1649,7 +1661,7 @@ i915_gem_do_execbuffer(struct drm_device *dev, void *data,
params->batch_obj = batch_obj;
params->ctx = ctx;
- ret = dev_priv->gt.execbuf_submit(params, args, &eb->vmas);
+ ret = execbuf_submit(params, args, &eb->vmas);
err_request:
i915_gem_execbuffer_retire_commands(params);
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index dd9acef414b6..bda66b91aa02 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -645,39 +645,6 @@ static void execlists_context_queue(struct drm_i915_gem_request *request)
spin_unlock_bh(&engine->execlist_lock);
}
-static int execlists_move_to_gpu(struct drm_i915_gem_request *req,
- struct list_head *vmas)
-{
- const unsigned other_rings = ~intel_engine_flag(req->engine);
- struct i915_vma *vma;
- uint32_t flush_domains = 0;
- bool flush_chipset = false;
- int ret;
-
- list_for_each_entry(vma, vmas, exec_list) {
- struct drm_i915_gem_object *obj = vma->obj;
-
- if (obj->active & other_rings) {
- ret = i915_gem_object_sync(obj, req);
- if (ret)
- return ret;
- }
-
- if (obj->base.write_domain & I915_GEM_DOMAIN_CPU)
- flush_chipset |= i915_gem_clflush_object(obj, false);
-
- flush_domains |= obj->base.write_domain;
- }
-
- if (flush_domains & I915_GEM_DOMAIN_GTT)
- wmb();
-
- /* Unconditionally invalidate gpu caches and ensure that we do flush
- * any residual writes from the previous batch.
- */
- return req->engine->emit_flush(req, I915_GEM_GPU_DOMAINS, 0);
-}
-
int intel_logical_ring_alloc_request_extras(struct drm_i915_gem_request *request)
{
struct intel_engine_cs *engine = request->engine;
@@ -779,96 +746,6 @@ intel_logical_ring_advance_and_submit(struct drm_i915_gem_request *request)
return 0;
}
-/**
- * execlists_submission() - submit a batchbuffer for execution, Execlists style
- * @params: execbuffer call parameters.
- * @args: execbuffer call arguments.
- * @vmas: list of vmas.
- *
- * This is the evil twin version of i915_gem_ringbuffer_submission. It abstracts
- * away the submission details of the execbuffer ioctl call.
- *
- * Return: non-zero if the submission fails.
- */
-int intel_execlists_submission(struct i915_execbuffer_params *params,
- struct drm_i915_gem_execbuffer2 *args,
- struct list_head *vmas)
-{
- struct drm_device *dev = params->dev;
- struct intel_engine_cs *engine = params->engine;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct intel_ring *ring = params->request->ring;
- u64 exec_start;
- int instp_mode;
- u32 instp_mask;
- int ret;
-
- instp_mode = args->flags & I915_EXEC_CONSTANTS_MASK;
- instp_mask = I915_EXEC_CONSTANTS_MASK;
- switch (instp_mode) {
- case I915_EXEC_CONSTANTS_REL_GENERAL:
- case I915_EXEC_CONSTANTS_ABSOLUTE:
- case I915_EXEC_CONSTANTS_REL_SURFACE:
- if (instp_mode != 0 && engine->id != RCS) {
- DRM_DEBUG("non-0 rel constants mode on non-RCS\n");
- return -EINVAL;
- }
-
- if (instp_mode != dev_priv->relative_constants_mode) {
- if (instp_mode == I915_EXEC_CONSTANTS_REL_SURFACE) {
- DRM_DEBUG("rel surface constants mode invalid on gen5+\n");
- return -EINVAL;
- }
-
- /* The HW changed the meaning on this bit on gen6 */
- instp_mask &= ~I915_EXEC_CONSTANTS_REL_SURFACE;
- }
- break;
- default:
- DRM_DEBUG("execbuf with unknown constants: %d\n", instp_mode);
- return -EINVAL;
- }
-
- if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
- DRM_DEBUG("sol reset is gen7 only\n");
- return -EINVAL;
- }
-
- ret = execlists_move_to_gpu(params->request, vmas);
- if (ret)
- return ret;
-
- if (engine->id == RCS &&
- instp_mode != dev_priv->relative_constants_mode) {
- ret = intel_ring_begin(params->request, 4);
- if (ret)
- return ret;
-
- intel_ring_emit(ring, MI_NOOP);
- intel_ring_emit(ring, MI_LOAD_REGISTER_IMM(1));
- intel_ring_emit_reg(ring, INSTPM);
- intel_ring_emit(ring, instp_mask << 16 | instp_mode);
- intel_ring_advance(ring);
-
- dev_priv->relative_constants_mode = instp_mode;
- }
-
- exec_start = params->batch_obj_vm_offset +
- args->batch_start_offset;
-
- ret = engine->emit_bb_start(params->request,
- exec_start, args->batch_len,
- params->dispatch_flags);
- if (ret)
- return ret;
-
- trace_i915_gem_ring_dispatch(params->request, params->dispatch_flags);
-
- i915_gem_execbuffer_move_to_active(vmas, params->request);
-
- return 0;
-}
-
void intel_execlists_cancel_requests(struct intel_engine_cs *engine)
{
struct drm_i915_gem_request *req, *tmp;
diff --git a/drivers/gpu/drm/i915/intel_lrc.h b/drivers/gpu/drm/i915/intel_lrc.h
index 5dae5bc5046e..bcd2b4f5b676 100644
--- a/drivers/gpu/drm/i915/intel_lrc.h
+++ b/drivers/gpu/drm/i915/intel_lrc.h
@@ -92,10 +92,6 @@ uint64_t intel_lr_context_descriptor(struct i915_gem_context *ctx,
/* Execlists */
int intel_sanitize_enable_execlists(struct drm_i915_private *dev_priv,
int enable_execlists);
-struct i915_execbuffer_params;
-int intel_execlists_submission(struct i915_execbuffer_params *params,
- struct drm_i915_gem_execbuffer2 *args,
- struct list_head *vmas);
void intel_execlists_cancel_requests(struct intel_engine_cs *engine);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 42/64] drm/i915: Simplify calling engine->sync_to
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (39 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 41/64] drm/i915: Unify legacy/execlists submit_execbuf callbacks Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 43/64] drm/i915: Amalgamate GGTT/ppGTT vma debug list walkers Chris Wilson
` (22 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Since requests can no longer be generated as a side-effect of
intel_ring_begin(), we know that the seqno will be unchanged during
ring-emission. This predicatablity then means we do not have to check
for the seqno wrapping around whilst emitting the semaphore for
engine->sync_to().
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 2 +-
drivers/gpu/drm/i915/i915_gem.c | 13 ++-----
drivers/gpu/drm/i915/i915_gem_request.c | 9 +----
drivers/gpu/drm/i915/intel_ringbuffer.c | 64 ++++++++++++---------------------
drivers/gpu/drm/i915/intel_ringbuffer.h | 5 ++-
5 files changed, 30 insertions(+), 63 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index bddb5ecec902..66490046ab37 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -1752,7 +1752,7 @@ struct drm_i915_private {
struct i915_gem_context *kernel_context;
struct intel_engine_cs engine[I915_NUM_ENGINES];
struct drm_i915_gem_object *semaphore_obj;
- uint32_t last_seqno, next_seqno;
+ u32 next_seqno;
struct drm_dma_handle *status_page_dmah;
struct resource mch_res;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 7a4841292dcc..dd09eb0ec851 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2859,22 +2859,15 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
i915_gem_object_retire_request(obj, from);
} else {
int idx = intel_engine_sync_index(from->engine, to->engine);
- u32 seqno = i915_gem_request_get_seqno(from);
-
- if (seqno <= from->engine->semaphore.sync_seqno[idx])
+ if (from->fence.seqno <= from->engine->semaphore.sync_seqno[idx])
return 0;
trace_i915_gem_ring_sync_to(to, from);
- ret = to->engine->semaphore.sync_to(to, from->engine, seqno);
+ ret = to->engine->semaphore.sync_to(to, from);
if (ret)
return ret;
- /* We use last_read_req because sync_to()
- * might have just caused seqno wrap under
- * the radar.
- */
- from->engine->semaphore.sync_seqno[idx] =
- i915_gem_request_get_seqno(obj->last_read_req[from->engine->id]);
+ from->engine->semaphore.sync_seqno[idx] = from->fence.seqno;
}
return 0;
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index bb5a15eedfa4..6e99776a800b 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -264,14 +264,7 @@ int i915_gem_set_seqno(struct drm_device *dev, u32 seqno)
if (ret)
return ret;
- /* Carefully set the last_seqno value so that wrap
- * detection still works
- */
dev_priv->next_seqno = seqno;
- dev_priv->last_seqno = seqno - 1;
- if (dev_priv->last_seqno == 0)
- dev_priv->last_seqno--;
-
return 0;
}
@@ -288,7 +281,7 @@ static int i915_gem_get_seqno(struct drm_i915_private *dev_priv, u32 *seqno)
dev_priv->next_seqno = 1;
}
- *seqno = dev_priv->last_seqno = dev_priv->next_seqno++;
+ *seqno = dev_priv->next_seqno++;
return 0;
}
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index dd5e13f0e7e9..c0c25e1703e0 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -1498,12 +1498,6 @@ static int gen8_render_emit_request(struct drm_i915_gem_request *req)
return 0;
}
-static inline bool i915_gem_has_seqno_wrapped(struct drm_i915_private *dev_priv,
- u32 seqno)
-{
- return dev_priv->last_seqno < seqno;
-}
-
/**
* intel_ring_sync - sync the waiter to the signaller on seqno
*
@@ -1513,24 +1507,23 @@ static inline bool i915_gem_has_seqno_wrapped(struct drm_i915_private *dev_priv,
*/
static int
-gen8_ring_sync(struct drm_i915_gem_request *waiter_req,
- struct intel_engine_cs *signaller,
- u32 seqno)
+gen8_ring_sync(struct drm_i915_gem_request *wait,
+ struct drm_i915_gem_request *signal)
{
- struct intel_ring *waiter = waiter_req->ring;
- struct drm_i915_private *dev_priv = waiter_req->i915;
- u64 offset = GEN8_WAIT_OFFSET(waiter_req->engine, signaller->id);
+ struct intel_ring *waiter = wait->ring;
+ struct drm_i915_private *dev_priv = wait->i915;
+ u64 offset = GEN8_WAIT_OFFSET(wait->engine, signal->engine->id);
struct i915_hw_ppgtt *ppgtt;
int ret;
- ret = intel_ring_begin(waiter_req, 4);
+ ret = intel_ring_begin(wait, 4);
if (ret)
return ret;
intel_ring_emit(waiter, MI_SEMAPHORE_WAIT |
MI_SEMAPHORE_GLOBAL_GTT |
MI_SEMAPHORE_SAD_GTE_SDD);
- intel_ring_emit(waiter, seqno);
+ intel_ring_emit(waiter, signal->fence.seqno);
intel_ring_emit(waiter, lower_32_bits(offset));
intel_ring_emit(waiter, upper_32_bits(offset));
intel_ring_advance(waiter);
@@ -1540,48 +1533,37 @@ gen8_ring_sync(struct drm_i915_gem_request *waiter_req,
* We do this on the i915_switch_context() following the wait and
* before the dispatch.
*/
- ppgtt = waiter_req->ctx->ppgtt;
- if (ppgtt && waiter_req->engine->id != RCS)
- ppgtt->pd_dirty_rings |= intel_engine_flag(waiter_req->engine);
+ ppgtt = wait->ctx->ppgtt;
+ if (ppgtt && wait->engine->id != RCS)
+ ppgtt->pd_dirty_rings |= intel_engine_flag(wait->engine);
return 0;
}
static int
-gen6_ring_sync(struct drm_i915_gem_request *waiter_req,
- struct intel_engine_cs *signaller,
- u32 seqno)
+gen6_ring_sync(struct drm_i915_gem_request *wait,
+ struct drm_i915_gem_request *signal)
{
- struct intel_ring *waiter = waiter_req->ring;
+ struct intel_ring *waiter = wait->ring;
u32 dw1 = MI_SEMAPHORE_MBOX |
MI_SEMAPHORE_COMPARE |
MI_SEMAPHORE_REGISTER;
- u32 wait_mbox = signaller->semaphore.mbox.wait[waiter_req->engine->id];
+ u32 wait_mbox = signal->engine->semaphore.mbox.wait[wait->engine->id];
int ret;
- /* Throughout all of the GEM code, seqno passed implies our current
- * seqno is >= the last seqno executed. However for hardware the
- * comparison is strictly greater than.
- */
- seqno -= 1;
-
WARN_ON(wait_mbox == MI_SEMAPHORE_SYNC_INVALID);
- ret = intel_ring_begin(waiter_req, 4);
+ ret = intel_ring_begin(wait, 4);
if (ret)
return ret;
- /* If seqno wrap happened, omit the wait with no-ops */
- if (likely(!i915_gem_has_seqno_wrapped(waiter_req->i915, seqno))) {
- intel_ring_emit(waiter, dw1 | wait_mbox);
- intel_ring_emit(waiter, seqno);
- intel_ring_emit(waiter, 0);
- intel_ring_emit(waiter, MI_NOOP);
- } else {
- intel_ring_emit(waiter, MI_NOOP);
- intel_ring_emit(waiter, MI_NOOP);
- intel_ring_emit(waiter, MI_NOOP);
- intel_ring_emit(waiter, MI_NOOP);
- }
+ intel_ring_emit(waiter, dw1 | wait_mbox);
+ /* Throughout all of the GEM code, seqno passed implies our current
+ * seqno is >= the last seqno executed. However for hardware the
+ * comparison is strictly greater than.
+ */
+ intel_ring_emit(waiter, signal->fence.seqno - 1);
+ intel_ring_emit(waiter, 0);
+ intel_ring_emit(waiter, MI_NOOP);
intel_ring_advance(waiter);
return 0;
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index ce620d426e8c..2622267bf064 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -274,9 +274,8 @@ struct intel_engine_cs {
};
/* AKA wait() */
- int (*sync_to)(struct drm_i915_gem_request *to_req,
- struct intel_engine_cs *from,
- u32 seqno);
+ int (*sync_to)(struct drm_i915_gem_request *to,
+ struct drm_i915_gem_request *from);
int (*signal)(struct drm_i915_gem_request *signaller_req);
} semaphore;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 43/64] drm/i915: Amalgamate GGTT/ppGTT vma debug list walkers
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (40 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 42/64] drm/i915: Simplify calling engine->sync_to Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 44/64] drm/i915: Split early global GTT initialisation Chris Wilson
` (21 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
As we can now have multiple VMA inside the global GTT (with partial
mappings, rotations, etc), it is no longer true that there may just be a
single GGTT entry and so we should walk the full vma_list to count up
the actual usage. In addition to unifying the two walkers, switch from
multiplying the object size for each vma to summing the bound vma sizes.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 46 +++++++++++++++----------------------
1 file changed, 18 insertions(+), 28 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index fc337f5a593a..c7dd739eada5 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -335,6 +335,7 @@ static int per_file_stats(int id, void *ptr, void *data)
struct drm_i915_gem_object *obj = ptr;
struct file_stats *stats = data;
struct i915_vma *vma;
+ int bound = 0;
stats->count++;
stats->total += obj->base.size;
@@ -342,41 +343,30 @@ static int per_file_stats(int id, void *ptr, void *data)
if (obj->base.name || obj->base.dma_buf)
stats->shared += obj->base.size;
- if (USES_FULL_PPGTT(obj->base.dev)) {
- list_for_each_entry(vma, &obj->vma_list, obj_link) {
- struct i915_hw_ppgtt *ppgtt;
+ list_for_each_entry(vma, &obj->vma_list, obj_link) {
+ if (!drm_mm_node_allocated(&vma->node))
+ continue;
- if (!drm_mm_node_allocated(&vma->node))
- continue;
+ bound++;
- if (vma->is_ggtt) {
- stats->global += obj->base.size;
- continue;
- }
-
- ppgtt = container_of(vma->vm, struct i915_hw_ppgtt, base);
+ if (vma->is_ggtt) {
+ stats->global += vma->node.size;
+ } else {
+ struct i915_hw_ppgtt *ppgtt
+ = container_of(vma->vm,
+ struct i915_hw_ppgtt,
+ base);
if (ppgtt->file_priv != stats->file_priv)
continue;
-
- if (obj->active) /* XXX per-vma statistic */
- stats->active += obj->base.size;
- else
- stats->inactive += obj->base.size;
-
- return 0;
- }
- } else {
- if (i915_gem_obj_ggtt_bound(obj)) {
- stats->global += obj->base.size;
- if (obj->active)
- stats->active += obj->base.size;
- else
- stats->inactive += obj->base.size;
- return 0;
}
+
+ if (obj->active) /* XXX per-vma statistic */
+ stats->active += vma->node.size;
+ else
+ stats->inactive += vma->node.size;
}
- if (!list_empty(&obj->global_list))
+ if (!bound)
stats->unbound += obj->base.size;
return 0;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 44/64] drm/i915: Split early global GTT initialisation
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (41 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 43/64] drm/i915: Amalgamate GGTT/ppGTT vma debug list walkers Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 45/64] drm/i915: Store owning file on the i915_address_space Chris Wilson
` (20 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Initialising the global GTT is tricky as we wish to use the drm_mm range
manager during the modesetting initialisation (to capture stolen
allocations from the BIOS) before we actually enable GEM. To overcome
this, we currently setup the drm_mm first and then carefully rebind
them.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.c | 19 -------
drivers/gpu/drm/i915/i915_gem.c | 6 ++-
drivers/gpu/drm/i915/i915_gem_gtt.c | 99 +++++++++++++---------------------
drivers/gpu/drm/i915/i915_gem_gtt.h | 2 +-
drivers/gpu/drm/i915/i915_gem_stolen.c | 17 +++---
5 files changed, 50 insertions(+), 93 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 5f2f064b3a48..21ef73e056b2 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -988,8 +988,6 @@ static void intel_sanitize_options(struct drm_i915_private *dev_priv)
static int i915_driver_init_hw(struct drm_i915_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
- struct i915_ggtt *ggtt = &dev_priv->ggtt;
- uint32_t aperture_size;
int ret;
if (i915_inject_load_failure())
@@ -1035,7 +1033,6 @@ static int i915_driver_init_hw(struct drm_i915_private *dev_priv)
}
}
-
/* 965GM sometimes incorrectly writes to hardware status page (HWS)
* using 32bit addressing, overwriting memory if HWS is located
* above 4GB.
@@ -1054,19 +1051,6 @@ static int i915_driver_init_hw(struct drm_i915_private *dev_priv)
}
}
- aperture_size = ggtt->mappable_end;
-
- ggtt->mappable =
- io_mapping_create_wc(ggtt->mappable_base,
- aperture_size);
- if (!ggtt->mappable) {
- ret = -EIO;
- goto out_ggtt;
- }
-
- ggtt->mtrr = arch_phys_wc_add(ggtt->mappable_base,
- aperture_size);
-
pm_qos_add_request(&dev_priv->pm_qos, PM_QOS_CPU_DMA_LATENCY,
PM_QOS_DEFAULT_VALUE);
@@ -1107,14 +1091,11 @@ out_ggtt:
static void i915_driver_cleanup_hw(struct drm_i915_private *dev_priv)
{
struct drm_device *dev = &dev_priv->drm;
- struct i915_ggtt *ggtt = &dev_priv->ggtt;
if (dev->pdev->msi_enabled)
pci_disable_msi(dev->pdev);
pm_qos_remove_request(&dev_priv->pm_qos);
- arch_phys_wc_del(ggtt->mtrr);
- io_mapping_free(ggtt->mappable);
i915_ggtt_cleanup_hw(dev);
}
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index dd09eb0ec851..98f3945fc50f 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -4548,7 +4548,10 @@ int i915_gem_init(struct drm_device *dev)
intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
i915_gem_init_userptr(dev_priv);
- i915_gem_init_ggtt(dev);
+
+ ret = i915_gem_init_ggtt(dev);
+ if (ret)
+ goto out_unlock;
ret = i915_gem_context_init(dev);
if (ret)
@@ -4639,7 +4642,6 @@ i915_gem_load_init(struct drm_device *dev)
SLAB_HWCACHE_ALIGN,
NULL);
- INIT_LIST_HEAD(&dev_priv->vm_list);
INIT_LIST_HEAD(&dev_priv->context_list);
INIT_LIST_HEAD(&dev_priv->mm.unbound_list);
INIT_LIST_HEAD(&dev_priv->mm.bound_list);
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index a3d9c723dd02..5b09db68df73 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -2751,10 +2751,7 @@ static void i915_gtt_color_adjust(struct drm_mm_node *node,
}
}
-static int i915_gem_setup_global_gtt(struct drm_device *dev,
- u64 start,
- u64 mappable_end,
- u64 end)
+int i915_gem_init_ggtt(struct drm_device *dev)
{
/* Let GEM Manage all of the aperture.
*
@@ -2767,46 +2764,14 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
*/
struct drm_i915_private *dev_priv = to_i915(dev);
struct i915_ggtt *ggtt = &dev_priv->ggtt;
- struct drm_mm_node *entry;
- struct drm_i915_gem_object *obj;
unsigned long hole_start, hole_end;
+ struct drm_mm_node *entry;
int ret;
- BUG_ON(mappable_end > end);
-
- ggtt->base.start = start;
-
- /* Subtract the guard page before address space initialization to
- * shrink the range used by drm_mm */
- ggtt->base.total = end - start - PAGE_SIZE;
- i915_address_space_init(&ggtt->base, dev_priv);
- ggtt->base.total += PAGE_SIZE;
-
ret = intel_vgt_balloon(dev_priv);
if (ret)
return ret;
- if (!HAS_LLC(dev))
- ggtt->base.mm.color_adjust = i915_gtt_color_adjust;
-
- /* Mark any preallocated objects as occupied */
- list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
- struct i915_vma *vma = i915_gem_obj_to_vma(obj, &ggtt->base);
-
- DRM_DEBUG_KMS("reserving preallocated space: %llx + %zx\n",
- i915_gem_obj_ggtt_offset(obj), obj->base.size);
-
- WARN_ON(i915_gem_obj_ggtt_bound(obj));
- ret = drm_mm_reserve_node(&ggtt->base.mm, &vma->node);
- if (ret) {
- DRM_DEBUG_KMS("Reservation failed: %i\n", ret);
- return ret;
- }
- vma->bound |= GLOBAL_BIND;
- __i915_vma_set_map_and_fenceable(vma);
- list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
- }
-
/* Clear any non-preallocated blocks */
drm_mm_for_each_hole(entry, &ggtt->base.mm, hole_start, hole_end) {
DRM_DEBUG_KMS("clearing unused GTT space: [%lx, %lx]\n",
@@ -2816,9 +2781,11 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
}
/* And finally clear the reserved guard page */
- ggtt->base.clear_range(&ggtt->base, end - PAGE_SIZE, PAGE_SIZE, true);
+ ggtt->base.clear_range(&ggtt->base,
+ ggtt->base.total - PAGE_SIZE, PAGE_SIZE,
+ true);
- if (USES_PPGTT(dev) && !USES_FULL_PPGTT(dev)) {
+ if (USES_PPGTT(dev_priv) && !USES_FULL_PPGTT(dev_priv)) {
struct i915_hw_ppgtt *ppgtt;
ppgtt = kzalloc(sizeof(*ppgtt), GFP_KERNEL);
@@ -2854,16 +2821,21 @@ static int i915_gem_setup_global_gtt(struct drm_device *dev,
return 0;
}
-/**
- * i915_gem_init_ggtt - Initialize GEM for Global GTT
- * @dev: DRM device
- */
-void i915_gem_init_ggtt(struct drm_device *dev)
+static void init_global_gtt(struct drm_i915_private *dev_priv)
{
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct i915_ggtt *ggtt = &dev_priv->ggtt;
+ struct i915_address_space *ggtt = &dev_priv->ggtt.base;
+
+ INIT_LIST_HEAD(&dev_priv->vm_list);
- i915_gem_setup_global_gtt(dev, 0, ggtt->mappable_end, ggtt->base.total);
+ /* Subtract the guard page before address space initialization to
+ * shrink the range used by drm_mm.
+ */
+ ggtt->total -= PAGE_SIZE;
+ i915_address_space_init(ggtt, dev_priv);
+ ggtt->total += PAGE_SIZE;
+
+ if (!HAS_LLC(dev_priv))
+ ggtt->mm.color_adjust = i915_gtt_color_adjust;
}
/**
@@ -2891,6 +2863,9 @@ void i915_ggtt_cleanup_hw(struct drm_device *dev)
}
ggtt->base.cleanup(&ggtt->base);
+
+ arch_phys_wc_del(ggtt->mtrr);
+ io_mapping_free(ggtt->mappable);
}
static unsigned int gen6_get_total_gtt_size(u16 snb_gmch_ctl)
@@ -3251,21 +3226,14 @@ int i915_ggtt_init_hw(struct drm_device *dev)
if (ret)
return ret;
- if ((ggtt->base.total - 1) >> 32) {
- DRM_ERROR("We never expected a Global GTT with more than 32bits"
- "of address space! Found %lldM!\n",
- ggtt->base.total >> 20);
- ggtt->base.total = 1ULL << 32;
- ggtt->mappable_end = min(ggtt->mappable_end, ggtt->base.total);
- }
+ init_global_gtt(dev_priv);
- /*
- * Initialise stolen early so that we may reserve preallocated
- * objects for the BIOS to KMS transition.
- */
- ret = i915_gem_init_stolen(dev);
- if (ret)
- goto out_gtt_cleanup;
+ ggtt->mappable =
+ io_mapping_create_wc(ggtt->mappable_base, ggtt->mappable_end);
+ if (!ggtt->mappable)
+ return -EIO;
+
+ ggtt->mtrr = arch_phys_wc_add(ggtt->mappable_base, ggtt->mappable_end);
/* GMADR is the PCI mmio aperture into the global GTT. */
DRM_INFO("Memory usable by graphics device = %lluM\n",
@@ -3277,11 +3245,18 @@ int i915_ggtt_init_hw(struct drm_device *dev)
DRM_INFO("VT-d active for gfx access\n");
#endif
+ /*
+ * Initialise stolen early so that we may reserve preallocated
+ * objects for the BIOS to KMS transition.
+ */
+ ret = i915_gem_init_stolen(dev);
+ if (ret)
+ goto out_gtt_cleanup;
+
return 0;
out_gtt_cleanup:
ggtt->base.cleanup(&ggtt->base);
-
return ret;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index aa5f31d1c2ed..a0370bf950f5 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -523,7 +523,7 @@ i915_page_dir_dma_addr(const struct i915_hw_ppgtt *ppgtt, const unsigned n)
int i915_ggtt_init_hw(struct drm_device *dev);
int i915_ggtt_enable_hw(struct drm_device *dev);
-void i915_gem_init_ggtt(struct drm_device *dev);
+int i915_gem_init_ggtt(struct drm_device *dev);
void i915_ggtt_cleanup_hw(struct drm_device *dev);
int i915_ppgtt_init_hw(struct drm_device *dev);
diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c
index 310756c30723..9a8cc8c51077 100644
--- a/drivers/gpu/drm/i915/i915_gem_stolen.c
+++ b/drivers/gpu/drm/i915/i915_gem_stolen.c
@@ -698,18 +698,17 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_device *dev,
*/
vma->node.start = gtt_offset;
vma->node.size = size;
- if (drm_mm_initialized(&ggtt->base.mm)) {
- ret = drm_mm_reserve_node(&ggtt->base.mm, &vma->node);
- if (ret) {
- DRM_DEBUG_KMS("failed to allocate stolen GTT space\n");
- goto err;
- }
- vma->bound |= GLOBAL_BIND;
- __i915_vma_set_map_and_fenceable(vma);
- list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
+ ret = drm_mm_reserve_node(&ggtt->base.mm, &vma->node);
+ if (ret) {
+ DRM_DEBUG_KMS("failed to allocate stolen GTT space\n");
+ goto err;
}
+ vma->bound |= GLOBAL_BIND;
+ __i915_vma_set_map_and_fenceable(vma);
+ list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
+
list_add_tail(&obj->global_list, &dev_priv->mm.bound_list);
i915_gem_object_pin_pages(obj);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 45/64] drm/i915: Store owning file on the i915_address_space
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (42 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 44/64] drm/i915: Split early global GTT initialisation Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 46/64] drm/i915: Count how many VMA are bound for an object Chris Wilson
` (19 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
For the global GTT (and aliasing GTT), the address space is owned by the
device (it is a global resource) and so the per-file owner field is
NULL. For per-process GTT (where we create an address space per
context), each is owned by the opening file. We can use this ownership
information to both distinguish GGTT and ppGTT address spaces, as well
as occasionally inspect the owner.
v2: Whitespace, tells us who owns i915_address_space
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 2 +-
drivers/gpu/drm/i915/i915_drv.h | 1 -
drivers/gpu/drm/i915/i915_gem_context.c | 3 ++-
drivers/gpu/drm/i915/i915_gem_gtt.c | 30 +++++++++++++++---------------
drivers/gpu/drm/i915/i915_gem_gtt.h | 17 +++++++++++------
5 files changed, 29 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index c7dd739eada5..327064823bb0 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -356,7 +356,7 @@ static int per_file_stats(int id, void *ptr, void *data)
= container_of(vma->vm,
struct i915_hw_ppgtt,
base);
- if (ppgtt->file_priv != stats->file_priv)
+ if (ppgtt->base.file != stats->file_priv)
continue;
}
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 66490046ab37..a19d18ea074e 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3280,7 +3280,6 @@ i915_vm_to_ppgtt(struct i915_address_space *vm)
return container_of(vm, struct i915_hw_ppgtt, base);
}
-
static inline bool i915_gem_obj_ggtt_bound(struct drm_i915_gem_object *obj)
{
return i915_gem_obj_ggtt_bound_view(obj, &i915_ggtt_view_normal);
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index ed071fb1b713..c213fafe41d9 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -327,7 +327,8 @@ i915_gem_create_context(struct drm_device *dev,
return ctx;
if (USES_FULL_PPGTT(dev)) {
- struct i915_hw_ppgtt *ppgtt = i915_ppgtt_create(dev, file_priv);
+ struct i915_hw_ppgtt *ppgtt =
+ i915_ppgtt_create(to_i915(dev), file_priv);
if (IS_ERR(ppgtt)) {
DRM_DEBUG_DRIVER("PPGTT setup failed (%ld)\n",
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 5b09db68df73..c25117bed1e9 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -2107,11 +2107,12 @@ static int gen6_ppgtt_init(struct i915_hw_ppgtt *ppgtt)
return 0;
}
-static int __hw_ppgtt_init(struct drm_device *dev, struct i915_hw_ppgtt *ppgtt)
+static int __hw_ppgtt_init(struct i915_hw_ppgtt *ppgtt,
+ struct drm_i915_private *dev_priv)
{
- ppgtt->base.dev = dev;
+ ppgtt->base.dev = &dev_priv->drm;
- if (INTEL_INFO(dev)->gen < 8)
+ if (INTEL_INFO(dev_priv)->gen < 8)
return gen6_ppgtt_init(ppgtt);
else
return gen8_ppgtt_init(ppgtt);
@@ -2146,15 +2147,17 @@ static void gtt_write_workarounds(struct drm_device *dev)
I915_WRITE(GEN8_L3_LRA_1_GPGPU, GEN9_L3_LRA_1_GPGPU_DEFAULT_VALUE_BXT);
}
-static int i915_ppgtt_init(struct drm_device *dev, struct i915_hw_ppgtt *ppgtt)
+static int i915_ppgtt_init(struct i915_hw_ppgtt *ppgtt,
+ struct drm_i915_private *dev_priv,
+ struct drm_i915_file_private *file_priv)
{
- struct drm_i915_private *dev_priv = to_i915(dev);
- int ret = 0;
+ int ret;
- ret = __hw_ppgtt_init(dev, ppgtt);
+ ret = __hw_ppgtt_init(ppgtt, dev_priv);
if (ret == 0) {
kref_init(&ppgtt->ref);
i915_address_space_init(&ppgtt->base, dev_priv);
+ ppgtt->base.file = file_priv;
}
return ret;
@@ -2186,7 +2189,8 @@ int i915_ppgtt_init_hw(struct drm_device *dev)
}
struct i915_hw_ppgtt *
-i915_ppgtt_create(struct drm_device *dev, struct drm_i915_file_private *fpriv)
+i915_ppgtt_create(struct drm_i915_private *dev_priv,
+ struct drm_i915_file_private *fpriv)
{
struct i915_hw_ppgtt *ppgtt;
int ret;
@@ -2195,14 +2199,12 @@ i915_ppgtt_create(struct drm_device *dev, struct drm_i915_file_private *fpriv)
if (!ppgtt)
return ERR_PTR(-ENOMEM);
- ret = i915_ppgtt_init(dev, ppgtt);
+ ret = i915_ppgtt_init(ppgtt, dev_priv, fpriv);
if (ret) {
kfree(ppgtt);
return ERR_PTR(ret);
}
- ppgtt->file_priv = fpriv;
-
trace_i915_ppgtt_create(&ppgtt->base);
return ppgtt;
@@ -2792,9 +2794,8 @@ int i915_gem_init_ggtt(struct drm_device *dev)
if (!ppgtt)
return -ENOMEM;
- ret = __hw_ppgtt_init(dev, ppgtt);
+ ret = __hw_ppgtt_init(ppgtt, dev_priv);
if (ret) {
- ppgtt->base.cleanup(&ppgtt->base);
kfree(ppgtt);
return ret;
}
@@ -3220,7 +3221,6 @@ int i915_ggtt_init_hw(struct drm_device *dev)
}
ggtt->base.dev = dev;
- ggtt->base.is_ggtt = true;
ret = ggtt->probe(ggtt);
if (ret)
@@ -3312,7 +3312,7 @@ void i915_gem_restore_gtt_mappings(struct drm_device *dev)
struct i915_hw_ppgtt *ppgtt;
- if (vm->is_ggtt)
+ if (i915_is_ggtt(vm))
ppgtt = dev_priv->mm.aliasing_ppgtt;
else
ppgtt = i915_vm_to_ppgtt(vm);
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index a0370bf950f5..33cc010e90f5 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -272,12 +272,19 @@ struct i915_pml4 {
struct i915_address_space {
struct drm_mm mm;
struct drm_device *dev;
+ /* Every address space belongs to a struct file - except for the global
+ * GTT that is owned by the driver (and so @file is set to NULL). In
+ * principle, no information should leak from one context to another
+ * (or between files/processes etc) unless explicitly shared by the
+ * owner. Tracking the owner is important in order to free up per-file
+ * objects along with the file, to aide resource tracking, and to
+ * assign blame.
+ */
+ struct drm_i915_file_private *file;
struct list_head global_link;
u64 start; /* Start offset always 0 for dri2 */
u64 total; /* size addr space maps (ex. 2GB for ggtt) */
- bool is_ggtt;
-
struct i915_page_scratch *scratch_page;
struct i915_page_table *scratch_pt;
struct i915_page_directory *scratch_pd;
@@ -338,7 +345,7 @@ struct i915_address_space {
u32 flags);
};
-#define i915_is_ggtt(V) ((V)->is_ggtt)
+#define i915_is_ggtt(V) (!(V)->file)
/* The Graphics Translation Table is the way in which GEN hardware translates a
* Graphics Virtual Address into a Physical Address. In addition to the normal
@@ -380,8 +387,6 @@ struct i915_hw_ppgtt {
struct i915_page_directory pd; /* GEN6-7 */
};
- struct drm_i915_file_private *file_priv;
-
gen6_pte_t __iomem *pd_addr;
int (*enable)(struct i915_hw_ppgtt *ppgtt);
@@ -528,7 +533,7 @@ void i915_ggtt_cleanup_hw(struct drm_device *dev);
int i915_ppgtt_init_hw(struct drm_device *dev);
void i915_ppgtt_release(struct kref *kref);
-struct i915_hw_ppgtt *i915_ppgtt_create(struct drm_device *dev,
+struct i915_hw_ppgtt *i915_ppgtt_create(struct drm_i915_private *dev_priv,
struct drm_i915_file_private *fpriv);
static inline void i915_ppgtt_get(struct i915_hw_ppgtt *ppgtt)
{
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 46/64] drm/i915: Count how many VMA are bound for an object
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (43 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 45/64] drm/i915: Store owning file on the i915_address_space Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-12 14:30 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 47/64] drm/i915: Be more careful when unbinding vma Chris Wilson
` (18 subsequent siblings)
63 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Since we may have VMA allocated for an object, but we interrupted their
binding, there is a disparity between have elements on the obj->vma_list
and being bound. i915_gem_obj_bound_any() does this check, but this is
not rigorously observed - add an explicit count to make it easier.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 12 +++++------
drivers/gpu/drm/i915/i915_drv.h | 3 ++-
drivers/gpu/drm/i915/i915_gem.c | 34 +++++++++++++-------------------
drivers/gpu/drm/i915/i915_gem_shrinker.c | 17 +---------------
drivers/gpu/drm/i915/i915_gem_stolen.c | 1 +
5 files changed, 23 insertions(+), 44 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 327064823bb0..d3852828878f 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -174,6 +174,9 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
if (obj->fence_reg != I915_FENCE_REG_NONE)
seq_printf(m, " (fence: %d)", obj->fence_reg);
list_for_each_entry(vma, &obj->vma_list, obj_link) {
+ if (!drm_mm_node_allocated(&vma->node))
+ continue;
+
seq_printf(m, " (%sgtt offset: %08llx, size: %08llx",
vma->is_ggtt ? "g" : "pp",
vma->node.start, vma->node.size);
@@ -335,11 +338,11 @@ static int per_file_stats(int id, void *ptr, void *data)
struct drm_i915_gem_object *obj = ptr;
struct file_stats *stats = data;
struct i915_vma *vma;
- int bound = 0;
stats->count++;
stats->total += obj->base.size;
-
+ if (!obj->bind_count)
+ stats->unbound += obj->base.size;
if (obj->base.name || obj->base.dma_buf)
stats->shared += obj->base.size;
@@ -347,8 +350,6 @@ static int per_file_stats(int id, void *ptr, void *data)
if (!drm_mm_node_allocated(&vma->node))
continue;
- bound++;
-
if (vma->is_ggtt) {
stats->global += vma->node.size;
} else {
@@ -366,9 +367,6 @@ static int per_file_stats(int id, void *ptr, void *data)
stats->inactive += vma->node.size;
}
- if (!bound)
- stats->unbound += obj->base.size;
-
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index a19d18ea074e..633585054669 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2217,6 +2217,8 @@ struct drm_i915_gem_object {
unsigned int frontbuffer_bits:INTEL_FRONTBUFFER_BITS;
unsigned int has_wc_mmap;
+ /** Count of VMA actually bound by this object */
+ unsigned int bind_count;
unsigned int pin_display;
struct sg_table *pages;
@@ -3246,7 +3248,6 @@ i915_gem_obj_ggtt_offset(struct drm_i915_gem_object *o)
return i915_gem_obj_ggtt_offset_view(o, &i915_ggtt_view_normal);
}
-bool i915_gem_obj_bound_any(struct drm_i915_gem_object *o);
bool i915_gem_obj_ggtt_bound_view(struct drm_i915_gem_object *o,
const struct i915_ggtt_view *view);
bool i915_gem_obj_bound(struct drm_i915_gem_object *o,
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 98f3945fc50f..c6816f9969d5 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2107,7 +2107,7 @@ i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
if (obj->pages_pin_count)
return -EBUSY;
- BUG_ON(i915_gem_obj_bound_any(obj));
+ GEM_BUG_ON(obj->bind_count);
/* ->put_pages might need to allocate memory for the bit17 swizzle
* array, hence protect them from being reaped by removing them from gtt
@@ -2957,7 +2957,6 @@ static void __i915_vma_iounmap(struct i915_vma *vma)
static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
{
struct drm_i915_gem_object *obj = vma->obj;
- struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
int ret;
if (list_empty(&vma->obj_link))
@@ -2971,7 +2970,8 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
if (vma->pin_count)
return -EBUSY;
- BUG_ON(obj->pages == NULL);
+ GEM_BUG_ON(obj->bind_count == 0);
+ GEM_BUG_ON(!obj->pages);
if (wait) {
ret = i915_gem_object_wait_rendering(obj, false);
@@ -3011,8 +3011,9 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
/* Since the unbound list is global, only move to that list if
* no more VMAs exist. */
- if (list_empty(&obj->vma_list))
- list_move_tail(&obj->global_list, &dev_priv->mm.unbound_list);
+ if (--obj->bind_count == 0)
+ list_move_tail(&obj->global_list,
+ &to_i915(obj->base.dev)->mm.unbound_list);
/* And finally now the object is completely decoupled from this vma,
* we can drop its hold on the backing storage and allow it to be
@@ -3247,6 +3248,7 @@ search_free:
list_move_tail(&obj->global_list, &dev_priv->mm.bound_list);
list_add_tail(&vma->vm_link, &vm->inactive_list);
+ obj->bind_count++;
return vma;
@@ -3442,7 +3444,6 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
{
struct drm_device *dev = obj->base.dev;
struct i915_vma *vma, *next;
- bool bound = false;
int ret = 0;
if (obj->cache_level == cache_level)
@@ -3466,8 +3467,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
ret = i915_vma_unbind(vma);
if (ret)
return ret;
- } else
- bound = true;
+ }
}
/* We can reuse the existing drm_mm nodes but need to change the
@@ -3477,7 +3477,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
* rewrite the PTE in the belief that doing so tramples upon less
* state and so involves less work.
*/
- if (bound) {
+ if (obj->bind_count) {
/* Before we change the PTE, the GPU must not be accessing it.
* If we wait upon the object, we know that all the bound
* VMA are no longer active.
@@ -3684,6 +3684,9 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
old_read_domains,
old_write_domain);
+ /* Increment the pages_pin_count to guard against the shrinker */
+ obj->pages_pin_count++;
+
return 0;
err_unpin_display:
@@ -3700,6 +3703,7 @@ i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj,
i915_gem_object_ggtt_unpin_view(obj, view);
+ obj->pages_pin_count--;
obj->pin_display--;
}
@@ -4215,6 +4219,7 @@ void i915_gem_free_object(struct drm_gem_object *gem_obj)
dev_priv->mm.interruptible = was_interruptible;
}
}
+ GEM_BUG_ON(obj->bind_count);
/* Stolen objects don't hold a ref, but do hold pin count. Fix that up
* before progressing. */
@@ -4851,17 +4856,6 @@ bool i915_gem_obj_ggtt_bound_view(struct drm_i915_gem_object *o,
return false;
}
-bool i915_gem_obj_bound_any(struct drm_i915_gem_object *o)
-{
- struct i915_vma *vma;
-
- list_for_each_entry(vma, &o->vma_list, obj_link)
- if (drm_mm_node_allocated(&vma->node))
- return true;
-
- return false;
-}
-
unsigned long i915_gem_obj_ggtt_size(struct drm_i915_gem_object *o)
{
struct i915_vma *vma;
diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
index e93a837cf2a1..725a8c894517 100644
--- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
@@ -48,21 +48,6 @@ static bool mutex_is_locked_by(struct mutex *mutex, struct task_struct *task)
#endif
}
-static int num_vma_bound(struct drm_i915_gem_object *obj)
-{
- struct i915_vma *vma;
- int count = 0;
-
- list_for_each_entry(vma, &obj->vma_list, obj_link) {
- if (drm_mm_node_allocated(&vma->node))
- count++;
- if (vma->pin_count)
- count++;
- }
-
- return count;
-}
-
static bool swap_available(void)
{
return get_nr_swap_pages() > 0;
@@ -82,7 +67,7 @@ static bool can_release_pages(struct drm_i915_gem_object *obj)
* to the GPU, simply unbinding from the GPU is not going to succeed
* in releasing our pin count on the pages themselves.
*/
- if (obj->pages_pin_count != num_vma_bound(obj))
+ if (obj->pages_pin_count != obj->bind_count)
return false;
/* We can only return physical pages to the system if we can either
diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c
index 9a8cc8c51077..2c321c8df7c3 100644
--- a/drivers/gpu/drm/i915/i915_gem_stolen.c
+++ b/drivers/gpu/drm/i915/i915_gem_stolen.c
@@ -708,6 +708,7 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_device *dev,
vma->bound |= GLOBAL_BIND;
__i915_vma_set_map_and_fenceable(vma);
list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
+ obj->bind_count++;
list_add_tail(&obj->global_list, &dev_priv->mm.bound_list);
i915_gem_object_pin_pages(obj);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 47/64] drm/i915: Be more careful when unbinding vma
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (44 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 46/64] drm/i915: Count how many VMA are bound for an object Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-12 15:04 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 48/64] drm/i915: Kill drop_pages() Chris Wilson
` (17 subsequent siblings)
63 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
When we call i915_vma_unbind(), we will wait upon outstanding rendering.
This will also trigger a retirement phase, which may update the object
lists. If, we extend request tracking to the VMA itself (rather than
keep it at the encompassing object), then there is a potential that the
obj->vma_list be modified for other elements upon i915_vma_unbind(). As
a result, if we walk over the object list and call i915_vma_unbind(), we
need to be prepared for that list to change.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 2 ++
drivers/gpu/drm/i915/i915_gem.c | 57 +++++++++++++++++++++++---------
drivers/gpu/drm/i915/i915_gem_shrinker.c | 7 +---
drivers/gpu/drm/i915/i915_gem_userptr.c | 4 +--
4 files changed, 46 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 633585054669..27e1182544a2 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3032,6 +3032,8 @@ int __must_check i915_vma_unbind(struct i915_vma *vma);
* _guarantee_ VMA in question is _not in use_ anywhere.
*/
int __must_check __i915_vma_unbind_no_wait(struct i915_vma *vma);
+
+int i915_gem_object_unbind(struct drm_i915_gem_object *obj);
int i915_gem_object_put_pages(struct drm_i915_gem_object *obj);
void i915_gem_release_all_mmaps(struct drm_i915_private *dev_priv);
void i915_gem_release_mmap(struct drm_i915_gem_object *obj);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index c6816f9969d5..28a3079a7892 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -283,18 +283,38 @@ static const struct drm_i915_gem_object_ops i915_gem_phys_ops = {
.release = i915_gem_object_release_phys,
};
+int
+i915_gem_object_unbind(struct drm_i915_gem_object *obj)
+{
+ struct i915_vma *vma;
+ LIST_HEAD(still_in_list);
+ int ret;
+
+ /* The vma will only be freed if it is marked as closed, and if we wait
+ * upon rendering to the vma, we may unbind anything in the list.
+ */
+ while ((vma = list_first_entry_or_null(&obj->vma_list,
+ struct i915_vma,
+ obj_link))) {
+ list_move_tail(&vma->obj_link, &still_in_list);
+ ret = i915_vma_unbind(vma);
+ if (ret)
+ break;
+ }
+ list_splice(&still_in_list, &obj->vma_list);
+
+ return ret;
+}
+
static int
drop_pages(struct drm_i915_gem_object *obj)
{
- struct i915_vma *vma, *next;
int ret;
i915_gem_object_get(obj);
- list_for_each_entry_safe(vma, next, &obj->vma_list, obj_link)
- if (i915_vma_unbind(vma))
- break;
-
- ret = i915_gem_object_put_pages(obj);
+ ret = i915_gem_object_unbind(obj);
+ if (ret == 0)
+ ret = i915_gem_object_put_pages(obj);
i915_gem_object_put(obj);
return ret;
@@ -3442,8 +3462,7 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
enum i915_cache_level cache_level)
{
- struct drm_device *dev = obj->base.dev;
- struct i915_vma *vma, *next;
+ struct i915_vma *vma;
int ret = 0;
if (obj->cache_level == cache_level)
@@ -3454,7 +3473,8 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
* catch the issue of the CS prefetch crossing page boundaries and
* reading an invalid PTE on older architectures.
*/
- list_for_each_entry_safe(vma, next, &obj->vma_list, obj_link) {
+restart:
+ list_for_each_entry(vma, &obj->vma_list, obj_link) {
if (!drm_mm_node_allocated(&vma->node))
continue;
@@ -3463,11 +3483,18 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
return -EBUSY;
}
- if (!i915_gem_valid_gtt_space(vma, cache_level)) {
- ret = i915_vma_unbind(vma);
- if (ret)
- return ret;
- }
+ if (i915_gem_valid_gtt_space(vma, cache_level))
+ continue;
+
+ ret = i915_vma_unbind(vma);
+ if (ret)
+ return ret;
+
+ /* As unbinding may affect other elements in the
+ * obj->vma_list (due to side-effects from retiring
+ * an active vma), play safe and restart the iterator.
+ */
+ goto restart;
}
/* We can reuse the existing drm_mm nodes but need to change the
@@ -3486,7 +3513,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
if (ret)
return ret;
- if (!HAS_LLC(dev) && cache_level != I915_CACHE_NONE) {
+ if (!HAS_LLC(obj->base.dev) && cache_level != I915_CACHE_NONE) {
/* Access to snoopable pages through the GTT is
* incoherent and on some machines causes a hard
* lockup. Relinquish the CPU mmaping to force
diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
index 725a8c894517..8ad95695f80c 100644
--- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
+++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
@@ -155,7 +155,6 @@ i915_gem_shrink(struct drm_i915_private *dev_priv,
INIT_LIST_HEAD(&still_in_list);
while (count < target && !list_empty(phase->list)) {
struct drm_i915_gem_object *obj;
- struct i915_vma *vma, *v;
obj = list_first_entry(phase->list,
typeof(*obj), global_list);
@@ -178,11 +177,7 @@ i915_gem_shrink(struct drm_i915_private *dev_priv,
i915_gem_object_get(obj);
/* For the unbound phase, this should be a no-op! */
- list_for_each_entry_safe(vma, v,
- &obj->vma_list, obj_link)
- if (i915_vma_unbind(vma))
- break;
-
+ i915_gem_object_unbind(obj);
if (i915_gem_object_put_pages(obj) == 0)
count += obj->base.size >> PAGE_SHIFT;
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index ca8b82ab93d6..e935b327f3f9 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -104,7 +104,6 @@ static void cancel_userptr(struct work_struct *work)
if (obj->pages != NULL) {
struct drm_i915_private *dev_priv = to_i915(dev);
- struct i915_vma *vma, *tmp;
bool was_interruptible;
wait_rendering(obj);
@@ -112,8 +111,7 @@ static void cancel_userptr(struct work_struct *work)
was_interruptible = dev_priv->mm.interruptible;
dev_priv->mm.interruptible = false;
- list_for_each_entry_safe(vma, tmp, &obj->vma_list, obj_link)
- WARN_ON(i915_vma_unbind(vma));
+ WARN_ON(i915_gem_object_unbind(obj));
WARN_ON(i915_gem_object_put_pages(obj));
dev_priv->mm.interruptible = was_interruptible;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 48/64] drm/i915: Kill drop_pages()
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (45 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 47/64] drm/i915: Be more careful when unbinding vma Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-12 15:14 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking Chris Wilson
` (16 subsequent siblings)
63 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
The drop_pages() function is a dangerous trap in that it can release the
passed in object pointer and so unless the caller is aware, it can
easily trick us into using the stale object afterwards. Move it into its
solitary callsite where we know it is safe.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem.c | 20 +++++---------------
1 file changed, 5 insertions(+), 15 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 28a3079a7892..b6497e9961ab 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -306,20 +306,6 @@ i915_gem_object_unbind(struct drm_i915_gem_object *obj)
return ret;
}
-static int
-drop_pages(struct drm_i915_gem_object *obj)
-{
- int ret;
-
- i915_gem_object_get(obj);
- ret = i915_gem_object_unbind(obj);
- if (ret == 0)
- ret = i915_gem_object_put_pages(obj);
- i915_gem_object_put(obj);
-
- return ret;
-}
-
int
i915_gem_object_attach_phys(struct drm_i915_gem_object *obj,
int align)
@@ -340,7 +326,11 @@ i915_gem_object_attach_phys(struct drm_i915_gem_object *obj,
if (obj->base.filp == NULL)
return -EINVAL;
- ret = drop_pages(obj);
+ ret = i915_gem_object_unbind(obj);
+ if (ret)
+ return ret;
+
+ ret = i915_gem_object_put_pages(obj);
if (ret)
return ret;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (46 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 48/64] drm/i915: Kill drop_pages() Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-12 16:05 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations Chris Wilson
` (15 subsequent siblings)
63 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
In the next patch, request tracking is made more generic and for that we
need a new expanded struct and to separate out the logic changes from
the mechanical churn, we split out the structure renaming into this
patch.
v2: Writer's block. Add some spiel about why we track requests.
v3: Now i915_gem_active.
v4: Now with i915_gem_active_set() for attaching to the active request.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 11 +++---
drivers/gpu/drm/i915/i915_drv.h | 9 +++--
drivers/gpu/drm/i915/i915_gem.c | 58 +++++++++++++++---------------
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 8 +++--
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 4 +--
drivers/gpu/drm/i915/i915_gem_fence.c | 6 ++--
drivers/gpu/drm/i915/i915_gem_request.h | 41 +++++++++++++++++++++
drivers/gpu/drm/i915/i915_gem_tiling.c | 2 +-
drivers/gpu/drm/i915/i915_gem_userptr.c | 2 +-
drivers/gpu/drm/i915/i915_gpu_error.c | 7 ++--
drivers/gpu/drm/i915/intel_display.c | 8 ++---
11 files changed, 98 insertions(+), 58 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index d3852828878f..dd832eace487 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -155,10 +155,10 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
obj->base.write_domain);
for_each_engine_id(engine, dev_priv, id)
seq_printf(m, "%x ",
- i915_gem_request_get_seqno(obj->last_read_req[id]));
+ i915_gem_request_get_seqno(obj->last_read[id].request));
seq_printf(m, "] %x %x%s%s%s",
- i915_gem_request_get_seqno(obj->last_write_req),
- i915_gem_request_get_seqno(obj->last_fenced_req),
+ i915_gem_request_get_seqno(obj->last_write.request),
+ i915_gem_request_get_seqno(obj->last_fence.request),
i915_cache_level_str(to_i915(obj->base.dev), obj->cache_level),
obj->dirty ? " dirty" : "",
obj->madv == I915_MADV_DONTNEED ? " purgeable" : "");
@@ -195,9 +195,8 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
*t = '\0';
seq_printf(m, " (%s mappable)", s);
}
- if (obj->last_write_req != NULL)
- seq_printf(m, " (%s)",
- i915_gem_request_get_engine(obj->last_write_req)->name);
+ if (obj->last_write.request)
+ seq_printf(m, " (%s)", obj->last_write.request->engine->name);
if (obj->frontbuffer_bits)
seq_printf(m, " (frontbuffer: 0x%03x)", obj->frontbuffer_bits);
}
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 27e1182544a2..5b096c65646c 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2238,11 +2238,10 @@ struct drm_i915_gem_object {
* requests on one ring where the write request is older than the
* read request. This allows for the CPU to read from an active
* buffer by only waiting for the write to complete.
- * */
- struct drm_i915_gem_request *last_read_req[I915_NUM_ENGINES];
- struct drm_i915_gem_request *last_write_req;
- /** Breadcrumb of last fenced GPU access to the buffer. */
- struct drm_i915_gem_request *last_fenced_req;
+ */
+ struct i915_gem_active last_read[I915_NUM_ENGINES];
+ struct i915_gem_active last_write;
+ struct i915_gem_active last_fence;
/** Current tiling stride for the object, if it's tiled. */
uint32_t stride;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index b6497e9961ab..5f302faf86e7 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1353,23 +1353,23 @@ i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
int ret, i;
if (readonly) {
- if (obj->last_write_req != NULL) {
- ret = i915_wait_request(obj->last_write_req);
+ if (obj->last_write.request) {
+ ret = i915_wait_request(obj->last_write.request);
if (ret)
return ret;
- i = obj->last_write_req->engine->id;
- if (obj->last_read_req[i] == obj->last_write_req)
+ i = obj->last_write.request->engine->id;
+ if (obj->last_read[i].request == obj->last_write.request)
i915_gem_object_retire__read(obj, i);
else
i915_gem_object_retire__write(obj);
}
} else {
for (i = 0; i < I915_NUM_ENGINES; i++) {
- if (obj->last_read_req[i] == NULL)
+ if (!obj->last_read[i].request)
continue;
- ret = i915_wait_request(obj->last_read_req[i]);
+ ret = i915_wait_request(obj->last_read[i].request);
if (ret)
return ret;
@@ -1397,9 +1397,9 @@ i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
{
int ring = req->engine->id;
- if (obj->last_read_req[ring] == req)
+ if (obj->last_read[ring].request == req)
i915_gem_object_retire__read(obj, ring);
- else if (obj->last_write_req == req)
+ else if (obj->last_write.request == req)
i915_gem_object_retire__write(obj);
if (!i915_reset_in_progress(&req->i915->gpu_error))
@@ -1428,7 +1428,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
if (readonly) {
struct drm_i915_gem_request *req;
- req = obj->last_write_req;
+ req = obj->last_write.request;
if (req == NULL)
return 0;
@@ -1437,7 +1437,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = obj->last_read_req[i];
+ req = obj->last_read[i].request;
if (req == NULL)
continue;
@@ -2375,7 +2375,7 @@ void i915_vma_move_to_active(struct i915_vma *vma,
obj->active |= intel_engine_flag(engine);
list_move_tail(&obj->engine_list[engine->id], &engine->active_list);
- i915_gem_request_assign(&obj->last_read_req[engine->id], req);
+ i915_gem_active_set(&obj->last_read[engine->id], req);
list_move_tail(&vma->vm_link, &vma->vm->active_list);
}
@@ -2383,10 +2383,10 @@ void i915_vma_move_to_active(struct i915_vma *vma,
static void
i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
{
- GEM_BUG_ON(obj->last_write_req == NULL);
- GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write_req->engine)));
+ GEM_BUG_ON(!obj->last_write.request);
+ GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
- i915_gem_request_assign(&obj->last_write_req, NULL);
+ i915_gem_request_assign(&obj->last_write.request, NULL);
intel_fb_obj_flush(obj, true, ORIGIN_CS);
}
@@ -2395,13 +2395,13 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
{
struct i915_vma *vma;
- GEM_BUG_ON(obj->last_read_req[ring] == NULL);
+ GEM_BUG_ON(!obj->last_read[ring].request);
GEM_BUG_ON(!(obj->active & (1 << ring)));
list_del_init(&obj->engine_list[ring]);
- i915_gem_request_assign(&obj->last_read_req[ring], NULL);
+ i915_gem_request_assign(&obj->last_read[ring].request, NULL);
- if (obj->last_write_req && obj->last_write_req->engine->id == ring)
+ if (obj->last_write.request && obj->last_write.request->engine->id == ring)
i915_gem_object_retire__write(obj);
obj->active &= ~(1 << ring);
@@ -2420,7 +2420,7 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
}
- i915_gem_request_assign(&obj->last_fenced_req, NULL);
+ i915_gem_request_assign(&obj->last_fence.request, NULL);
i915_gem_object_put(obj);
}
@@ -2618,7 +2618,7 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
struct drm_i915_gem_object,
engine_list[engine->id]);
- if (!list_empty(&obj->last_read_req[engine->id]->list))
+ if (!list_empty(&obj->last_read[engine->id].request->list))
break;
i915_gem_object_retire__read(obj, engine->id);
@@ -2746,7 +2746,7 @@ i915_gem_object_flush_active(struct drm_i915_gem_object *obj)
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = obj->last_read_req[i];
+ req = obj->last_read[i].request;
if (req == NULL)
continue;
@@ -2822,10 +2822,10 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
i915_gem_object_put(obj);
for (i = 0; i < I915_NUM_ENGINES; i++) {
- if (obj->last_read_req[i] == NULL)
+ if (!obj->last_read[i].request)
continue;
- req[n++] = i915_gem_request_get(obj->last_read_req[i]);
+ req[n++] = i915_gem_request_get(obj->last_read[i].request);
}
mutex_unlock(&dev->struct_mutex);
@@ -2916,12 +2916,12 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
n = 0;
if (readonly) {
- if (obj->last_write_req)
- req[n++] = obj->last_write_req;
+ if (obj->last_write.request)
+ req[n++] = obj->last_write.request;
} else {
for (i = 0; i < I915_NUM_ENGINES; i++)
- if (obj->last_read_req[i])
- req[n++] = obj->last_read_req[i];
+ if (obj->last_read[i].request)
+ req[n++] = obj->last_read[i].request;
}
for (i = 0; i < n; i++) {
ret = __i915_gem_object_sync(obj, to, req[i]);
@@ -4022,12 +4022,12 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = obj->last_read_req[i];
+ req = obj->last_read[i].request;
if (req)
args->busy |= 1 << (16 + req->engine->exec_id);
}
- if (obj->last_write_req)
- args->busy |= obj->last_write_req->engine->exec_id;
+ if (obj->last_write.request)
+ args->busy |= obj->last_write.request->engine->exec_id;
}
unref:
diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
index 85a5c86ba80a..aa767ca28532 100644
--- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
@@ -227,13 +227,15 @@ static void export_fences(struct drm_i915_gem_object *obj,
{
struct reservation_object *resv = dma_buf->resv;
struct drm_i915_gem_request *req;
+ unsigned long active;
int idx;
mutex_lock(&obj->base.dev->struct_mutex);
mutex_lock(&resv->lock.base);
- for (idx = 0; idx < ARRAY_SIZE(obj->last_read_req); idx++) {
- req = obj->last_read_req[idx];
+ active = obj->active;
+ for_each_active(active, idx) {
+ req = obj->last_read[idx].request;
if (!req)
continue;
@@ -241,7 +243,7 @@ static void export_fences(struct drm_i915_gem_object *obj,
reservation_object_add_shared_fence(resv, &req->fence);
}
- req = obj->last_write_req;
+ req = obj->last_write.request;
if (req)
reservation_object_add_excl_fence(resv, &req->fence);
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 5bfb2e2f3e67..22261a511d7d 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1143,7 +1143,7 @@ i915_gem_execbuffer_move_to_active(struct list_head *vmas,
i915_vma_move_to_active(vma, req);
eb_export_fence(obj, req);
if (obj->base.write_domain) {
- i915_gem_request_assign(&obj->last_write_req, req);
+ i915_gem_active_set(&obj->last_write, req);
intel_fb_obj_invalidate(obj, ORIGIN_CS);
@@ -1151,7 +1151,7 @@ i915_gem_execbuffer_move_to_active(struct list_head *vmas,
obj->base.write_domain &= ~I915_GEM_GPU_DOMAINS;
}
if (entry->flags & EXEC_OBJECT_NEEDS_FENCE) {
- i915_gem_request_assign(&obj->last_fenced_req, req);
+ i915_gem_active_set(&obj->last_fence, req);
if (entry->flags & __EXEC_OBJECT_HAS_FENCE) {
struct drm_i915_private *dev_priv = engine->i915;
list_move_tail(&dev_priv->fence_regs[obj->fence_reg].lru_list,
diff --git a/drivers/gpu/drm/i915/i915_gem_fence.c b/drivers/gpu/drm/i915/i915_gem_fence.c
index 251d7a95af89..9838046801bd 100644
--- a/drivers/gpu/drm/i915/i915_gem_fence.c
+++ b/drivers/gpu/drm/i915/i915_gem_fence.c
@@ -261,12 +261,12 @@ static inline void i915_gem_object_fence_lost(struct drm_i915_gem_object *obj)
static int
i915_gem_object_wait_fence(struct drm_i915_gem_object *obj)
{
- if (obj->last_fenced_req) {
- int ret = i915_wait_request(obj->last_fenced_req);
+ if (obj->last_fence.request) {
+ int ret = i915_wait_request(obj->last_fence.request);
if (ret)
return ret;
- i915_gem_request_assign(&obj->last_fenced_req, NULL);
+ i915_gem_request_assign(&obj->last_fence.request, NULL);
}
return 0;
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index 928fc8baf435..ff8c54fa955f 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -247,4 +247,45 @@ static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
__i915_spin_request(request, state, timeout_us));
}
+/* We treat requests as fences. This is not be to confused with our
+ * "fence registers" but pipeline synchronisation objects ala GL_ARB_sync.
+ * We use the fences to synchronize access from the CPU with activity on the
+ * GPU, for example, we should not rewrite an object's PTE whilst the GPU
+ * is reading them. We also track fences at a higher level to provide
+ * implicit synchronisation around GEM objects, e.g. set-domain will wait
+ * for outstanding GPU rendering before marking the object ready for CPU
+ * access, or a pageflip will wait until the GPU is complete before showing
+ * the frame on the scanout.
+ *
+ * In order to use a fence, the object must track the fence it needs to
+ * serialise with. For example, GEM objects want to track both read and
+ * write access so that we can perform concurrent read operations between
+ * the CPU and GPU engines, as well as waiting for all rendering to
+ * complete, or waiting for the last GPU user of a "fence register". The
+ * object then embeds a @i915_gem_active to track the most recent (in
+ * retirment order) request relevant for the desired mode of access.
+ * The @i915_gem_active is updated with i915_gem_request_mark_active() to
+ * track the most recent fence request, typically this is done as part of
+ * i915_vma_move_to_active().
+ *
+ * When the @i915_gem_active completes (is retired), it will
+ * signal its completion to the owner through a callback as well as mark
+ * itself as idle (i915_gem_active.request == NULL). The owner
+ * can then perform any action, such as delayed freeing of an active
+ * resource including itself.
+ */
+struct i915_gem_active {
+ struct drm_i915_gem_request *request;
+};
+
+static inline void
+i915_gem_active_set(struct i915_gem_active *active,
+ struct drm_i915_gem_request *request)
+{
+ i915_gem_request_assign(&active->request, request);
+}
+
+#define for_each_active(mask, idx) \
+ for (; mask ? idx = ffs(mask) - 1, 1 : 0; mask &= ~(1 << idx))
+
#endif /* I915_GEM_REQUEST_H */
diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
index e83fc2d0433c..00d796da65fb 100644
--- a/drivers/gpu/drm/i915/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
@@ -242,7 +242,7 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
}
obj->fence_dirty =
- obj->last_fenced_req ||
+ obj->last_fence.request ||
obj->fence_reg != I915_FENCE_REG_NONE;
obj->tiling_mode = args->tiling_mode;
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index e935b327f3f9..32f50a70ea42 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -74,7 +74,7 @@ static void wait_rendering(struct drm_i915_gem_object *obj)
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = obj->last_read_req[i];
+ req = obj->last_read[i].request;
if (req == NULL)
continue;
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 2fbe81d51af1..5e12b8ee49d2 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -749,8 +749,8 @@ static void capture_bo(struct drm_i915_error_buffer *err,
err->size = obj->base.size;
err->name = obj->base.name;
for (i = 0; i < I915_NUM_ENGINES; i++)
- err->rseqno[i] = i915_gem_request_get_seqno(obj->last_read_req[i]);
- err->wseqno = i915_gem_request_get_seqno(obj->last_write_req);
+ err->rseqno[i] = i915_gem_request_get_seqno(obj->last_read[i].request);
+ err->wseqno = i915_gem_request_get_seqno(obj->last_write.request);
err->gtt_offset = vma->node.start;
err->read_domains = obj->base.read_domains;
err->write_domain = obj->base.write_domain;
@@ -762,8 +762,7 @@ static void capture_bo(struct drm_i915_error_buffer *err,
err->dirty = obj->dirty;
err->purgeable = obj->madv != I915_MADV_WILLNEED;
err->userptr = obj->userptr.mm != NULL;
- err->ring = obj->last_write_req ?
- i915_gem_request_get_engine(obj->last_write_req)->id : -1;
+ err->ring = obj->last_write.request ? obj->last_write.request->engine->id : -1;
err->cache_level = obj->cache_level;
}
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 5f4bdc3d4972..27ac6db4e26a 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11465,7 +11465,7 @@ static bool use_mmio_flip(struct intel_engine_cs *engine,
if (resv && !reservation_object_test_signaled_rcu(resv, false))
return true;
- return engine != i915_gem_request_get_engine(obj->last_write_req);
+ return engine != i915_gem_request_get_engine(obj->last_write.request);
}
static void skl_do_mmio_flip(struct intel_crtc *intel_crtc,
@@ -11768,7 +11768,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
} else if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev)) {
engine = &dev_priv->engine[BCS];
} else if (INTEL_INFO(dev)->gen >= 7) {
- engine = i915_gem_request_get_engine(obj->last_write_req);
+ engine = i915_gem_request_get_engine(obj->last_write.request);
if (engine == NULL || engine->id != RCS)
engine = &dev_priv->engine[BCS];
} else {
@@ -11790,7 +11790,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
INIT_WORK(&work->mmio_work, intel_mmio_flip_work_func);
i915_gem_request_assign(&work->flip_queued_req,
- obj->last_write_req);
+ obj->last_write.request);
schedule_work(&work->mmio_work);
} else {
@@ -14096,7 +14096,7 @@ intel_prepare_plane_fb(struct drm_plane *plane,
to_intel_plane_state(new_state);
i915_gem_request_assign(&plane_state->wait_req,
- obj->last_write_req);
+ obj->last_write.request);
}
return ret;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (47 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-13 15:40 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 51/64] drm/i915: Mark up i915_gem_active for locking annotation Chris Wilson
` (14 subsequent siblings)
63 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
In the future, we will want to add annotations to the i915_gem_active
struct. The API is thus expanded to hide direct access to the contents
of i915_gem_active and mediated instead through a number of helpers.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 13 ++--
drivers/gpu/drm/i915/i915_gem.c | 91 +++++++++++++----------
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 4 +-
drivers/gpu/drm/i915/i915_gem_fence.c | 11 ++-
drivers/gpu/drm/i915/i915_gem_request.h | 128 +++++++++++++++++++++++++++++++-
drivers/gpu/drm/i915/i915_gem_tiling.c | 2 +-
drivers/gpu/drm/i915/i915_gem_userptr.c | 8 +-
drivers/gpu/drm/i915/i915_gpu_error.c | 9 ++-
drivers/gpu/drm/i915/intel_display.c | 15 ++--
9 files changed, 208 insertions(+), 73 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index dd832eace487..ae1c640fc1c8 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -155,10 +155,10 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
obj->base.write_domain);
for_each_engine_id(engine, dev_priv, id)
seq_printf(m, "%x ",
- i915_gem_request_get_seqno(obj->last_read[id].request));
+ i915_gem_active_get_seqno(&obj->last_read[id]));
seq_printf(m, "] %x %x%s%s%s",
- i915_gem_request_get_seqno(obj->last_write.request),
- i915_gem_request_get_seqno(obj->last_fence.request),
+ i915_gem_active_get_seqno(&obj->last_write),
+ i915_gem_active_get_seqno(&obj->last_fence),
i915_cache_level_str(to_i915(obj->base.dev), obj->cache_level),
obj->dirty ? " dirty" : "",
obj->madv == I915_MADV_DONTNEED ? " purgeable" : "");
@@ -195,8 +195,11 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
*t = '\0';
seq_printf(m, " (%s mappable)", s);
}
- if (obj->last_write.request)
- seq_printf(m, " (%s)", obj->last_write.request->engine->name);
+
+ engine = i915_gem_active_get_engine(&obj->last_write);
+ if (engine)
+ seq_printf(m, " (%s)", engine->name);
+
if (obj->frontbuffer_bits)
seq_printf(m, " (frontbuffer: 0x%03x)", obj->frontbuffer_bits);
}
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 5f302faf86e7..9c371e84b1bb 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1349,27 +1349,30 @@ int
i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
bool readonly)
{
+ struct drm_i915_gem_request *request;
struct reservation_object *resv;
int ret, i;
if (readonly) {
- if (obj->last_write.request) {
- ret = i915_wait_request(obj->last_write.request);
+ request = i915_gem_active_peek(&obj->last_write);
+ if (request) {
+ ret = i915_wait_request(request);
if (ret)
return ret;
- i = obj->last_write.request->engine->id;
- if (obj->last_read[i].request == obj->last_write.request)
+ i = request->engine->id;
+ if (i915_gem_active_peek(&obj->last_read[i]) == request)
i915_gem_object_retire__read(obj, i);
else
i915_gem_object_retire__write(obj);
}
} else {
for (i = 0; i < I915_NUM_ENGINES; i++) {
- if (!obj->last_read[i].request)
+ request = i915_gem_active_peek(&obj->last_read[i]);
+ if (!request)
continue;
- ret = i915_wait_request(obj->last_read[i].request);
+ ret = i915_wait_request(request);
if (ret)
return ret;
@@ -1397,9 +1400,9 @@ i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
{
int ring = req->engine->id;
- if (obj->last_read[ring].request == req)
+ if (i915_gem_active_peek(&obj->last_read[ring]) == req)
i915_gem_object_retire__read(obj, ring);
- else if (obj->last_write.request == req)
+ else if (i915_gem_active_peek(&obj->last_write) == req)
i915_gem_object_retire__write(obj);
if (!i915_reset_in_progress(&req->i915->gpu_error))
@@ -1428,20 +1431,20 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
if (readonly) {
struct drm_i915_gem_request *req;
- req = obj->last_write.request;
+ req = i915_gem_active_peek(&obj->last_write);
if (req == NULL)
return 0;
- requests[n++] = i915_gem_request_get(req);
+ requests[n++] = req;
} else {
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = obj->last_read[i].request;
+ req = i915_gem_active_peek(&obj->last_read[i]);
if (req == NULL)
continue;
- requests[n++] = i915_gem_request_get(req);
+ requests[n++] = req;
}
}
@@ -2383,25 +2386,27 @@ void i915_vma_move_to_active(struct i915_vma *vma,
static void
i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
{
- GEM_BUG_ON(!obj->last_write.request);
- GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
+ GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_write));
+ GEM_BUG_ON(!(obj->active & intel_engine_flag(i915_gem_active_get_engine(&obj->last_write))));
- i915_gem_request_assign(&obj->last_write.request, NULL);
+ i915_gem_active_set(&obj->last_write, NULL);
intel_fb_obj_flush(obj, true, ORIGIN_CS);
}
static void
i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
{
+ struct intel_engine_cs *engine;
struct i915_vma *vma;
- GEM_BUG_ON(!obj->last_read[ring].request);
+ GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_read[ring]));
GEM_BUG_ON(!(obj->active & (1 << ring)));
list_del_init(&obj->engine_list[ring]);
- i915_gem_request_assign(&obj->last_read[ring].request, NULL);
+ i915_gem_active_set(&obj->last_read[ring], NULL);
- if (obj->last_write.request && obj->last_write.request->engine->id == ring)
+ engine = i915_gem_active_get_engine(&obj->last_write);
+ if (engine && engine->id == ring)
i915_gem_object_retire__write(obj);
obj->active &= ~(1 << ring);
@@ -2420,7 +2425,7 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
}
- i915_gem_request_assign(&obj->last_fence.request, NULL);
+ i915_gem_active_set(&obj->last_fence, NULL);
i915_gem_object_put(obj);
}
@@ -2618,7 +2623,7 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
struct drm_i915_gem_object,
engine_list[engine->id]);
- if (!list_empty(&obj->last_read[engine->id].request->list))
+ if (!list_empty(&i915_gem_active_peek(&obj->last_read[engine->id])->list))
break;
i915_gem_object_retire__read(obj, engine->id);
@@ -2746,7 +2751,7 @@ i915_gem_object_flush_active(struct drm_i915_gem_object *obj)
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = obj->last_read[i].request;
+ req = i915_gem_active_peek(&obj->last_read[i]);
if (req == NULL)
continue;
@@ -2786,7 +2791,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
{
struct drm_i915_gem_wait *args = data;
struct drm_i915_gem_object *obj;
- struct drm_i915_gem_request *req[I915_NUM_ENGINES];
+ struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
int i, n = 0;
int ret;
@@ -2822,20 +2827,21 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
i915_gem_object_put(obj);
for (i = 0; i < I915_NUM_ENGINES; i++) {
- if (!obj->last_read[i].request)
- continue;
+ struct drm_i915_gem_request *req;
- req[n++] = i915_gem_request_get(obj->last_read[i].request);
+ req = i915_gem_active_get(&obj->last_read[i]);
+ if (req)
+ requests[n++] = req;
}
mutex_unlock(&dev->struct_mutex);
for (i = 0; i < n; i++) {
if (ret == 0)
- ret = __i915_wait_request(req[i], true,
+ ret = __i915_wait_request(requests[i], true,
args->timeout_ns > 0 ? &args->timeout_ns : NULL,
to_rps_client(file));
- i915_gem_request_put(req[i]);
+ i915_gem_request_put(requests[i]);
}
return ret;
@@ -2908,7 +2914,7 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct drm_i915_gem_request *to)
{
const bool readonly = obj->base.pending_write_domain == 0;
- struct drm_i915_gem_request *req[I915_NUM_ENGINES];
+ struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
int ret, i, n;
if (!obj->active)
@@ -2916,15 +2922,22 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
n = 0;
if (readonly) {
- if (obj->last_write.request)
- req[n++] = obj->last_write.request;
+ struct drm_i915_gem_request *req;
+
+ req = i915_gem_active_peek(&obj->last_write);
+ if (req)
+ requests[n++] = req;
} else {
- for (i = 0; i < I915_NUM_ENGINES; i++)
- if (obj->last_read[i].request)
- req[n++] = obj->last_read[i].request;
+ for (i = 0; i < I915_NUM_ENGINES; i++) {
+ struct drm_i915_gem_request *req;
+
+ req = i915_gem_active_peek(&obj->last_read[i]);
+ if (req)
+ requests[n++] = req;
+ }
}
for (i = 0; i < n; i++) {
- ret = __i915_gem_object_sync(obj, to, req[i]);
+ ret = __i915_gem_object_sync(obj, to, requests[i]);
if (ret)
return ret;
}
@@ -4017,17 +4030,17 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
args->busy = 0;
if (obj->active) {
+ struct drm_i915_gem_request *req;
int i;
for (i = 0; i < I915_NUM_ENGINES; i++) {
- struct drm_i915_gem_request *req;
-
- req = obj->last_read[i].request;
+ req = i915_gem_active_peek(&obj->last_read[i]);
if (req)
args->busy |= 1 << (16 + req->engine->exec_id);
}
- if (obj->last_write.request)
- args->busy |= obj->last_write.request->engine->exec_id;
+ req = i915_gem_active_peek(&obj->last_write);
+ if (req)
+ args->busy |= req->engine->exec_id;
}
unref:
diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
index aa767ca28532..38000c59d456 100644
--- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
@@ -235,7 +235,7 @@ static void export_fences(struct drm_i915_gem_object *obj,
active = obj->active;
for_each_active(active, idx) {
- req = obj->last_read[idx].request;
+ req = i915_gem_active_peek(&obj->last_read[idx]);
if (!req)
continue;
@@ -243,7 +243,7 @@ static void export_fences(struct drm_i915_gem_object *obj,
reservation_object_add_shared_fence(resv, &req->fence);
}
- req = obj->last_write.request;
+ req = i915_gem_active_peek(&obj->last_write);
if (req)
reservation_object_add_excl_fence(resv, &req->fence);
diff --git a/drivers/gpu/drm/i915/i915_gem_fence.c b/drivers/gpu/drm/i915/i915_gem_fence.c
index 9838046801bd..9fdbd66128a6 100644
--- a/drivers/gpu/drm/i915/i915_gem_fence.c
+++ b/drivers/gpu/drm/i915/i915_gem_fence.c
@@ -261,14 +261,13 @@ static inline void i915_gem_object_fence_lost(struct drm_i915_gem_object *obj)
static int
i915_gem_object_wait_fence(struct drm_i915_gem_object *obj)
{
- if (obj->last_fence.request) {
- int ret = i915_wait_request(obj->last_fence.request);
- if (ret)
- return ret;
+ int ret;
- i915_gem_request_assign(&obj->last_fence.request, NULL);
- }
+ ret = i915_gem_active_wait(&obj->last_fence);
+ if (ret)
+ return ret;
+ i915_gem_active_set(&obj->last_fence, NULL);
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index ff8c54fa955f..46d9b00a28d7 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -275,14 +275,138 @@ static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
* resource including itself.
*/
struct i915_gem_active {
- struct drm_i915_gem_request *request;
+ struct drm_i915_gem_request *__request;
};
+/**
+ * i915_gem_active_set - updates the tracker to watch the current request
+ * @active - the active tracker
+ * @request - the request to watch
+ *
+ * i915_gem_active_set() watches the given @request for completion. Whilst
+ * that @request is busy, the @active reports busy. When that @request is
+ * retired, the @active tracker is updated to report idle.
+ */
static inline void
i915_gem_active_set(struct i915_gem_active *active,
struct drm_i915_gem_request *request)
{
- i915_gem_request_assign(&active->request, request);
+ i915_gem_request_assign(&active->__request, request);
+}
+
+/**
+ * i915_gem_active_peek - report the request being monitored
+ * @active - the active tracker
+ *
+ * i915_gem_active_peek() returns the current request being tracked, or NULL.
+ * It does not obtain a reference on the request for the caller, so the
+ * caller must hold struct_mutex.
+ */
+static inline struct drm_i915_gem_request *
+i915_gem_active_peek(const struct i915_gem_active *active)
+{
+ return active->__request;
+}
+
+/**
+ * i915_gem_active_get - return a reference to the active request
+ * @active - the active tracker
+ *
+ * i915_gem_active_get() returns a reference to the active request, or NULL
+ * if the active tracker is idle. The caller must hold struct_mutex.
+ */
+static inline struct drm_i915_gem_request *
+i915_gem_active_get(const struct i915_gem_active *active)
+{
+ struct drm_i915_gem_request *request;
+
+ request = i915_gem_active_peek(active);
+ if (!request || i915_gem_request_completed(request))
+ return NULL;
+
+ return i915_gem_request_get(request);
+}
+
+/**
+ * __i915_gem_active_is_busy - report whether the active tracker is assigned
+ * @active - the active tracker
+ *
+ * __i915_gem_active_is_busy() returns true if the active tracker is currently
+ * assigned to a request. Due to the lazy retiring, that request may be idle
+ * and this may report stale information.
+ */
+static inline bool
+__i915_gem_active_is_busy(const struct i915_gem_active *active)
+{
+ return i915_gem_active_peek(active);
+}
+
+/**
+ * i915_gem_active_is_idle - report whether the active tracker is idle
+ * @active - the active tracker
+ *
+ * i915_gem_active_is_idle() returns true if the active tracker is currently
+ * unassigned or if the request is complete (but not yet retired). Requires
+ * the caller to hold struct_mutex (but that can be relaxed if desired).
+ */
+static inline bool
+i915_gem_active_is_idle(const struct i915_gem_active *active)
+{
+ struct drm_i915_gem_request *request;
+
+ request = i915_gem_active_peek(active);
+ if (!request || i915_gem_request_completed(request))
+ return true;
+
+ return false;
+}
+
+/**
+ * i915_gem_active_wait - waits until the request is completed
+ * @active - the active request on which to wait
+ *
+ * i915_gem_active_wait() waits until the request is completed before
+ * returning.
+ */
+static inline int __must_check
+i915_gem_active_wait(const struct i915_gem_active *active)
+{
+ struct drm_i915_gem_request *request;
+
+ request = i915_gem_active_peek(active);
+ if (!request)
+ return 0;
+
+ return i915_wait_request(request);
+}
+
+/**
+ * i915_gem_active_retire - waits until the request is retired
+ * @active - the active request on which to wait
+ *
+ * Unlike i915_gem_active_eait(), this i915_gem_active_retire() will
+ * make sure the request is retired before returning.
+ */
+static inline int __must_check
+i915_gem_active_retire(const struct i915_gem_active *active)
+{
+ return i915_gem_active_wait(active);
+}
+
+/* Convenience functions for peeking at state inside active's request whilst
+ * guarded by the struct_mutex.
+ */
+
+static inline uint32_t
+i915_gem_active_get_seqno(const struct i915_gem_active *active)
+{
+ return i915_gem_request_get_seqno(i915_gem_active_peek(active));
+}
+
+static inline struct intel_engine_cs *
+i915_gem_active_get_engine(const struct i915_gem_active *active)
+{
+ return i915_gem_request_get_engine(i915_gem_active_peek(active));
}
#define for_each_active(mask, idx) \
diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
index 00d796da65fb..8cef2d6b291a 100644
--- a/drivers/gpu/drm/i915/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
@@ -242,7 +242,7 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
}
obj->fence_dirty =
- obj->last_fence.request ||
+ !i915_gem_active_is_idle(&obj->last_fence) ||
obj->fence_reg != I915_FENCE_REG_NONE;
obj->tiling_mode = args->tiling_mode;
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index 32f50a70ea42..00ab5e9d2eb7 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -74,11 +74,9 @@ static void wait_rendering(struct drm_i915_gem_object *obj)
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = obj->last_read[i].request;
- if (req == NULL)
- continue;
-
- requests[n++] = i915_gem_request_get(req);
+ req = i915_gem_active_get(&obj->last_read[i]);
+ if (req)
+ requests[n++] = req;
}
mutex_unlock(&dev->struct_mutex);
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 5e12b8ee49d2..9e1949f2f4dd 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -744,13 +744,14 @@ static void capture_bo(struct drm_i915_error_buffer *err,
struct i915_vma *vma)
{
struct drm_i915_gem_object *obj = vma->obj;
+ struct intel_engine_cs *engine;
int i;
err->size = obj->base.size;
err->name = obj->base.name;
for (i = 0; i < I915_NUM_ENGINES; i++)
- err->rseqno[i] = i915_gem_request_get_seqno(obj->last_read[i].request);
- err->wseqno = i915_gem_request_get_seqno(obj->last_write.request);
+ err->rseqno[i] = i915_gem_active_get_seqno(&obj->last_read[i]);
+ err->wseqno = i915_gem_active_get_seqno(&obj->last_write);
err->gtt_offset = vma->node.start;
err->read_domains = obj->base.read_domains;
err->write_domain = obj->base.write_domain;
@@ -762,8 +763,10 @@ static void capture_bo(struct drm_i915_error_buffer *err,
err->dirty = obj->dirty;
err->purgeable = obj->madv != I915_MADV_WILLNEED;
err->userptr = obj->userptr.mm != NULL;
- err->ring = obj->last_write.request ? obj->last_write.request->engine->id : -1;
err->cache_level = obj->cache_level;
+
+ engine = i915_gem_active_get_engine(&obj->last_write);
+ err->ring = engine ? engine->id : -1;
}
static u32 capture_active_bo(struct drm_i915_error_buffer *err,
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index 27ac6db4e26a..a0abe5588d17 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11465,7 +11465,7 @@ static bool use_mmio_flip(struct intel_engine_cs *engine,
if (resv && !reservation_object_test_signaled_rcu(resv, false))
return true;
- return engine != i915_gem_request_get_engine(obj->last_write.request);
+ return engine != i915_gem_active_get_engine(&obj->last_write);
}
static void skl_do_mmio_flip(struct intel_crtc *intel_crtc,
@@ -11768,7 +11768,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
} else if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev)) {
engine = &dev_priv->engine[BCS];
} else if (INTEL_INFO(dev)->gen >= 7) {
- engine = i915_gem_request_get_engine(obj->last_write.request);
+ engine = i915_gem_active_get_engine(&obj->last_write);
if (engine == NULL || engine->id != RCS)
engine = &dev_priv->engine[BCS];
} else {
@@ -11789,9 +11789,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
if (mmio_flip) {
INIT_WORK(&work->mmio_work, intel_mmio_flip_work_func);
- i915_gem_request_assign(&work->flip_queued_req,
- obj->last_write.request);
-
+ work->flip_queued_req = i915_gem_active_get(&obj->last_write);
schedule_work(&work->mmio_work);
} else {
request = i915_gem_request_alloc(engine, engine->last_context);
@@ -14092,11 +14090,8 @@ intel_prepare_plane_fb(struct drm_plane *plane,
}
if (ret == 0) {
- struct intel_plane_state *plane_state =
- to_intel_plane_state(new_state);
-
- i915_gem_request_assign(&plane_state->wait_req,
- obj->last_write.request);
+ to_intel_plane_state(new_state)->wait_req =
+ i915_gem_active_get(&obj->last_write);
}
return ret;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 51/64] drm/i915: Mark up i915_gem_active for locking annotation
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (48 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 52/64] drm/i915: Refactor blocking waits Chris Wilson
` (13 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
The future annotations will track the locking used for access to ensure
that it is always sufficient. We make the preparations now to present
the API ahead and to make sure that GCC can eliminate the unused
parameter.
Before: 6298417 3619610 696320 10614347 a1f64b vmlinux
After: 6298417 3619610 696320 10614347 a1f64b vmlinux
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 12 +++++---
drivers/gpu/drm/i915/i915_gem.c | 49 ++++++++++++++++++++++-----------
drivers/gpu/drm/i915/i915_gem_dmabuf.c | 6 ++--
drivers/gpu/drm/i915/i915_gem_fence.c | 3 +-
drivers/gpu/drm/i915/i915_gem_request.h | 38 +++++++++++++++----------
drivers/gpu/drm/i915/i915_gem_tiling.c | 3 +-
drivers/gpu/drm/i915/i915_gem_userptr.c | 3 +-
drivers/gpu/drm/i915/i915_gpu_error.c | 29 +++++++++++++++----
drivers/gpu/drm/i915/intel_display.c | 12 +++++---
9 files changed, 106 insertions(+), 49 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index ae1c640fc1c8..b260d241f955 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -155,10 +155,13 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
obj->base.write_domain);
for_each_engine_id(engine, dev_priv, id)
seq_printf(m, "%x ",
- i915_gem_active_get_seqno(&obj->last_read[id]));
+ i915_gem_active_get_seqno(&obj->last_read[id],
+ &obj->base.dev->struct_mutex));
seq_printf(m, "] %x %x%s%s%s",
- i915_gem_active_get_seqno(&obj->last_write),
- i915_gem_active_get_seqno(&obj->last_fence),
+ i915_gem_active_get_seqno(&obj->last_write,
+ &obj->base.dev->struct_mutex),
+ i915_gem_active_get_seqno(&obj->last_fence,
+ &obj->base.dev->struct_mutex),
i915_cache_level_str(to_i915(obj->base.dev), obj->cache_level),
obj->dirty ? " dirty" : "",
obj->madv == I915_MADV_DONTNEED ? " purgeable" : "");
@@ -196,7 +199,8 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
seq_printf(m, " (%s mappable)", s);
}
- engine = i915_gem_active_get_engine(&obj->last_write);
+ engine = i915_gem_active_get_engine(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (engine)
seq_printf(m, " (%s)", engine->name);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 9c371e84b1bb..54aca084b80a 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1354,21 +1354,24 @@ i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
int ret, i;
if (readonly) {
- request = i915_gem_active_peek(&obj->last_write);
+ request = i915_gem_active_peek(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (request) {
ret = i915_wait_request(request);
if (ret)
return ret;
i = request->engine->id;
- if (i915_gem_active_peek(&obj->last_read[i]) == request)
+ if (i915_gem_active_peek(&obj->last_read[i],
+ &obj->base.dev->struct_mutex) == request)
i915_gem_object_retire__read(obj, i);
else
i915_gem_object_retire__write(obj);
}
} else {
for (i = 0; i < I915_NUM_ENGINES; i++) {
- request = i915_gem_active_peek(&obj->last_read[i]);
+ request = i915_gem_active_peek(&obj->last_read[i],
+ &obj->base.dev->struct_mutex);
if (!request)
continue;
@@ -1400,9 +1403,11 @@ i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
{
int ring = req->engine->id;
- if (i915_gem_active_peek(&obj->last_read[ring]) == req)
+ if (i915_gem_active_peek(&obj->last_read[ring],
+ &obj->base.dev->struct_mutex) == req)
i915_gem_object_retire__read(obj, ring);
- else if (i915_gem_active_peek(&obj->last_write) == req)
+ else if (i915_gem_active_peek(&obj->last_write,
+ &obj->base.dev->struct_mutex) == req)
i915_gem_object_retire__write(obj);
if (!i915_reset_in_progress(&req->i915->gpu_error))
@@ -1431,7 +1436,8 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
if (readonly) {
struct drm_i915_gem_request *req;
- req = i915_gem_active_peek(&obj->last_write);
+ req = i915_gem_active_peek(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (req == NULL)
return 0;
@@ -1440,7 +1446,8 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = i915_gem_active_peek(&obj->last_read[i]);
+ req = i915_gem_active_peek(&obj->last_read[i],
+ &obj->base.dev->struct_mutex);
if (req == NULL)
continue;
@@ -2387,7 +2394,9 @@ static void
i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
{
GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_write));
- GEM_BUG_ON(!(obj->active & intel_engine_flag(i915_gem_active_get_engine(&obj->last_write))));
+ GEM_BUG_ON(!(obj->active &
+ intel_engine_flag(i915_gem_active_get_engine(&obj->last_write,
+ &obj->base.dev->struct_mutex))));
i915_gem_active_set(&obj->last_write, NULL);
intel_fb_obj_flush(obj, true, ORIGIN_CS);
@@ -2405,7 +2414,8 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
list_del_init(&obj->engine_list[ring]);
i915_gem_active_set(&obj->last_read[ring], NULL);
- engine = i915_gem_active_get_engine(&obj->last_write);
+ engine = i915_gem_active_get_engine(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (engine && engine->id == ring)
i915_gem_object_retire__write(obj);
@@ -2623,7 +2633,8 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
struct drm_i915_gem_object,
engine_list[engine->id]);
- if (!list_empty(&i915_gem_active_peek(&obj->last_read[engine->id])->list))
+ if (!list_empty(&i915_gem_active_peek(&obj->last_read[engine->id],
+ &obj->base.dev->struct_mutex)->list))
break;
i915_gem_object_retire__read(obj, engine->id);
@@ -2751,7 +2762,8 @@ i915_gem_object_flush_active(struct drm_i915_gem_object *obj)
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = i915_gem_active_peek(&obj->last_read[i]);
+ req = i915_gem_active_peek(&obj->last_read[i],
+ &obj->base.dev->struct_mutex);
if (req == NULL)
continue;
@@ -2829,7 +2841,8 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = i915_gem_active_get(&obj->last_read[i]);
+ req = i915_gem_active_get(&obj->last_read[i],
+ &obj->base.dev->struct_mutex);
if (req)
requests[n++] = req;
}
@@ -2924,14 +2937,16 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (readonly) {
struct drm_i915_gem_request *req;
- req = i915_gem_active_peek(&obj->last_write);
+ req = i915_gem_active_peek(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (req)
requests[n++] = req;
} else {
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = i915_gem_active_peek(&obj->last_read[i]);
+ req = i915_gem_active_peek(&obj->last_read[i],
+ &obj->base.dev->struct_mutex);
if (req)
requests[n++] = req;
}
@@ -4034,11 +4049,13 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
int i;
for (i = 0; i < I915_NUM_ENGINES; i++) {
- req = i915_gem_active_peek(&obj->last_read[i]);
+ req = i915_gem_active_peek(&obj->last_read[i],
+ &obj->base.dev->struct_mutex);
if (req)
args->busy |= 1 << (16 + req->engine->exec_id);
}
- req = i915_gem_active_peek(&obj->last_write);
+ req = i915_gem_active_peek(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (req)
args->busy |= req->engine->exec_id;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
index 38000c59d456..b2b638335d8e 100644
--- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
+++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
@@ -235,7 +235,8 @@ static void export_fences(struct drm_i915_gem_object *obj,
active = obj->active;
for_each_active(active, idx) {
- req = i915_gem_active_peek(&obj->last_read[idx]);
+ req = i915_gem_active_peek(&obj->last_read[idx],
+ &obj->base.dev->struct_mutex);
if (!req)
continue;
@@ -243,7 +244,8 @@ static void export_fences(struct drm_i915_gem_object *obj,
reservation_object_add_shared_fence(resv, &req->fence);
}
- req = i915_gem_active_peek(&obj->last_write);
+ req = i915_gem_active_peek(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (req)
reservation_object_add_excl_fence(resv, &req->fence);
diff --git a/drivers/gpu/drm/i915/i915_gem_fence.c b/drivers/gpu/drm/i915/i915_gem_fence.c
index 9fdbd66128a6..a4ec4fecbea0 100644
--- a/drivers/gpu/drm/i915/i915_gem_fence.c
+++ b/drivers/gpu/drm/i915/i915_gem_fence.c
@@ -263,7 +263,8 @@ i915_gem_object_wait_fence(struct drm_i915_gem_object *obj)
{
int ret;
- ret = i915_gem_active_wait(&obj->last_fence);
+ ret = i915_gem_active_wait(&obj->last_fence,
+ &obj->base.dev->struct_mutex);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index 46d9b00a28d7..34e65e07cad5 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -294,6 +294,12 @@ i915_gem_active_set(struct i915_gem_active *active,
i915_gem_request_assign(&active->__request, request);
}
+static inline struct drm_i915_gem_request *
+__i915_gem_active_peek(const struct i915_gem_active *active)
+{
+ return active->__request;
+}
+
/**
* i915_gem_active_peek - report the request being monitored
* @active - the active tracker
@@ -303,7 +309,7 @@ i915_gem_active_set(struct i915_gem_active *active,
* caller must hold struct_mutex.
*/
static inline struct drm_i915_gem_request *
-i915_gem_active_peek(const struct i915_gem_active *active)
+i915_gem_active_peek(const struct i915_gem_active *active, struct mutex *mutex)
{
return active->__request;
}
@@ -316,11 +322,11 @@ i915_gem_active_peek(const struct i915_gem_active *active)
* if the active tracker is idle. The caller must hold struct_mutex.
*/
static inline struct drm_i915_gem_request *
-i915_gem_active_get(const struct i915_gem_active *active)
+i915_gem_active_get(const struct i915_gem_active *active, struct mutex *mutex)
{
struct drm_i915_gem_request *request;
- request = i915_gem_active_peek(active);
+ request = i915_gem_active_peek(active, mutex);
if (!request || i915_gem_request_completed(request))
return NULL;
@@ -338,7 +344,7 @@ i915_gem_active_get(const struct i915_gem_active *active)
static inline bool
__i915_gem_active_is_busy(const struct i915_gem_active *active)
{
- return i915_gem_active_peek(active);
+ return __i915_gem_active_peek(active);
}
/**
@@ -350,11 +356,12 @@ __i915_gem_active_is_busy(const struct i915_gem_active *active)
* the caller to hold struct_mutex (but that can be relaxed if desired).
*/
static inline bool
-i915_gem_active_is_idle(const struct i915_gem_active *active)
+i915_gem_active_is_idle(const struct i915_gem_active *active,
+ struct mutex *mutex)
{
struct drm_i915_gem_request *request;
- request = i915_gem_active_peek(active);
+ request = i915_gem_active_peek(active, mutex);
if (!request || i915_gem_request_completed(request))
return true;
@@ -369,11 +376,11 @@ i915_gem_active_is_idle(const struct i915_gem_active *active)
* returning.
*/
static inline int __must_check
-i915_gem_active_wait(const struct i915_gem_active *active)
+i915_gem_active_wait(const struct i915_gem_active *active, struct mutex *mutex)
{
struct drm_i915_gem_request *request;
- request = i915_gem_active_peek(active);
+ request = i915_gem_active_peek(active, mutex);
if (!request)
return 0;
@@ -388,9 +395,10 @@ i915_gem_active_wait(const struct i915_gem_active *active)
* make sure the request is retired before returning.
*/
static inline int __must_check
-i915_gem_active_retire(const struct i915_gem_active *active)
+i915_gem_active_retire(const struct i915_gem_active *active,
+ struct mutex *mutex)
{
- return i915_gem_active_wait(active);
+ return i915_gem_active_wait(active, mutex);
}
/* Convenience functions for peeking at state inside active's request whilst
@@ -398,15 +406,17 @@ i915_gem_active_retire(const struct i915_gem_active *active)
*/
static inline uint32_t
-i915_gem_active_get_seqno(const struct i915_gem_active *active)
+i915_gem_active_get_seqno(const struct i915_gem_active *active,
+ struct mutex *mutex)
{
- return i915_gem_request_get_seqno(i915_gem_active_peek(active));
+ return i915_gem_request_get_seqno(i915_gem_active_peek(active, mutex));
}
static inline struct intel_engine_cs *
-i915_gem_active_get_engine(const struct i915_gem_active *active)
+i915_gem_active_get_engine(const struct i915_gem_active *active,
+ struct mutex *mutex)
{
- return i915_gem_request_get_engine(i915_gem_active_peek(active));
+ return i915_gem_request_get_engine(i915_gem_active_peek(active, mutex));
}
#define for_each_active(mask, idx) \
diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
index 8cef2d6b291a..fa2eb4a4f212 100644
--- a/drivers/gpu/drm/i915/i915_gem_tiling.c
+++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
@@ -242,7 +242,8 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
}
obj->fence_dirty =
- !i915_gem_active_is_idle(&obj->last_fence) ||
+ !i915_gem_active_is_idle(&obj->last_fence,
+ &dev->struct_mutex) ||
obj->fence_reg != I915_FENCE_REG_NONE;
obj->tiling_mode = args->tiling_mode;
diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
index 00ab5e9d2eb7..e57521dbddc6 100644
--- a/drivers/gpu/drm/i915/i915_gem_userptr.c
+++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
@@ -74,7 +74,8 @@ static void wait_rendering(struct drm_i915_gem_object *obj)
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
- req = i915_gem_active_get(&obj->last_read[i]);
+ req = i915_gem_active_get(&obj->last_read[i],
+ &obj->base.dev->struct_mutex);
if (req)
requests[n++] = req;
}
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index 9e1949f2f4dd..d6e85555c228 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -740,18 +740,38 @@ unwind:
#define i915_error_ggtt_object_create(dev_priv, src) \
i915_error_object_create((dev_priv), (src), &(dev_priv)->ggtt.base)
+/* The error capture is special as tries to run underneath the normal
+ * locking rules - so we use the raw version of the i915_gem_active lookup.
+ */
+static inline uint32_t
+__active_get_seqno(struct i915_gem_active *active)
+{
+ return i915_gem_request_get_seqno(__i915_gem_active_peek(active));
+}
+
+static inline int
+__active_get_engine_id(struct i915_gem_active *active)
+{
+ struct intel_engine_cs *engine;
+
+ engine = i915_gem_request_get_engine(__i915_gem_active_peek(active));
+ return engine ? engine->id : -1;
+}
+
static void capture_bo(struct drm_i915_error_buffer *err,
struct i915_vma *vma)
{
struct drm_i915_gem_object *obj = vma->obj;
- struct intel_engine_cs *engine;
int i;
err->size = obj->base.size;
err->name = obj->base.name;
+
for (i = 0; i < I915_NUM_ENGINES; i++)
- err->rseqno[i] = i915_gem_active_get_seqno(&obj->last_read[i]);
- err->wseqno = i915_gem_active_get_seqno(&obj->last_write);
+ err->rseqno[i] = __active_get_seqno(&obj->last_read[i]);
+ err->wseqno = __active_get_seqno(&obj->last_write);
+ err->ring = __active_get_engine_id(&obj->last_write);
+
err->gtt_offset = vma->node.start;
err->read_domains = obj->base.read_domains;
err->write_domain = obj->base.write_domain;
@@ -764,9 +784,6 @@ static void capture_bo(struct drm_i915_error_buffer *err,
err->purgeable = obj->madv != I915_MADV_WILLNEED;
err->userptr = obj->userptr.mm != NULL;
err->cache_level = obj->cache_level;
-
- engine = i915_gem_active_get_engine(&obj->last_write);
- err->ring = engine ? engine->id : -1;
}
static u32 capture_active_bo(struct drm_i915_error_buffer *err,
diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
index a0abe5588d17..345b24bedc6c 100644
--- a/drivers/gpu/drm/i915/intel_display.c
+++ b/drivers/gpu/drm/i915/intel_display.c
@@ -11465,7 +11465,8 @@ static bool use_mmio_flip(struct intel_engine_cs *engine,
if (resv && !reservation_object_test_signaled_rcu(resv, false))
return true;
- return engine != i915_gem_active_get_engine(&obj->last_write);
+ return engine != i915_gem_active_get_engine(&obj->last_write,
+ &obj->base.dev->struct_mutex);
}
static void skl_do_mmio_flip(struct intel_crtc *intel_crtc,
@@ -11768,7 +11769,8 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
} else if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev)) {
engine = &dev_priv->engine[BCS];
} else if (INTEL_INFO(dev)->gen >= 7) {
- engine = i915_gem_active_get_engine(&obj->last_write);
+ engine = i915_gem_active_get_engine(&obj->last_write,
+ &obj->base.dev->struct_mutex);
if (engine == NULL || engine->id != RCS)
engine = &dev_priv->engine[BCS];
} else {
@@ -11789,7 +11791,8 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
if (mmio_flip) {
INIT_WORK(&work->mmio_work, intel_mmio_flip_work_func);
- work->flip_queued_req = i915_gem_active_get(&obj->last_write);
+ work->flip_queued_req = i915_gem_active_get(&obj->last_write,
+ &obj->base.dev->struct_mutex);
schedule_work(&work->mmio_work);
} else {
request = i915_gem_request_alloc(engine, engine->last_context);
@@ -14091,7 +14094,8 @@ intel_prepare_plane_fb(struct drm_plane *plane,
if (ret == 0) {
to_intel_plane_state(new_state)->wait_req =
- i915_gem_active_get(&obj->last_write);
+ i915_gem_active_get(&obj->last_write,
+ &obj->base.dev->struct_mutex);
}
return ret;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 52/64] drm/i915: Refactor blocking waits
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (49 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 51/64] drm/i915: Mark up i915_gem_active for locking annotation Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 53/64] drm/i915: Rename request->list to link for consistency Chris Wilson
` (12 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
Tidy up the for loops that handle waiting for read/write vs read-only
access.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem.c | 158 +++++++++++++++++++---------------------
1 file changed, 75 insertions(+), 83 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 54aca084b80a..cbe48720aea7 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -1339,6 +1339,23 @@ put_rpm:
return ret;
}
+static void
+i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
+ struct drm_i915_gem_request *req)
+{
+ int ring = req->engine->id;
+
+ if (i915_gem_active_peek(&obj->last_read[ring],
+ &obj->base.dev->struct_mutex) == req)
+ i915_gem_object_retire__read(obj, ring);
+ else if (i915_gem_active_peek(&obj->last_write,
+ &obj->base.dev->struct_mutex) == req)
+ i915_gem_object_retire__write(obj);
+
+ if (!i915_reset_in_progress(&req->i915->gpu_error))
+ i915_gem_request_retire_upto(req);
+}
+
/**
* Ensures that all rendering to the object has completed and the object is
* safe to unbind from the GTT or access from the CPU.
@@ -1349,39 +1366,34 @@ int
i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
bool readonly)
{
- struct drm_i915_gem_request *request;
struct reservation_object *resv;
- int ret, i;
+ struct i915_gem_active *active;
+ unsigned long active_mask;
+ int idx, ret;
- if (readonly) {
- request = i915_gem_active_peek(&obj->last_write,
- &obj->base.dev->struct_mutex);
- if (request) {
- ret = i915_wait_request(request);
- if (ret)
- return ret;
+ lockdep_assert_held(&obj->base.dev->struct_mutex);
- i = request->engine->id;
- if (i915_gem_active_peek(&obj->last_read[i],
- &obj->base.dev->struct_mutex) == request)
- i915_gem_object_retire__read(obj, i);
- else
- i915_gem_object_retire__write(obj);
- }
+ if (!readonly) {
+ active = obj->last_read;
+ active_mask = obj->active;
} else {
- for (i = 0; i < I915_NUM_ENGINES; i++) {
- request = i915_gem_active_peek(&obj->last_read[i],
- &obj->base.dev->struct_mutex);
- if (!request)
- continue;
+ active_mask = 1;
+ active = &obj->last_write;
+ }
- ret = i915_wait_request(request);
- if (ret)
- return ret;
+ for_each_active(active_mask, idx) {
+ struct drm_i915_gem_request *request;
- i915_gem_object_retire__read(obj, i);
- }
- GEM_BUG_ON(obj->active);
+ request = i915_gem_active_peek(&active[idx],
+ &obj->base.dev->struct_mutex);
+ if (!request)
+ continue;
+
+ ret = i915_wait_request(request);
+ if (ret)
+ return ret;
+
+ i915_gem_object_retire_request(obj, request);
}
resv = i915_gem_object_get_dmabuf_resv(obj);
@@ -1397,23 +1409,6 @@ i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
return 0;
}
-static void
-i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
- struct drm_i915_gem_request *req)
-{
- int ring = req->engine->id;
-
- if (i915_gem_active_peek(&obj->last_read[ring],
- &obj->base.dev->struct_mutex) == req)
- i915_gem_object_retire__read(obj, ring);
- else if (i915_gem_active_peek(&obj->last_write,
- &obj->base.dev->struct_mutex) == req)
- i915_gem_object_retire__write(obj);
-
- if (!i915_reset_in_progress(&req->i915->gpu_error))
- i915_gem_request_retire_upto(req);
-}
-
/* A nonblocking variant of the above wait. This is a highly dangerous routine
* as the object state may change during this call.
*/
@@ -1425,34 +1420,31 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
struct drm_device *dev = obj->base.dev;
struct drm_i915_private *dev_priv = to_i915(dev);
struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
+ struct i915_gem_active *active;
+ unsigned long active_mask;
int ret, i, n = 0;
BUG_ON(!mutex_is_locked(&dev->struct_mutex));
BUG_ON(!dev_priv->mm.interruptible);
- if (!obj->active)
+ active_mask = obj->active;
+ if (!active_mask)
return 0;
- if (readonly) {
- struct drm_i915_gem_request *req;
-
- req = i915_gem_active_peek(&obj->last_write,
- &obj->base.dev->struct_mutex);
- if (req == NULL)
- return 0;
-
- requests[n++] = req;
+ if (!readonly) {
+ active = obj->last_read;
} else {
- for (i = 0; i < I915_NUM_ENGINES; i++) {
- struct drm_i915_gem_request *req;
+ active_mask = 1;
+ active = &obj->last_write;
+ }
- req = i915_gem_active_peek(&obj->last_read[i],
- &obj->base.dev->struct_mutex);
- if (req == NULL)
- continue;
+ for_each_active(active_mask, i) {
+ struct drm_i915_gem_request *req;
+ req = i915_gem_active_get(&active[i],
+ &obj->base.dev->struct_mutex);
+ if (req)
requests[n++] = req;
- }
}
mutex_unlock(&dev->struct_mutex);
@@ -2926,33 +2918,33 @@ int
i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct drm_i915_gem_request *to)
{
- const bool readonly = obj->base.pending_write_domain == 0;
- struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
- int ret, i, n;
+ struct i915_gem_active *active;
+ unsigned long active_mask;
+ int idx;
- if (!obj->active)
- return 0;
+ lockdep_assert_held(&obj->base.dev->struct_mutex);
- n = 0;
- if (readonly) {
- struct drm_i915_gem_request *req;
+ active_mask = obj->active;
+ if (!active_mask)
+ return 0;
- req = i915_gem_active_peek(&obj->last_write,
- &obj->base.dev->struct_mutex);
- if (req)
- requests[n++] = req;
+ if (obj->base.pending_write_domain) {
+ active = obj->last_read;
} else {
- for (i = 0; i < I915_NUM_ENGINES; i++) {
- struct drm_i915_gem_request *req;
-
- req = i915_gem_active_peek(&obj->last_read[i],
- &obj->base.dev->struct_mutex);
- if (req)
- requests[n++] = req;
- }
+ active_mask = 1;
+ active = &obj->last_write;
}
- for (i = 0; i < n; i++) {
- ret = __i915_gem_object_sync(obj, to, requests[i]);
+
+ for_each_active(active_mask, idx) {
+ struct drm_i915_gem_request *request;
+ int ret;
+
+ request = i915_gem_active_peek(&active[idx],
+ &obj->base.dev->struct_mutex);
+ if (!request)
+ continue;
+
+ ret = __i915_gem_object_sync(obj, to, request);
if (ret)
return ret;
}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 53/64] drm/i915: Rename request->list to link for consistency
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (50 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 52/64] drm/i915: Refactor blocking waits Chris Wilson
@ 2016-07-07 8:41 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 54/64] drm/i915: Remove obsolete i915_gem_object_flush_active() Chris Wilson
` (11 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:41 UTC (permalink / raw)
To: intel-gfx
We use "list" to denote the list and "link" to denote an element on that
list. Rename request->list to match this idiom.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 4 ++--
drivers/gpu/drm/i915/i915_gem.c | 10 +++++-----
drivers/gpu/drm/i915/i915_gem_request.c | 12 ++++++------
drivers/gpu/drm/i915/i915_gem_request.h | 4 ++--
drivers/gpu/drm/i915/i915_gpu_error.c | 4 ++--
drivers/gpu/drm/i915/intel_ringbuffer.c | 6 +++---
6 files changed, 20 insertions(+), 20 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index b260d241f955..7fdef8058d31 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -748,13 +748,13 @@ static int i915_gem_request_info(struct seq_file *m, void *data)
int count;
count = 0;
- list_for_each_entry(req, &engine->request_list, list)
+ list_for_each_entry(req, &engine->request_list, link)
count++;
if (count == 0)
continue;
seq_printf(m, "%s requests: %d\n", engine->name, count);
- list_for_each_entry(req, &engine->request_list, list) {
+ list_for_each_entry(req, &engine->request_list, link) {
struct task_struct *task;
rcu_read_lock();
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index cbe48720aea7..436ced7acf2e 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2475,7 +2475,7 @@ i915_gem_find_active_request(struct intel_engine_cs *engine)
* extra delay for a recent interrupt is pointless. Hence, we do
* not need an engine->irq_seqno_barrier() before the seqno reads.
*/
- list_for_each_entry(request, &engine->request_list, list) {
+ list_for_each_entry(request, &engine->request_list, link) {
if (i915_gem_request_completed(request))
continue;
@@ -2497,7 +2497,7 @@ static void i915_gem_reset_engine_status(struct intel_engine_cs *engine)
ring_hung = engine->hangcheck.score >= HANGCHECK_SCORE_RING_HUNG;
i915_set_reset_status(request->ctx, ring_hung);
- list_for_each_entry_continue(request, &engine->request_list, list)
+ list_for_each_entry_continue(request, &engine->request_list, link)
i915_set_reset_status(request->ctx, false);
}
@@ -2546,7 +2546,7 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
request = list_last_entry(&engine->request_list,
struct drm_i915_gem_request,
- list);
+ link);
i915_gem_request_retire_upto(request);
}
@@ -2606,7 +2606,7 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
request = list_first_entry(&engine->request_list,
struct drm_i915_gem_request,
- list);
+ link);
if (!i915_gem_request_completed(request))
break;
@@ -2626,7 +2626,7 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
engine_list[engine->id]);
if (!list_empty(&i915_gem_active_peek(&obj->last_read[engine->id],
- &obj->base.dev->struct_mutex)->list))
+ &obj->base.dev->struct_mutex)->link))
break;
i915_gem_object_retire__read(obj, engine->id);
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 6e99776a800b..c43f9f42125f 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -160,7 +160,7 @@ i915_gem_request_remove_from_client(struct drm_i915_gem_request *request)
static void i915_gem_request_retire(struct drm_i915_gem_request *request)
{
trace_i915_gem_request_retire(request);
- list_del_init(&request->list);
+ list_del_init(&request->link);
/* We know the GPU must have read the request to have
* sent us the seqno + interrupt, so use the position
@@ -191,12 +191,12 @@ void i915_gem_request_retire_upto(struct drm_i915_gem_request *req)
lockdep_assert_held(&req->i915->drm.struct_mutex);
- if (list_empty(&req->list))
+ if (list_empty(&req->link))
return;
do {
tmp = list_first_entry(&engine->request_list,
- typeof(*tmp), list);
+ typeof(*tmp), link);
i915_gem_request_retire(tmp);
} while (tmp != req);
@@ -317,7 +317,7 @@ i915_gem_request_alloc(struct intel_engine_cs *engine,
if (!list_empty(&engine->request_list)) {
req = list_first_entry(&engine->request_list,
- typeof(*req), list);
+ typeof(*req), link);
if (i915_gem_request_completed(req))
i915_gem_request_retire(req);
}
@@ -452,7 +452,7 @@ void __i915_add_request(struct drm_i915_gem_request *request,
request->emitted_jiffies = jiffies;
request->previous_seqno = engine->last_submitted_seqno;
smp_store_mb(engine->last_submitted_seqno, request->fence.seqno);
- list_add_tail(&request->list, &engine->request_list);
+ list_add_tail(&request->link, &engine->request_list);
/* Record the position of the start of the request so that
* should we detect the updated seqno part-way through the
@@ -572,7 +572,7 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
might_sleep();
- if (list_empty(&req->list))
+ if (list_empty(&req->link))
return 0;
if (i915_gem_request_completed(req))
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index 34e65e07cad5..08bc7189b508 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -105,8 +105,8 @@ struct drm_i915_gem_request {
/** Time at which this request was emitted, in jiffies. */
unsigned long emitted_jiffies;
- /** global list entry for this request */
- struct list_head list;
+ /** engine->request_list entry for this request */
+ struct list_head link;
struct drm_i915_file_private *file_priv;
/** file_priv list entry for this request */
diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
index d6e85555c228..7d773d7bc66e 100644
--- a/drivers/gpu/drm/i915/i915_gpu_error.c
+++ b/drivers/gpu/drm/i915/i915_gpu_error.c
@@ -1163,7 +1163,7 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
i915_gem_record_active_context(engine, error, &error->ring[i]);
count = 0;
- list_for_each_entry(request, &engine->request_list, list)
+ list_for_each_entry(request, &engine->request_list, link)
count++;
error->ring[i].num_requests = count;
@@ -1176,7 +1176,7 @@ static void i915_gem_record_rings(struct drm_i915_private *dev_priv,
}
count = 0;
- list_for_each_entry(request, &engine->request_list, list) {
+ list_for_each_entry(request, &engine->request_list, link) {
struct drm_i915_error_request *erq;
if (count >= error->ring[i].num_requests) {
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index c0c25e1703e0..8fddc8d2358b 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2250,7 +2250,7 @@ int intel_engine_idle(struct intel_engine_cs *engine)
req = list_entry(engine->request_list.prev,
struct drm_i915_gem_request,
- list);
+ link);
/* Make sure we do not trigger any retires */
return __i915_wait_request(req,
@@ -2299,7 +2299,7 @@ static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
*/
GEM_BUG_ON(!req->reserved_space);
- list_for_each_entry(target, &engine->request_list, list) {
+ list_for_each_entry(target, &engine->request_list, link) {
unsigned space;
/*
@@ -2317,7 +2317,7 @@ static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
break;
}
- if (WARN_ON(&target->list == &engine->request_list))
+ if (WARN_ON(&target->link == &engine->request_list))
return -ENOSPC;
return i915_wait_request(target);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 54/64] drm/i915: Remove obsolete i915_gem_object_flush_active()
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (51 preceding siblings ...)
2016-07-07 8:41 ` [PATCH 53/64] drm/i915: Rename request->list to link for consistency Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 55/64] drm/i915: Refactor activity tracking for requests Chris Wilson
` (10 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
Since we track requests, and requests are always added to the GPU fully
formed, we never have to flush the incomplete request and know that the
given request will eventually complete without any further action on our
part.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem.c | 59 +++--------------------------------------
1 file changed, 3 insertions(+), 56 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 436ced7acf2e..60818edfa04d 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2738,35 +2738,6 @@ out_rearm:
}
/**
- * Ensures that an object will eventually get non-busy by flushing any required
- * write domains, emitting any outstanding lazy request and retiring and
- * completed requests.
- * @obj: object to flush
- */
-static int
-i915_gem_object_flush_active(struct drm_i915_gem_object *obj)
-{
- int i;
-
- if (!obj->active)
- return 0;
-
- for (i = 0; i < I915_NUM_ENGINES; i++) {
- struct drm_i915_gem_request *req;
-
- req = i915_gem_active_peek(&obj->last_read[i],
- &obj->base.dev->struct_mutex);
- if (req == NULL)
- continue;
-
- if (i915_gem_request_completed(req))
- i915_gem_object_retire__read(obj, i);
- }
-
- return 0;
-}
-
-/**
* i915_gem_wait_ioctl - implements DRM_IOCTL_I915_GEM_WAIT
* @dev: drm device pointer
* @data: ioctl data blob
@@ -2812,24 +2783,9 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
return -ENOENT;
}
- /* Need to make sure the object gets inactive eventually. */
- ret = i915_gem_object_flush_active(obj);
- if (ret)
- goto out;
-
if (!obj->active)
goto out;
- /* Do this after OLR check to make sure we make forward progress polling
- * on this IOCTL with a timeout == 0 (like busy ioctl)
- */
- if (args->timeout_ns == 0) {
- ret = -ETIME;
- goto out;
- }
-
- i915_gem_object_put(obj);
-
for (i = 0; i < I915_NUM_ENGINES; i++) {
struct drm_i915_gem_request *req;
@@ -2839,6 +2795,8 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
requests[n++] = req;
}
+out:
+ i915_gem_object_put(obj);
mutex_unlock(&dev->struct_mutex);
for (i = 0; i < n; i++) {
@@ -2849,11 +2807,6 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
i915_gem_request_put(requests[i]);
}
return ret;
-
-out:
- i915_gem_object_put(obj);
- mutex_unlock(&dev->struct_mutex);
- return ret;
}
static int
@@ -4028,13 +3981,8 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
/* Count all active objects as busy, even if they are currently not used
* by the gpu. Users of this interface expect objects to eventually
- * become non-busy without any further actions, therefore emit any
- * necessary flushes here.
+ * become non-busy without any further actions.
*/
- ret = i915_gem_object_flush_active(obj);
- if (ret)
- goto unref;
-
args->busy = 0;
if (obj->active) {
struct drm_i915_gem_request *req;
@@ -4052,7 +4000,6 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
args->busy |= req->engine->exec_id;
}
-unref:
i915_gem_object_put(obj);
unlock:
mutex_unlock(&dev->struct_mutex);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 55/64] drm/i915: Refactor activity tracking for requests
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (52 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 54/64] drm/i915: Remove obsolete i915_gem_object_flush_active() Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 56/64] drm/i915: Convert intel_overlay to request tracking Chris Wilson
` (9 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
With the introduction of requests, we amplified the number of atomic
refcounted objects we use and update every execbuffer; from none to
several references, and a set of references that need to be changed. We
also introduced interesting side-effects in the order of retiring
requests and objects.
Instead of independently tracking the last request for an object, track
the active objects for each request. The object will reside in the
buffer list of its most recent active request and so we reduce the kref
interchange to a list_move. Now retirements are entirely driven by the
request, dramatically simplifying activity tracking on the object
themselves, and removing the ambiguity between retiring objects and
retiring requests.
Furthermore with the consolidation of managing the activity tracking
centrally, we can look forward to using RCU to enable lockless lookup of
the current active requests for an object. In the future, we will be
able to query the status or wait upon rendering to an object without
even touching the struct_mutex BKL.
All told, less code, simpler and faster, and more extensible.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/Makefile | 1 -
drivers/gpu/drm/i915/i915_drv.h | 10 ---
drivers/gpu/drm/i915/i915_gem.c | 139 +++++++-------------------------
drivers/gpu/drm/i915/i915_gem_debug.c | 70 ----------------
drivers/gpu/drm/i915/i915_gem_fence.c | 11 +--
drivers/gpu/drm/i915/i915_gem_request.c | 35 ++++++--
drivers/gpu/drm/i915/i915_gem_request.h | 86 +++++++++++++-------
drivers/gpu/drm/i915/intel_lrc.c | 1 -
drivers/gpu/drm/i915/intel_ringbuffer.c | 1 -
drivers/gpu/drm/i915/intel_ringbuffer.h | 12 ---
10 files changed, 118 insertions(+), 248 deletions(-)
delete mode 100644 drivers/gpu/drm/i915/i915_gem_debug.c
diff --git a/drivers/gpu/drm/i915/Makefile b/drivers/gpu/drm/i915/Makefile
index f7a3f1421869..5318ce058e21 100644
--- a/drivers/gpu/drm/i915/Makefile
+++ b/drivers/gpu/drm/i915/Makefile
@@ -25,7 +25,6 @@ i915-$(CONFIG_DEBUG_FS) += i915_debugfs.o
i915-y += i915_cmd_parser.o \
i915_gem_batch_pool.o \
i915_gem_context.o \
- i915_gem_debug.o \
i915_gem_dmabuf.o \
i915_gem_evict.o \
i915_gem_execbuffer.o \
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 5b096c65646c..fa42cb14ed9b 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -429,8 +429,6 @@ void intel_link_compute_m_n(int bpp, int nlanes,
#define DRIVER_MINOR 6
#define DRIVER_PATCHLEVEL 0
-#define WATCH_LISTS 0
-
struct opregion_header;
struct opregion_acpi;
struct opregion_swsci;
@@ -2149,7 +2147,6 @@ struct drm_i915_gem_object {
struct drm_mm_node *stolen;
struct list_head global_list;
- struct list_head engine_list[I915_NUM_ENGINES];
/** Used in execbuf to temporarily hold a ref */
struct list_head obj_exec_link;
@@ -3442,13 +3439,6 @@ static inline bool i915_gem_object_needs_bit17_swizzle(struct drm_i915_gem_objec
obj->tiling_mode != I915_TILING_NONE;
}
-/* i915_gem_debug.c */
-#if WATCH_LISTS
-int i915_verify_lists(struct drm_device *dev);
-#else
-#define i915_verify_lists(dev) 0
-#endif
-
/* i915_debugfs.c */
#ifdef CONFIG_DEBUG_FS
int i915_debugfs_register(struct drm_i915_private *dev_priv);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 60818edfa04d..a38939dab741 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -43,10 +43,6 @@
static void i915_gem_object_flush_gtt_write_domain(struct drm_i915_gem_object *obj);
static void i915_gem_object_flush_cpu_write_domain(struct drm_i915_gem_object *obj);
-static void
-i915_gem_object_retire__write(struct drm_i915_gem_object *obj);
-static void
-i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring);
static bool cpu_cache_is_coherent(struct drm_device *dev,
enum i915_cache_level level)
@@ -141,7 +137,6 @@ int i915_mutex_lock_interruptible(struct drm_device *dev)
if (ret)
return ret;
- WARN_ON(i915_verify_lists(dev));
return 0;
}
@@ -1339,23 +1334,6 @@ put_rpm:
return ret;
}
-static void
-i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
- struct drm_i915_gem_request *req)
-{
- int ring = req->engine->id;
-
- if (i915_gem_active_peek(&obj->last_read[ring],
- &obj->base.dev->struct_mutex) == req)
- i915_gem_object_retire__read(obj, ring);
- else if (i915_gem_active_peek(&obj->last_write,
- &obj->base.dev->struct_mutex) == req)
- i915_gem_object_retire__write(obj);
-
- if (!i915_reset_in_progress(&req->i915->gpu_error))
- i915_gem_request_retire_upto(req);
-}
-
/**
* Ensures that all rendering to the object has completed and the object is
* safe to unbind from the GTT or access from the CPU.
@@ -1382,18 +1360,10 @@ i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
}
for_each_active(active_mask, idx) {
- struct drm_i915_gem_request *request;
-
- request = i915_gem_active_peek(&active[idx],
- &obj->base.dev->struct_mutex);
- if (!request)
- continue;
-
- ret = i915_wait_request(request);
+ ret = i915_gem_active_wait(&active[idx],
+ &obj->base.dev->struct_mutex);
if (ret)
return ret;
-
- i915_gem_object_retire_request(obj, request);
}
resv = i915_gem_object_get_dmabuf_resv(obj);
@@ -1453,11 +1423,8 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
ret = __i915_wait_request(requests[i], true, NULL, rps);
mutex_lock(&dev->struct_mutex);
- for (i = 0; i < n; i++) {
- if (ret == 0)
- i915_gem_object_retire_request(obj, requests[i]);
+ for (i = 0; i < n; i++)
i915_gem_request_put(requests[i]);
- }
return ret;
}
@@ -2376,40 +2343,38 @@ void i915_vma_move_to_active(struct i915_vma *vma,
i915_gem_object_get(obj);
obj->active |= intel_engine_flag(engine);
- list_move_tail(&obj->engine_list[engine->id], &engine->active_list);
i915_gem_active_set(&obj->last_read[engine->id], req);
list_move_tail(&vma->vm_link, &vma->vm->active_list);
}
static void
-i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
+i915_gem_object_retire__fence(struct i915_gem_active *active,
+ struct drm_i915_gem_request *req)
{
- GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_write));
- GEM_BUG_ON(!(obj->active &
- intel_engine_flag(i915_gem_active_get_engine(&obj->last_write,
- &obj->base.dev->struct_mutex))));
+}
- i915_gem_active_set(&obj->last_write, NULL);
- intel_fb_obj_flush(obj, true, ORIGIN_CS);
+static void
+i915_gem_object_retire__write(struct i915_gem_active *active,
+ struct drm_i915_gem_request *request)
+{
+ intel_fb_obj_flush(container_of(active,
+ struct drm_i915_gem_object,
+ last_write),
+ true,
+ ORIGIN_CS);
}
static void
-i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
+i915_gem_object_retire__read(struct i915_gem_active *active,
+ struct drm_i915_gem_request *request)
{
- struct intel_engine_cs *engine;
+ int ring = request->engine->id;
+ struct drm_i915_gem_object *obj =
+ container_of(active, struct drm_i915_gem_object, last_read[ring]);
struct i915_vma *vma;
- GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_read[ring]));
- GEM_BUG_ON(!(obj->active & (1 << ring)));
-
- list_del_init(&obj->engine_list[ring]);
- i915_gem_active_set(&obj->last_read[ring], NULL);
-
- engine = i915_gem_active_get_engine(&obj->last_write,
- &obj->base.dev->struct_mutex);
- if (engine && engine->id == ring)
- i915_gem_object_retire__write(obj);
+ GEM_BUG_ON((obj->active & (1 << ring)) == 0);
obj->active &= ~(1 << ring);
if (obj->active)
@@ -2419,15 +2384,13 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
* so that we don't steal from recently used but inactive objects
* (unless we are forced to ofc!)
*/
- list_move_tail(&obj->global_list,
- &to_i915(obj->base.dev)->mm.bound_list);
+ list_move_tail(&obj->global_list, &request->i915->mm.bound_list);
list_for_each_entry(vma, &obj->vma_list, obj_link) {
if (!list_empty(&vma->vm_link))
list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
}
- i915_gem_active_set(&obj->last_fence, NULL);
i915_gem_object_put(obj);
}
@@ -2505,16 +2468,6 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
{
struct intel_ring *ring;
- while (!list_empty(&engine->active_list)) {
- struct drm_i915_gem_object *obj;
-
- obj = list_first_entry(&engine->active_list,
- struct drm_i915_gem_object,
- engine_list[engine->id]);
-
- i915_gem_object_retire__read(obj, engine->id);
- }
-
/* Mark all pending requests as complete so that any concurrent
* (lockless) lookup doesn't try and wait upon the request as we
* reset it.
@@ -2583,8 +2536,6 @@ void i915_gem_reset(struct drm_device *dev)
i915_gem_context_reset(dev);
i915_gem_restore_fences(dev);
-
- WARN_ON(i915_verify_lists(dev));
}
/**
@@ -2594,13 +2545,6 @@ void i915_gem_reset(struct drm_device *dev)
void
i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
{
- WARN_ON(i915_verify_lists(engine->dev));
-
- /* Retire requests first as we use it above for the early return.
- * If we retire requests last, we may use a later seqno and so clear
- * the requests lists without clearing the active list, leading to
- * confusion.
- */
while (!list_empty(&engine->request_list)) {
struct drm_i915_gem_request *request;
@@ -2613,26 +2557,6 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
i915_gem_request_retire_upto(request);
}
-
- /* Move any buffers on the active list that are no longer referenced
- * by the ringbuffer to the flushing/inactive lists as appropriate,
- * before we free the context associated with the requests.
- */
- while (!list_empty(&engine->active_list)) {
- struct drm_i915_gem_object *obj;
-
- obj = list_first_entry(&engine->active_list,
- struct drm_i915_gem_object,
- engine_list[engine->id]);
-
- if (!list_empty(&i915_gem_active_peek(&obj->last_read[engine->id],
- &obj->base.dev->struct_mutex)->link))
- break;
-
- i915_gem_object_retire__read(obj, engine->id);
- }
-
- WARN_ON(i915_verify_lists(engine->dev));
}
void i915_gem_retire_requests(struct drm_i915_private *dev_priv)
@@ -2810,8 +2734,7 @@ out:
}
static int
-__i915_gem_object_sync(struct drm_i915_gem_object *obj,
- struct drm_i915_gem_request *to,
+__i915_gem_object_sync(struct drm_i915_gem_request *to,
struct drm_i915_gem_request *from)
{
int ret;
@@ -2819,9 +2742,6 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (to->engine == from->engine)
return 0;
- if (i915_gem_request_completed(from))
- return 0;
-
if (!i915.semaphores) {
ret = __i915_wait_request(from,
from->i915->mm.interruptible,
@@ -2829,8 +2749,6 @@ __i915_gem_object_sync(struct drm_i915_gem_object *obj,
NO_WAITBOOST);
if (ret)
return ret;
-
- i915_gem_object_retire_request(obj, from);
} else {
int idx = intel_engine_sync_index(from->engine, to->engine);
if (from->fence.seqno <= from->engine->semaphore.sync_seqno[idx])
@@ -2897,7 +2815,7 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
if (!request)
continue;
- ret = __i915_gem_object_sync(obj, to, request);
+ ret = __i915_gem_object_sync(to, request);
if (ret)
return ret;
}
@@ -3033,7 +2951,6 @@ int i915_gem_wait_for_idle(struct drm_i915_private *dev_priv)
return ret;
}
- WARN_ON(i915_verify_lists(dev));
return 0;
}
@@ -4077,7 +3994,12 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
INIT_LIST_HEAD(&obj->global_list);
for (i = 0; i < I915_NUM_ENGINES; i++)
- INIT_LIST_HEAD(&obj->engine_list[i]);
+ init_request_active(&obj->last_read[i],
+ i915_gem_object_retire__read);
+ init_request_active(&obj->last_write,
+ i915_gem_object_retire__write);
+ init_request_active(&obj->last_fence,
+ i915_gem_object_retire__fence);
INIT_LIST_HEAD(&obj->obj_exec_link);
INIT_LIST_HEAD(&obj->vma_list);
INIT_LIST_HEAD(&obj->batch_pool_link);
@@ -4583,7 +4505,6 @@ i915_gem_cleanup_engines(struct drm_device *dev)
static void
init_engine_lists(struct intel_engine_cs *engine)
{
- INIT_LIST_HEAD(&engine->active_list);
INIT_LIST_HEAD(&engine->request_list);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_debug.c b/drivers/gpu/drm/i915/i915_gem_debug.c
deleted file mode 100644
index a56516482394..000000000000
--- a/drivers/gpu/drm/i915/i915_gem_debug.c
+++ /dev/null
@@ -1,70 +0,0 @@
-/*
- * Copyright © 2008 Intel Corporation
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
- * IN THE SOFTWARE.
- *
- * Authors:
- * Keith Packard <keithp@keithp.com>
- *
- */
-
-#include <drm/drmP.h>
-#include <drm/i915_drm.h>
-#include "i915_drv.h"
-
-#if WATCH_LISTS
-int
-i915_verify_lists(struct drm_device *dev)
-{
- static int warned;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct drm_i915_gem_object *obj;
- struct intel_engine_cs *engine;
- int err = 0;
-
- if (warned)
- return 0;
-
- for_each_engine(engine, dev_priv) {
- list_for_each_entry(obj, &engine->active_list,
- engine_list[engine->id]) {
- if (obj->base.dev != dev ||
- !atomic_read(&obj->base.refcount.refcount)) {
- DRM_ERROR("%s: freed active obj %p\n",
- engine->name, obj);
- err++;
- break;
- } else if (!obj->active ||
- obj->last_read_req[engine->id] == NULL) {
- DRM_ERROR("%s: invalid active obj %p\n",
- engine->name, obj);
- err++;
- } else if (obj->base.write_domain) {
- DRM_ERROR("%s: invalid write obj %p (w %x)\n",
- engine->name,
- obj, obj->base.write_domain);
- err++;
- }
- }
- }
-
- return warned = err;
-}
-#endif /* WATCH_LIST */
diff --git a/drivers/gpu/drm/i915/i915_gem_fence.c b/drivers/gpu/drm/i915/i915_gem_fence.c
index a4ec4fecbea0..dbaab9ce29c9 100644
--- a/drivers/gpu/drm/i915/i915_gem_fence.c
+++ b/drivers/gpu/drm/i915/i915_gem_fence.c
@@ -261,15 +261,8 @@ static inline void i915_gem_object_fence_lost(struct drm_i915_gem_object *obj)
static int
i915_gem_object_wait_fence(struct drm_i915_gem_object *obj)
{
- int ret;
-
- ret = i915_gem_active_wait(&obj->last_fence,
- &obj->base.dev->struct_mutex);
- if (ret)
- return ret;
-
- i915_gem_active_set(&obj->last_fence, NULL);
- return 0;
+ return i915_gem_active_retire(&obj->last_fence,
+ &obj->base.dev->struct_mutex);
}
/**
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index c43f9f42125f..6c0447b6e694 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -22,6 +22,8 @@
*
*/
+#include <linux/prefetch.h>
+
#include "i915_drv.h"
static const char *i915_fence_get_driver_name(struct fence *fence)
@@ -159,6 +161,8 @@ i915_gem_request_remove_from_client(struct drm_i915_gem_request *request)
static void i915_gem_request_retire(struct drm_i915_gem_request *request)
{
+ struct i915_gem_active *active, *next;
+
trace_i915_gem_request_retire(request);
list_del_init(&request->link);
@@ -172,6 +176,24 @@ static void i915_gem_request_retire(struct drm_i915_gem_request *request)
*/
request->ring->last_retired_head = request->postfix;
+ /* Walk through the active list, calling retire on each. This allows
+ * objects to track their GPU activity and mark themselves as idle
+ * when their *last* active request is completed (updating state
+ * tracking lists for eviction, active references for GEM, etc).
+ *
+ * As the ->retire() may free the node, we decouple it first and
+ * pass along the auxiliary information (to avoid dereferencing
+ * the node after the callback).
+ */
+ list_for_each_entry_safe(active, next, &request->active_list, link) {
+ prefetchw(next);
+
+ INIT_LIST_HEAD(&active->link);
+ active->__request = NULL;
+
+ active->retire(active, request);
+ }
+
i915_gem_request_remove_from_client(request);
if (request->previous_context) {
@@ -200,8 +222,6 @@ void i915_gem_request_retire_upto(struct drm_i915_gem_request *req)
i915_gem_request_retire(tmp);
} while (tmp != req);
-
- WARN_ON(i915_verify_lists(engine->dev));
}
static int i915_gem_check_wedge(unsigned int reset_counter, bool interruptible)
@@ -337,6 +357,7 @@ i915_gem_request_alloc(struct intel_engine_cs *engine,
engine->fence_context,
seqno);
+ INIT_LIST_HEAD(&req->active_list);
req->i915 = dev_priv;
req->engine = engine;
req->ctx = i915_gem_context_get(ctx);
@@ -572,9 +593,6 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
might_sleep();
- if (list_empty(&req->link))
- return 0;
-
if (i915_gem_request_completed(req))
return 0;
@@ -707,10 +725,13 @@ int i915_wait_request(struct drm_i915_gem_request *req)
{
int ret;
- GEM_BUG_ON(!req);
lockdep_assert_held(&req->i915->drm.struct_mutex);
+ GEM_BUG_ON(list_empty(&req->link));
- ret = __i915_wait_request(req, req->i915->mm.interruptible, NULL, NULL);
+ ret = __i915_wait_request(req,
+ req->i915->mm.interruptible,
+ NULL,
+ NULL);
if (ret)
return ret;
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index 08bc7189b508..8e1c5dc1f3c3 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -101,6 +101,7 @@ struct drm_i915_gem_request {
* error state dump only).
*/
struct drm_i915_gem_object *batch_obj;
+ struct list_head active_list;
/** Time at which this request was emitted, in jiffies. */
unsigned long emitted_jiffies;
@@ -211,8 +212,12 @@ struct intel_rps_client;
int __i915_wait_request(struct drm_i915_gem_request *req,
bool interruptible,
s64 *timeout,
- struct intel_rps_client *rps);
-int __must_check i915_wait_request(struct drm_i915_gem_request *req);
+ struct intel_rps_client *rps)
+ __attribute__((nonnull(1)));
+
+int __must_check
+i915_wait_request(struct drm_i915_gem_request *req)
+ __attribute__((nonnull));
static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine);
@@ -276,6 +281,9 @@ static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
*/
struct i915_gem_active {
struct drm_i915_gem_request *__request;
+ struct list_head link;
+ void (*retire)(struct i915_gem_active *,
+ struct drm_i915_gem_request *);
};
/**
@@ -288,10 +296,20 @@ struct i915_gem_active {
* retired, the @active tracker is updated to report idle.
*/
static inline void
+init_request_active(struct i915_gem_active *active,
+ void (*func)(struct i915_gem_active *,
+ struct drm_i915_gem_request *))
+{
+ INIT_LIST_HEAD(&active->link);
+ active->retire = func;
+}
+
+static inline void
i915_gem_active_set(struct i915_gem_active *active,
struct drm_i915_gem_request *request)
{
- i915_gem_request_assign(&active->__request, request);
+ list_move(&active->link, &request->active_list);
+ active->__request = request;
}
static inline struct drm_i915_gem_request *
@@ -301,17 +319,23 @@ __i915_gem_active_peek(const struct i915_gem_active *active)
}
/**
- * i915_gem_active_peek - report the request being monitored
+ * i915_gem_active_peek - report the active request being monitored
* @active - the active tracker
*
- * i915_gem_active_peek() returns the current request being tracked, or NULL.
- * It does not obtain a reference on the request for the caller, so the
- * caller must hold struct_mutex.
+ * i915_gem_active_peek() returns the current request being tracked if
+ * still active, or NULL. It does not obtain a reference on the request
+ * for the caller, so the caller must hold struct_mutex.
*/
static inline struct drm_i915_gem_request *
i915_gem_active_peek(const struct i915_gem_active *active, struct mutex *mutex)
{
- return active->__request;
+ struct drm_i915_gem_request *request;
+
+ request = active->__request;
+ if (!request || i915_gem_request_completed(request))
+ return NULL;
+
+ return request;
}
/**
@@ -324,13 +348,7 @@ i915_gem_active_peek(const struct i915_gem_active *active, struct mutex *mutex)
static inline struct drm_i915_gem_request *
i915_gem_active_get(const struct i915_gem_active *active, struct mutex *mutex)
{
- struct drm_i915_gem_request *request;
-
- request = i915_gem_active_peek(active, mutex);
- if (!request || i915_gem_request_completed(request))
- return NULL;
-
- return i915_gem_request_get(request);
+ return i915_gem_request_get(i915_gem_active_peek(active, mutex));
}
/**
@@ -359,13 +377,7 @@ static inline bool
i915_gem_active_is_idle(const struct i915_gem_active *active,
struct mutex *mutex)
{
- struct drm_i915_gem_request *request;
-
- request = i915_gem_active_peek(active, mutex);
- if (!request || i915_gem_request_completed(request))
- return true;
-
- return false;
+ return !i915_gem_active_peek(active, mutex);
}
/**
@@ -373,7 +385,9 @@ i915_gem_active_is_idle(const struct i915_gem_active *active,
* @active - the active request on which to wait
*
* i915_gem_active_wait() waits until the request is completed before
- * returning.
+ * returning. i915_gem_active_wait() returns immediately if the active
+ * request is already complete. i915_gem_active_wait() does not automatically
+ * retire the request, i.e. @active->retire callback will not be triggered.
*/
static inline int __must_check
i915_gem_active_wait(const struct i915_gem_active *active, struct mutex *mutex)
@@ -384,21 +398,37 @@ i915_gem_active_wait(const struct i915_gem_active *active, struct mutex *mutex)
if (!request)
return 0;
- return i915_wait_request(request);
+ return __i915_wait_request(request, true, NULL, NULL);
}
/**
* i915_gem_active_retire - waits until the request is retired
* @active - the active request on which to wait
*
- * Unlike i915_gem_active_eait(), this i915_gem_active_retire() will
- * make sure the request is retired before returning.
+ * Unlike i915_gem_active_wait(), i915_gem_active_retire() will
+ * make sure the active request is retired (i.e. has completed and run the
+ * retirement callbacks) before returning.
*/
static inline int __must_check
-i915_gem_active_retire(const struct i915_gem_active *active,
+i915_gem_active_retire(struct i915_gem_active *active,
struct mutex *mutex)
{
- return i915_gem_active_wait(active, mutex);
+ struct drm_i915_gem_request *request;
+ int ret;
+
+ request = active->__request;
+ if (!request)
+ return 0;
+
+ ret = __i915_wait_request(request, true, NULL, NULL);
+ if (ret)
+ return ret;
+
+ list_del_init(&active->link);
+ active->__request = NULL;
+ active->retire(active, request);
+
+ return 0;
}
/* Convenience functions for peeking at state inside active's request whilst
diff --git a/drivers/gpu/drm/i915/intel_lrc.c b/drivers/gpu/drm/i915/intel_lrc.c
index bda66b91aa02..201d649eacd5 100644
--- a/drivers/gpu/drm/i915/intel_lrc.c
+++ b/drivers/gpu/drm/i915/intel_lrc.c
@@ -1944,7 +1944,6 @@ logical_ring_setup(struct drm_i915_private *dev_priv, enum intel_engine_id id)
engine->fw_domains = fw_domains;
- INIT_LIST_HEAD(&engine->active_list);
INIT_LIST_HEAD(&engine->request_list);
INIT_LIST_HEAD(&engine->buffers);
INIT_LIST_HEAD(&engine->execlist_queue);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index 8fddc8d2358b..edbfa0f9b9ed 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2143,7 +2143,6 @@ static int intel_init_engine(struct drm_device *dev,
engine->i915 = dev_priv;
engine->fence_context = fence_context_alloc(1);
- INIT_LIST_HEAD(&engine->active_list);
INIT_LIST_HEAD(&engine->request_list);
INIT_LIST_HEAD(&engine->execlist_queue);
INIT_LIST_HEAD(&engine->buffers);
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.h b/drivers/gpu/drm/i915/intel_ringbuffer.h
index 2622267bf064..e478da6880b4 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.h
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.h
@@ -290,18 +290,6 @@ struct intel_engine_cs {
u32 ctx_desc_template;
/**
- * List of objects currently involved in rendering from the
- * ringbuffer.
- *
- * Includes buffers having the contents of their GPU caches
- * flushed, not necessarily primitives. last_read_req
- * represents when the rendering involved will be completed.
- *
- * A reference is held on the buffer while on this list.
- */
- struct list_head active_list;
-
- /**
* List of breadcrumbs associated with GPU requests currently
* outstanding.
*/
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 56/64] drm/i915: Convert intel_overlay to request tracking
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (53 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 55/64] drm/i915: Refactor activity tracking for requests Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 57/64] drm/i915: Move the special case wait-request handling to its one caller Chris Wilson
` (8 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
intel_overlay already tracks its last flip request, along with action to
take after its completion. Refactor intel_overlay to reuse the common
i915_gem_active tracker.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/intel_overlay.c | 84 ++++++++++++++++--------------------
1 file changed, 36 insertions(+), 48 deletions(-)
diff --git a/drivers/gpu/drm/i915/intel_overlay.c b/drivers/gpu/drm/i915/intel_overlay.c
index 356a1f6f95aa..434772520bce 100644
--- a/drivers/gpu/drm/i915/intel_overlay.c
+++ b/drivers/gpu/drm/i915/intel_overlay.c
@@ -183,8 +183,7 @@ struct intel_overlay {
u32 flip_addr;
struct drm_i915_gem_object *reg_bo;
/* flip handling */
- struct drm_i915_gem_request *last_flip_req;
- void (*flip_tail)(struct intel_overlay *);
+ struct i915_gem_active last_flip;
};
static struct overlay_registers __iomem *
@@ -210,23 +209,26 @@ static void intel_overlay_unmap_regs(struct intel_overlay *overlay,
io_mapping_unmap(regs);
}
-static int intel_overlay_do_wait_request(struct intel_overlay *overlay,
+static void intel_overlay_submit_request(struct intel_overlay *overlay,
struct drm_i915_gem_request *req,
- void (*tail)(struct intel_overlay *))
+ void (*retire)(struct i915_gem_active *,
+ struct drm_i915_gem_request *))
{
- int ret;
-
- WARN_ON(overlay->last_flip_req);
- i915_gem_request_assign(&overlay->last_flip_req, req);
+ GEM_BUG_ON(i915_gem_active_peek(&overlay->last_flip,
+ &overlay->i915->drm.struct_mutex));
+ overlay->last_flip.retire = retire;
+ i915_gem_active_set(&overlay->last_flip, req);
i915_add_request(req);
+}
- overlay->flip_tail = tail;
- ret = i915_wait_request(overlay->last_flip_req);
- if (ret)
- return ret;
-
- i915_gem_request_assign(&overlay->last_flip_req, NULL);
- return 0;
+static int intel_overlay_do_wait_request(struct intel_overlay *overlay,
+ struct drm_i915_gem_request *req,
+ void (*retire)(struct i915_gem_active *,
+ struct drm_i915_gem_request *))
+{
+ intel_overlay_submit_request(overlay, req, retire);
+ return i915_gem_active_retire(&overlay->last_flip,
+ &overlay->i915->drm.struct_mutex);
}
static struct drm_i915_gem_request *alloc_request(struct intel_overlay *overlay)
@@ -306,25 +308,32 @@ static int intel_overlay_continue(struct intel_overlay *overlay,
intel_ring_emit(ring, flip_addr);
intel_ring_advance(ring);
- WARN_ON(overlay->last_flip_req);
- i915_gem_request_assign(&overlay->last_flip_req, req);
- i915_add_request(req);
+ intel_overlay_submit_request(overlay, req, NULL);
return 0;
}
-static void intel_overlay_release_old_vid_tail(struct intel_overlay *overlay)
+static void intel_overlay_release_old_vid_tail(struct i915_gem_active *active,
+ struct drm_i915_gem_request *req)
{
+ struct intel_overlay *overlay =
+ container_of(active, typeof(*overlay), last_flip);
struct drm_i915_gem_object *obj = overlay->old_vid_bo;
+ i915_gem_track_fb(obj, NULL,
+ INTEL_FRONTBUFFER_OVERLAY(overlay->crtc->pipe));
+
i915_gem_object_ggtt_unpin(obj);
i915_gem_object_put(obj);
overlay->old_vid_bo = NULL;
}
-static void intel_overlay_off_tail(struct intel_overlay *overlay)
+static void intel_overlay_off_tail(struct i915_gem_active *active,
+ struct drm_i915_gem_request *req)
{
+ struct intel_overlay *overlay =
+ container_of(active, typeof(*overlay), last_flip);
struct drm_i915_gem_object *obj = overlay->vid_bo;
/* never have the overlay hw on without showing a frame */
@@ -387,27 +396,16 @@ static int intel_overlay_off(struct intel_overlay *overlay)
}
intel_ring_advance(ring);
- return intel_overlay_do_wait_request(overlay, req, intel_overlay_off_tail);
+ return intel_overlay_do_wait_request(overlay, req,
+ intel_overlay_off_tail);
}
/* recover from an interruption due to a signal
* We have to be careful not to repeat work forever an make forward progess. */
static int intel_overlay_recover_from_interrupt(struct intel_overlay *overlay)
{
- int ret;
-
- if (overlay->last_flip_req == NULL)
- return 0;
-
- ret = i915_wait_request(overlay->last_flip_req);
- if (ret)
- return ret;
-
- if (overlay->flip_tail)
- overlay->flip_tail(overlay);
-
- i915_gem_request_assign(&overlay->last_flip_req, NULL);
- return 0;
+ return i915_gem_active_retire(&overlay->last_flip,
+ &overlay->i915->drm.struct_mutex);
}
/* Wait for pending overlay flip and release old frame.
@@ -452,13 +450,9 @@ static int intel_overlay_release_old_vid(struct intel_overlay *overlay)
intel_overlay_release_old_vid_tail);
if (ret)
return ret;
- }
+ } else
+ intel_overlay_release_old_vid_tail(&overlay->last_flip, NULL);
- intel_overlay_release_old_vid_tail(overlay);
-
-
- i915_gem_track_fb(overlay->old_vid_bo, NULL,
- INTEL_FRONTBUFFER_OVERLAY(overlay->crtc->pipe));
return 0;
}
@@ -471,7 +465,6 @@ void intel_overlay_reset(struct drm_i915_private *dev_priv)
intel_overlay_release_old_vid(overlay);
- overlay->last_flip_req = NULL;
overlay->old_xscale = 0;
overlay->old_yscale = 0;
overlay->crtc = NULL;
@@ -882,12 +875,7 @@ int intel_overlay_switch_off(struct intel_overlay *overlay)
iowrite32(0, ®s->OCMD);
intel_overlay_unmap_regs(overlay, regs);
- ret = intel_overlay_off(overlay);
- if (ret != 0)
- return ret;
-
- intel_overlay_off_tail(overlay);
- return 0;
+ return intel_overlay_off(overlay);
}
static int check_overlay_possible_on_crtc(struct intel_overlay *overlay,
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 57/64] drm/i915: Move the special case wait-request handling to its one caller
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (54 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 56/64] drm/i915: Convert intel_overlay to request tracking Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 58/64] drm/i915: Double check activity before relocations Chris Wilson
` (7 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_request.c | 25 -------------------------
drivers/gpu/drm/i915/i915_gem_request.h | 4 ----
drivers/gpu/drm/i915/intel_ringbuffer.c | 18 +++++++++++++-----
3 files changed, 13 insertions(+), 34 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index 6c0447b6e694..c8045f61ada6 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -716,28 +716,3 @@ complete:
return ret;
}
-
-/**
- * Waits for a request to be signaled, and cleans up the
- * request and object lists appropriately for that event.
- */
-int i915_wait_request(struct drm_i915_gem_request *req)
-{
- int ret;
-
- lockdep_assert_held(&req->i915->drm.struct_mutex);
- GEM_BUG_ON(list_empty(&req->link));
-
- ret = __i915_wait_request(req,
- req->i915->mm.interruptible,
- NULL,
- NULL);
- if (ret)
- return ret;
-
- /* If the GPU hung, we want to keep the requests to find the guilty. */
- if (!i915_reset_in_progress(&req->i915->gpu_error))
- i915_gem_request_retire_upto(req);
-
- return 0;
-}
diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
index 8e1c5dc1f3c3..9a1342d968f9 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.h
+++ b/drivers/gpu/drm/i915/i915_gem_request.h
@@ -215,10 +215,6 @@ int __i915_wait_request(struct drm_i915_gem_request *req,
struct intel_rps_client *rps)
__attribute__((nonnull(1)));
-int __must_check
-i915_wait_request(struct drm_i915_gem_request *req)
- __attribute__((nonnull));
-
static inline u32 intel_engine_get_seqno(struct intel_engine_cs *engine);
/**
diff --git a/drivers/gpu/drm/i915/intel_ringbuffer.c b/drivers/gpu/drm/i915/intel_ringbuffer.c
index edbfa0f9b9ed..c0ca0684e079 100644
--- a/drivers/gpu/drm/i915/intel_ringbuffer.c
+++ b/drivers/gpu/drm/i915/intel_ringbuffer.c
@@ -2282,6 +2282,7 @@ static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
struct intel_ring *ring = req->ring;
struct intel_engine_cs *engine = req->engine;
struct drm_i915_gem_request *target;
+ int ret;
intel_ring_update_space(ring);
if (ring->space >= bytes)
@@ -2319,7 +2320,18 @@ static int wait_for_space(struct drm_i915_gem_request *req, int bytes)
if (WARN_ON(&target->link == &engine->request_list))
return -ENOSPC;
- return i915_wait_request(target);
+ ret = __i915_wait_request(target, true, NULL, NULL);
+ if (ret)
+ return ret;
+
+ if (i915_reset_in_progress(&target->i915->gpu_error))
+ return -EAGAIN;
+
+ i915_gem_request_retire_upto(target);
+
+ intel_ring_update_space(ring);
+ GEM_BUG_ON(ring->space < bytes);
+ return 0;
}
int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
@@ -2357,10 +2369,6 @@ int intel_ring_begin(struct drm_i915_gem_request *req, int num_dwords)
int ret = wait_for_space(req, wait_bytes);
if (unlikely(ret))
return ret;
-
- intel_ring_update_space(ring);
- if (unlikely(ring->space < wait_bytes))
- return -EAGAIN;
}
if (unlikely(need_wrap)) {
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 58/64] drm/i915: Double check activity before relocations
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (55 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 57/64] drm/i915: Move the special case wait-request handling to its one caller Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 59/64] drm/i915: Move request list retirement to i915_gem_request.c Chris Wilson
` (6 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
If the object is active and we need to perform a relocation upon it, we
need to take the slow relocation path. Before we do, double check the
active requests to see if they have completed.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 16 +++++++++++++++-
1 file changed, 15 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 22261a511d7d..6e43d96a87ca 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -415,6 +415,20 @@ relocate_entry_clflush(struct drm_i915_gem_object *obj,
return 0;
}
+static bool object_is_idle(struct drm_i915_gem_object *obj)
+{
+ unsigned long active = obj->active;
+ int idx;
+
+ for_each_active(active, idx) {
+ if (!i915_gem_active_is_idle(&obj->last_read[idx],
+ &obj->base.dev->struct_mutex))
+ return false;
+ }
+
+ return true;
+}
+
static int
i915_gem_execbuffer_relocate_entry(struct drm_i915_gem_object *obj,
struct eb_vmas *eb,
@@ -498,7 +512,7 @@ i915_gem_execbuffer_relocate_entry(struct drm_i915_gem_object *obj,
}
/* We can't wait for rendering with pagefaults disabled */
- if (obj->active && pagefault_disabled())
+ if (pagefault_disabled() && !object_is_idle(obj))
return -EFAULT;
if (use_cpu_reloc(obj))
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 59/64] drm/i915: Move request list retirement to i915_gem_request.c
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (56 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 58/64] drm/i915: Double check activity before relocations Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 60/64] drm/i915: i915_vma_move_to_active prep patch Chris Wilson
` (5 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
As the list retirement is now clean of implementation details, we can
move it closer to the request management.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_gem.c | 44 ---------------------------------
drivers/gpu/drm/i915/i915_gem_request.c | 35 ++++++++++++++++++++++++++
2 files changed, 35 insertions(+), 44 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index a38939dab741..ba6cc3642862 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2538,50 +2538,6 @@ void i915_gem_reset(struct drm_device *dev)
i915_gem_restore_fences(dev);
}
-/**
- * This function clears the request list as sequence numbers are passed.
- * @engine: engine to retire requests on
- */
-void
-i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
-{
- while (!list_empty(&engine->request_list)) {
- struct drm_i915_gem_request *request;
-
- request = list_first_entry(&engine->request_list,
- struct drm_i915_gem_request,
- link);
-
- if (!i915_gem_request_completed(request))
- break;
-
- i915_gem_request_retire_upto(request);
- }
-}
-
-void i915_gem_retire_requests(struct drm_i915_private *dev_priv)
-{
- struct intel_engine_cs *engine;
-
- lockdep_assert_held(&dev_priv->drm.struct_mutex);
-
- if (dev_priv->gt.active_engines == 0)
- return;
-
- GEM_BUG_ON(!dev_priv->gt.awake);
-
- for_each_engine(engine, dev_priv) {
- i915_gem_retire_requests_ring(engine);
- if (list_empty(&engine->request_list))
- dev_priv->gt.active_engines &= ~intel_engine_flag(engine);
- }
-
- if (dev_priv->gt.active_engines == 0)
- queue_delayed_work(dev_priv->wq,
- &dev_priv->gt.idle_work,
- msecs_to_jiffies(100));
-}
-
static void
i915_gem_retire_work_handler(struct work_struct *work)
{
diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
index c8045f61ada6..04d805f66844 100644
--- a/drivers/gpu/drm/i915/i915_gem_request.c
+++ b/drivers/gpu/drm/i915/i915_gem_request.c
@@ -716,3 +716,38 @@ complete:
return ret;
}
+
+void i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
+{
+ struct drm_i915_gem_request *request, *next;
+
+ list_for_each_entry_safe(request, next, &engine->request_list, link) {
+ if (!i915_gem_request_completed(request))
+ break;
+
+ i915_gem_request_retire(request);
+ }
+}
+
+void i915_gem_retire_requests(struct drm_i915_private *dev_priv)
+{
+ struct intel_engine_cs *engine;
+
+ lockdep_assert_held(&dev_priv->drm.struct_mutex);
+
+ if (dev_priv->gt.active_engines == 0)
+ return;
+
+ GEM_BUG_ON(!dev_priv->gt.awake);
+
+ for_each_engine(engine, dev_priv) {
+ i915_gem_retire_requests_ring(engine);
+ if (list_empty(&engine->request_list))
+ dev_priv->gt.active_engines &= ~intel_engine_flag(engine);
+ }
+
+ if (dev_priv->gt.active_engines == 0)
+ queue_delayed_work(dev_priv->wq,
+ &dev_priv->gt.idle_work,
+ msecs_to_jiffies(100));
+}
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 60/64] drm/i915: i915_vma_move_to_active prep patch
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (57 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 59/64] drm/i915: Move request list retirement to i915_gem_request.c Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 61/64] drm/i915: Track active vma requests Chris Wilson
` (4 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
This patch is broken out of the next just to remove the code motion from
that patch and make it more readable. What we do here is move the
i915_vma_move_to_active() to i915_gem_execbuffer.c and put the three
stages (read, write, fenced) together so that future modifications to
active handling are all located in the same spot. The importance of this
is so that we can more simply control the order in which the requests
are place in the retirement list (i.e. control the order at which we
retire and so control the lifetimes to avoid having to hold onto
references).
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 3 +-
drivers/gpu/drm/i915/i915_gem.c | 18 -------
drivers/gpu/drm/i915/i915_gem_context.c | 9 ++--
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 72 ++++++++++++++++++----------
drivers/gpu/drm/i915/i915_gem_render_state.c | 2 +-
5 files changed, 55 insertions(+), 49 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index fa42cb14ed9b..42f2ec097e7c 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3133,7 +3133,8 @@ int __must_check i915_mutex_lock_interruptible(struct drm_device *dev);
int i915_gem_object_sync(struct drm_i915_gem_object *obj,
struct drm_i915_gem_request *to);
void i915_vma_move_to_active(struct i915_vma *vma,
- struct drm_i915_gem_request *req);
+ struct drm_i915_gem_request *req,
+ unsigned int flags);
int i915_gem_dumb_create(struct drm_file *file_priv,
struct drm_device *dev,
struct drm_mode_create_dumb *args);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index ba6cc3642862..08623d8d22dd 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2330,24 +2330,6 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj)
return obj->mapping;
}
-void i915_vma_move_to_active(struct i915_vma *vma,
- struct drm_i915_gem_request *req)
-{
- struct drm_i915_gem_object *obj = vma->obj;
- struct intel_engine_cs *engine;
-
- engine = i915_gem_request_get_engine(req);
-
- /* Add a reference if we're newly entering the active list. */
- if (obj->active == 0)
- i915_gem_object_get(obj);
- obj->active |= intel_engine_flag(engine);
-
- i915_gem_active_set(&obj->last_read[engine->id], req);
-
- list_move_tail(&vma->vm_link, &vma->vm->active_list);
-}
-
static void
i915_gem_object_retire__fence(struct i915_gem_active *active,
struct drm_i915_gem_request *req)
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index c213fafe41d9..ce3de01f4a6e 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -815,8 +815,8 @@ static int do_rcs_switch(struct drm_i915_gem_request *req)
* MI_SET_CONTEXT instead of when the next seqno has completed.
*/
if (from != NULL) {
- from->engine[RCS].state->base.read_domains = I915_GEM_DOMAIN_INSTRUCTION;
- i915_vma_move_to_active(i915_gem_obj_to_ggtt(from->engine[RCS].state), req);
+ struct drm_i915_gem_object *obj = from->engine[RCS].state;
+
/* As long as MI_SET_CONTEXT is serializing, ie. it flushes the
* whole damn pipeline, we don't need to explicitly mark the
* object dirty. The only exception is that the context must be
@@ -824,10 +824,11 @@ static int do_rcs_switch(struct drm_i915_gem_request *req)
* able to defer doing this until we know the object would be
* swapped, but there is no way to do that yet.
*/
- from->engine[RCS].state->dirty = 1;
+ obj->base.read_domains = I915_GEM_DOMAIN_INSTRUCTION;
+ i915_vma_move_to_active(i915_gem_obj_to_ggtt(obj), req, 0);
/* obj is kept alive until the next request by its active ref */
- i915_gem_object_ggtt_unpin(from->engine[RCS].state);
+ i915_gem_object_ggtt_unpin(obj);
i915_gem_context_put(from);
}
engine->last_context = i915_gem_context_get(to);
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 6e43d96a87ca..9b6befa17655 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1114,8 +1114,48 @@ i915_gem_validate_context(struct drm_device *dev, struct drm_file *file,
return ctx;
}
+void i915_vma_move_to_active(struct i915_vma *vma,
+ struct drm_i915_gem_request *req,
+ unsigned int flags)
+{
+ struct drm_i915_gem_object *obj = vma->obj;
+ const unsigned int idx = req->engine->id;
+
+ GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
+
+ obj->dirty = 1; /* be paranoid */
+
+ /* Add a reference if we're newly entering the active list. */
+ if (obj->active == 0)
+ i915_gem_object_get(obj);
+ obj->active |= 1 << idx;
+ i915_gem_active_set(&obj->last_read[idx], req);
+
+ if (flags & EXEC_OBJECT_WRITE) {
+ i915_gem_active_set(&obj->last_write, req);
+
+ intel_fb_obj_invalidate(obj, ORIGIN_CS);
+
+ /* update for the implicit flush after a batch */
+ obj->base.write_domain &= ~I915_GEM_GPU_DOMAINS;
+ }
+
+ if (flags & EXEC_OBJECT_NEEDS_FENCE) {
+ i915_gem_active_set(&obj->last_fence, req);
+ if (flags & __EXEC_OBJECT_HAS_FENCE) {
+ struct drm_i915_private *dev_priv = req->i915;
+
+ list_move_tail(&dev_priv->fence_regs[obj->fence_reg].lru_list,
+ &dev_priv->mm.fence_list);
+ }
+ }
+
+ list_move_tail(&vma->vm_link, &vma->vm->active_list);
+}
+
static void eb_export_fence(struct drm_i915_gem_object *obj,
- struct drm_i915_gem_request *req)
+ struct drm_i915_gem_request *req,
+ unsigned int flags)
{
struct reservation_object *resv;
@@ -1128,7 +1168,7 @@ static void eb_export_fence(struct drm_i915_gem_object *obj,
* synchronisation leading to rendering corruption.
*/
mutex_lock(&resv->lock.base);
- if (obj->base.write_domain)
+ if (flags & EXEC_OBJECT_WRITE)
reservation_object_add_excl_fence(resv, &req->fence);
else if (reservation_object_reserve_shared(resv) == 0)
reservation_object_add_shared_fence(resv, &req->fence);
@@ -1139,40 +1179,22 @@ static void
i915_gem_execbuffer_move_to_active(struct list_head *vmas,
struct drm_i915_gem_request *req)
{
- struct intel_engine_cs *engine = i915_gem_request_get_engine(req);
struct i915_vma *vma;
list_for_each_entry(vma, vmas, exec_list) {
- struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
struct drm_i915_gem_object *obj = vma->obj;
u32 old_read = obj->base.read_domains;
u32 old_write = obj->base.write_domain;
- obj->dirty = 1; /* be paranoid */
obj->base.write_domain = obj->base.pending_write_domain;
- if (obj->base.write_domain == 0)
+ if (obj->base.write_domain)
+ vma->exec_entry->flags |= EXEC_OBJECT_WRITE;
+ else
obj->base.pending_read_domains |= obj->base.read_domains;
obj->base.read_domains = obj->base.pending_read_domains;
- i915_vma_move_to_active(vma, req);
- eb_export_fence(obj, req);
- if (obj->base.write_domain) {
- i915_gem_active_set(&obj->last_write, req);
-
- intel_fb_obj_invalidate(obj, ORIGIN_CS);
-
- /* update for the implicit flush after a batch */
- obj->base.write_domain &= ~I915_GEM_GPU_DOMAINS;
- }
- if (entry->flags & EXEC_OBJECT_NEEDS_FENCE) {
- i915_gem_active_set(&obj->last_fence, req);
- if (entry->flags & __EXEC_OBJECT_HAS_FENCE) {
- struct drm_i915_private *dev_priv = engine->i915;
- list_move_tail(&dev_priv->fence_regs[obj->fence_reg].lru_list,
- &dev_priv->mm.fence_list);
- }
- }
-
+ i915_vma_move_to_active(vma, req, vma->exec_entry->flags);
+ eb_export_fence(obj, req, vma->exec_entry->flags);
trace_i915_gem_object_change_domain(obj, old_read, old_write);
}
}
diff --git a/drivers/gpu/drm/i915/i915_gem_render_state.c b/drivers/gpu/drm/i915/i915_gem_render_state.c
index dd84793d75d9..2165cd604691 100644
--- a/drivers/gpu/drm/i915/i915_gem_render_state.c
+++ b/drivers/gpu/drm/i915/i915_gem_render_state.c
@@ -260,7 +260,7 @@ int i915_gem_render_state_init(struct drm_i915_gem_request *req)
goto out;
}
- i915_vma_move_to_active(i915_gem_obj_to_ggtt(so.obj), req);
+ i915_vma_move_to_active(i915_gem_obj_to_ggtt(so.obj), req, 0);
out:
render_state_fini(&so);
return ret;
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 61/64] drm/i915: Track active vma requests
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (58 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 60/64] drm/i915: i915_vma_move_to_active prep patch Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 62/64] drm/i915: Release vma when the handle is closed Chris Wilson
` (3 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
Hook the vma itself into the i915_gem_request_retire() so that we can
accurately track when a solitary vma is inactive (as opposed to having
to wait for the entire object to be idle). This improves the interaction
when using multiple contexts (with full-ppgtt) and eliminates some
frequent list walking when retiring objects after a completed request.
A side-effect is that we get an active vma reference for free. The
consequence of this is shown in the next patch...
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_debugfs.c | 2 +-
drivers/gpu/drm/i915/i915_gem.c | 50 +++++++++++++++++-------------
drivers/gpu/drm/i915/i915_gem_execbuffer.c | 10 +++++-
drivers/gpu/drm/i915/i915_gem_gtt.c | 20 ++++++++++++
drivers/gpu/drm/i915/i915_gem_gtt.h | 28 +++++++++++++++++
5 files changed, 86 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
index 7fdef8058d31..c3fe61f3da2e 100644
--- a/drivers/gpu/drm/i915/i915_debugfs.c
+++ b/drivers/gpu/drm/i915/i915_debugfs.c
@@ -367,7 +367,7 @@ static int per_file_stats(int id, void *ptr, void *data)
continue;
}
- if (obj->active) /* XXX per-vma statistic */
+ if (i915_vma_is_active(vma))
stats->active += vma->node.size;
else
stats->inactive += vma->node.size;
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 08623d8d22dd..4ebed1eec769 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2354,7 +2354,6 @@ i915_gem_object_retire__read(struct i915_gem_active *active,
int ring = request->engine->id;
struct drm_i915_gem_object *obj =
container_of(active, struct drm_i915_gem_object, last_read[ring]);
- struct i915_vma *vma;
GEM_BUG_ON((obj->active & (1 << ring)) == 0);
@@ -2366,12 +2365,9 @@ i915_gem_object_retire__read(struct i915_gem_active *active,
* so that we don't steal from recently used but inactive objects
* (unless we are forced to ofc!)
*/
- list_move_tail(&obj->global_list, &request->i915->mm.bound_list);
-
- list_for_each_entry(vma, &obj->vma_list, obj_link) {
- if (!list_empty(&vma->vm_link))
- list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
- }
+ if (obj->bind_count)
+ list_move_tail(&obj->global_list,
+ &request->i915->mm.bound_list);
i915_gem_object_put(obj);
}
@@ -2796,8 +2792,29 @@ static void __i915_vma_iounmap(struct i915_vma *vma)
static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
{
struct drm_i915_gem_object *obj = vma->obj;
+ unsigned long active;
int ret;
+ /* First wait upon any activity as retiring the request may
+ * have side-effects such as unpinning or even unbinding this vma.
+ */
+ active = vma->active;
+ if (active && wait) {
+ int idx;
+
+ for_each_active(active, idx) {
+ ret = i915_gem_active_retire(&vma->last_read[idx],
+ &vma->vm->dev->struct_mutex);
+ if (ret)
+ return ret;
+ }
+
+ GEM_BUG_ON(i915_vma_is_active(vma));
+ }
+
+ if (vma->pin_count)
+ return -EBUSY;
+
if (list_empty(&vma->obj_link))
return 0;
@@ -2806,18 +2823,9 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
return 0;
}
- if (vma->pin_count)
- return -EBUSY;
-
GEM_BUG_ON(obj->bind_count == 0);
GEM_BUG_ON(!obj->pages);
- if (wait) {
- ret = i915_gem_object_wait_rendering(obj, false);
- if (ret)
- return ret;
- }
-
if (vma->is_ggtt && vma->ggtt_view.type == I915_GGTT_VIEW_NORMAL) {
i915_gem_object_finish_gtt(obj);
@@ -3200,9 +3208,6 @@ i915_gem_object_flush_cpu_write_domain(struct drm_i915_gem_object *obj)
int
i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
{
- struct drm_device *dev = obj->base.dev;
- struct drm_i915_private *dev_priv = to_i915(dev);
- struct i915_ggtt *ggtt = &dev_priv->ggtt;
uint32_t old_write_domain, old_read_domains;
struct i915_vma *vma;
int ret;
@@ -3255,9 +3260,10 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
/* And bump the LRU for this access */
vma = i915_gem_obj_to_ggtt(obj);
- if (vma && drm_mm_node_allocated(&vma->node) && !obj->active)
- list_move_tail(&vma->vm_link,
- &ggtt->base.inactive_list);
+ if (vma &&
+ drm_mm_node_allocated(&vma->node) &&
+ !i915_vma_is_active(vma))
+ list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
return 0;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
index 9b6befa17655..b25ca42df901 100644
--- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
+++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
@@ -1125,7 +1125,13 @@ void i915_vma_move_to_active(struct i915_vma *vma,
obj->dirty = 1; /* be paranoid */
- /* Add a reference if we're newly entering the active list. */
+ /* Add a reference if we're newly entering the active list.
+ * The order in which we add operations to the retirement queue is
+ * vital here: mark_active adds to the start of the callback list,
+ * such that subsequent callbacks are called first. Therefore we
+ * add the active reference first and queue for it to be dropped
+ * *last*.
+ */
if (obj->active == 0)
i915_gem_object_get(obj);
obj->active |= 1 << idx;
@@ -1150,6 +1156,8 @@ void i915_vma_move_to_active(struct i915_vma *vma,
}
}
+ i915_vma_set_active(vma, idx);
+ i915_gem_active_set(&vma->last_read[idx], req);
list_move_tail(&vma->vm_link, &vma->vm->active_list);
}
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index c25117bed1e9..72d2812bbbf5 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -3325,12 +3325,30 @@ void i915_gem_restore_gtt_mappings(struct drm_device *dev)
i915_ggtt_flush(dev_priv);
}
+static void
+i915_vma_retire(struct i915_gem_active *active,
+ struct drm_i915_gem_request *rq)
+{
+ const unsigned int idx = rq->engine->id;
+ struct i915_vma *vma =
+ container_of(active, struct i915_vma, last_read[idx]);
+
+ GEM_BUG_ON(!i915_vma_has_active_engine(vma, idx));
+
+ i915_vma_unset_active(vma, idx);
+ if (i915_vma_is_active(vma))
+ return;
+
+ list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
+}
+
static struct i915_vma *
__i915_gem_vma_create(struct drm_i915_gem_object *obj,
struct i915_address_space *vm,
const struct i915_ggtt_view *ggtt_view)
{
struct i915_vma *vma;
+ int i;
if (WARN_ON(i915_is_ggtt(vm) != !!ggtt_view))
return ERR_PTR(-EINVAL);
@@ -3342,6 +3360,8 @@ __i915_gem_vma_create(struct drm_i915_gem_object *obj,
INIT_LIST_HEAD(&vma->vm_link);
INIT_LIST_HEAD(&vma->obj_link);
INIT_LIST_HEAD(&vma->exec_list);
+ for (i = 0; i < ARRAY_SIZE(vma->last_read); i++)
+ init_request_active(&vma->last_read[i], i915_vma_retire);
vma->vm = vm;
vma->obj = obj;
vma->is_ggtt = i915_is_ggtt(vm);
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index 33cc010e90f5..7efe135c0944 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -36,6 +36,8 @@
#include <linux/io-mapping.h>
+#include "i915_gem_request.h"
+
struct drm_i915_file_private;
typedef uint32_t gen6_pte_t;
@@ -179,10 +181,13 @@ struct i915_vma {
struct i915_address_space *vm;
void __iomem *iomap;
+ struct i915_gem_active last_read[I915_NUM_ENGINES];
+
/** Flags and address space this VMA is bound to */
#define GLOBAL_BIND (1<<0)
#define LOCAL_BIND (1<<1)
unsigned int bound : 4;
+ unsigned int active : I915_NUM_ENGINES;
bool is_ggtt : 1;
/**
@@ -222,6 +227,29 @@ struct i915_vma {
#define DRM_I915_GEM_OBJECT_MAX_PIN_COUNT 0xf
};
+static inline bool i915_vma_is_active(const struct i915_vma *vma)
+{
+ return vma->active;
+}
+
+static inline void i915_vma_set_active(struct i915_vma *vma,
+ unsigned int engine)
+{
+ vma->active |= 1 << engine;
+}
+
+static inline void i915_vma_unset_active(struct i915_vma *vma,
+ unsigned int engine)
+{
+ vma->active &= ~(1 << engine);
+}
+
+static inline bool i915_vma_has_active_engine(const struct i915_vma *vma,
+ unsigned int engine)
+{
+ return vma->active & (1 << engine);
+}
+
struct i915_page_dma {
struct page *page;
union {
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 62/64] drm/i915: Release vma when the handle is closed
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (59 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 61/64] drm/i915: Track active vma requests Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 63/64] drm/i915: Mark the context and address space as closed Chris Wilson
` (2 subsequent siblings)
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
In order to prevent a leak of the vma on shared objects, we need to
hook into the object_close callback to destroy the vma on the object for
this file. However, if we destroyed that vma immediately we may cause
unexpected application stalls as we try to unbind a busy vma - hence we
defer the unbind to when we retire the vma.
v2: Keep vma allocated until closed. This is useful for a later
optimisation, but it is required now in order to handle potential
recursion of i915_vma_unbind() by retiring itself.
v3: Comments are important.
Testcase: igt/gem_ppggtt/flink-and-close-vma-leak
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Daniele Ceraolo Spurio <daniele.ceraolospurio@intel.com>
---
drivers/gpu/drm/i915/i915_drv.c | 1 +
drivers/gpu/drm/i915/i915_drv.h | 4 +-
drivers/gpu/drm/i915/i915_gem.c | 88 ++++++++++++++++++-----------------
drivers/gpu/drm/i915/i915_gem_evict.c | 8 +---
drivers/gpu/drm/i915/i915_gem_gtt.c | 25 ++++++++++
drivers/gpu/drm/i915/i915_gem_gtt.h | 1 +
6 files changed, 77 insertions(+), 50 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.c b/drivers/gpu/drm/i915/i915_drv.c
index 21ef73e056b2..326493de30a2 100644
--- a/drivers/gpu/drm/i915/i915_drv.c
+++ b/drivers/gpu/drm/i915/i915_drv.c
@@ -2572,6 +2572,7 @@ static struct drm_driver driver = {
.postclose = i915_driver_postclose,
.set_busid = drm_pci_set_busid,
+ .gem_close_object = i915_gem_close_object,
.gem_free_object = i915_gem_free_object,
.gem_vm_ops = &i915_gem_vm_ops,
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index 42f2ec097e7c..d4165cb388de 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -2994,8 +2994,8 @@ struct drm_i915_gem_object *i915_gem_object_create(struct drm_device *dev,
size_t size);
struct drm_i915_gem_object *i915_gem_object_create_from_data(
struct drm_device *dev, const void *data, size_t size);
+void i915_gem_close_object(struct drm_gem_object *gem, struct drm_file *file);
void i915_gem_free_object(struct drm_gem_object *obj);
-void i915_gem_vma_destroy(struct i915_vma *vma);
/* Flags used by pin/bind&friends. */
#define PIN_MAPPABLE (1<<0)
@@ -3028,6 +3028,8 @@ int __must_check i915_vma_unbind(struct i915_vma *vma);
* _guarantee_ VMA in question is _not in use_ anywhere.
*/
int __must_check __i915_vma_unbind_no_wait(struct i915_vma *vma);
+void i915_vma_close(struct i915_vma *vma);
+void i915_vma_destroy(struct i915_vma *vma);
int i915_gem_object_unbind(struct drm_i915_gem_object *obj);
int i915_gem_object_put_pages(struct drm_i915_gem_object *obj);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 4ebed1eec769..0813ef41a1d1 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2595,6 +2595,19 @@ out_rearm:
}
}
+void i915_gem_close_object(struct drm_gem_object *gem, struct drm_file *file)
+{
+ struct drm_i915_gem_object *obj = to_intel_bo(gem);
+ struct drm_i915_file_private *fpriv = file->driver_priv;
+ struct i915_vma *vma, *vn;
+
+ mutex_lock(&obj->base.dev->struct_mutex);
+ list_for_each_entry_safe(vma, vn, &obj->vma_list, obj_link)
+ if (vma->vm->file == fpriv)
+ i915_vma_close(vma);
+ mutex_unlock(&obj->base.dev->struct_mutex);
+}
+
/**
* i915_gem_wait_ioctl - implements DRM_IOCTL_I915_GEM_WAIT
* @dev: drm device pointer
@@ -2802,26 +2815,32 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
if (active && wait) {
int idx;
+ /* When a closed VMA is retired, it is unbound - eek.
+ * In order to prevent it from being recursively closed,
+ * take a pin on the vma so that the second unbind is
+ * aborted.
+ */
+ vma->pin_count++;
+
for_each_active(active, idx) {
ret = i915_gem_active_retire(&vma->last_read[idx],
&vma->vm->dev->struct_mutex);
if (ret)
- return ret;
+ break;
}
+ vma->pin_count--;
+ if (ret)
+ return ret;
+
GEM_BUG_ON(i915_vma_is_active(vma));
}
if (vma->pin_count)
return -EBUSY;
- if (list_empty(&vma->obj_link))
- return 0;
-
- if (!drm_mm_node_allocated(&vma->node)) {
- i915_gem_vma_destroy(vma);
- return 0;
- }
+ if (!drm_mm_node_allocated(&vma->node))
+ goto destroy;
GEM_BUG_ON(obj->bind_count == 0);
GEM_BUG_ON(!obj->pages);
@@ -2854,7 +2873,6 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
}
drm_mm_remove_node(&vma->node);
- i915_gem_vma_destroy(vma);
/* Since the unbound list is global, only move to that list if
* no more VMAs exist. */
@@ -2868,6 +2886,10 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
*/
i915_gem_object_unpin_pages(obj);
+destroy:
+ if (unlikely(vma->closed))
+ i915_vma_destroy(vma);
+
return 0;
}
@@ -3042,7 +3064,7 @@ i915_gem_object_bind_to_vm(struct drm_i915_gem_object *obj,
if (offset & (alignment - 1) || offset + size > end) {
ret = -EINVAL;
- goto err_free_vma;
+ goto err_vma;
}
vma->node.start = offset;
vma->node.size = size;
@@ -3054,7 +3076,7 @@ i915_gem_object_bind_to_vm(struct drm_i915_gem_object *obj,
ret = drm_mm_reserve_node(&vm->mm, &vma->node);
}
if (ret)
- goto err_free_vma;
+ goto err_vma;
} else {
if (flags & PIN_HIGH) {
search_flag = DRM_MM_SEARCH_BELOW;
@@ -3079,7 +3101,7 @@ search_free:
if (ret == 0)
goto search_free;
- goto err_free_vma;
+ goto err_vma;
}
}
if (WARN_ON(!i915_gem_valid_gtt_space(vma, obj->cache_level))) {
@@ -3100,8 +3122,7 @@ search_free:
err_remove_node:
drm_mm_remove_node(&vma->node);
-err_free_vma:
- i915_gem_vma_destroy(vma);
+err_vma:
vma = ERR_PTR(ret);
err_unpin:
i915_gem_object_unpin_pages(obj);
@@ -4055,21 +4076,18 @@ void i915_gem_free_object(struct drm_gem_object *gem_obj)
trace_i915_gem_object_destroy(obj);
+ /* All file-owned VMA should have been released by this point through
+ * i915_gem_close_object(), or earlier by i915_gem_context_close().
+ * However, the object may also be bound into the global GTT (e.g.
+ * older GPUs without per-process support, or for direct access through
+ * the GTT either for the user or for scanout). Those VMA still need to
+ * unbound now.
+ */
list_for_each_entry_safe(vma, next, &obj->vma_list, obj_link) {
- int ret;
-
+ GEM_BUG_ON(!vma->is_ggtt);
+ GEM_BUG_ON(i915_vma_is_active(vma));
vma->pin_count = 0;
- ret = i915_vma_unbind(vma);
- if (WARN_ON(ret == -ERESTARTSYS)) {
- bool was_interruptible;
-
- was_interruptible = dev_priv->mm.interruptible;
- dev_priv->mm.interruptible = false;
-
- WARN_ON(i915_vma_unbind(vma));
-
- dev_priv->mm.interruptible = was_interruptible;
- }
+ i915_vma_close(vma);
}
GEM_BUG_ON(obj->bind_count);
@@ -4134,22 +4152,6 @@ struct i915_vma *i915_gem_obj_to_ggtt_view(struct drm_i915_gem_object *obj,
return NULL;
}
-void i915_gem_vma_destroy(struct i915_vma *vma)
-{
- WARN_ON(vma->node.allocated);
-
- /* Keep the vma as a placeholder in the execbuffer reservation lists */
- if (!list_empty(&vma->exec_list))
- return;
-
- if (!vma->is_ggtt)
- i915_ppgtt_put(i915_vm_to_ppgtt(vma->vm));
-
- list_del(&vma->obj_link);
-
- kmem_cache_free(to_i915(vma->obj->base.dev)->vmas, vma);
-}
-
static void
i915_gem_stop_engines(struct drm_device *dev)
{
diff --git a/drivers/gpu/drm/i915/i915_gem_evict.c b/drivers/gpu/drm/i915/i915_gem_evict.c
index 5a02c32e9ae6..2a9adc802e85 100644
--- a/drivers/gpu/drm/i915/i915_gem_evict.c
+++ b/drivers/gpu/drm/i915/i915_gem_evict.c
@@ -213,8 +213,8 @@ found:
struct i915_vma,
exec_list);
if (drm_mm_scan_remove_block(&vma->node)) {
+ vma->pin_count++;
list_move(&vma->exec_list, &eviction_list);
- i915_gem_object_get(vma->obj);
continue;
}
list_del_init(&vma->exec_list);
@@ -222,18 +222,14 @@ found:
/* Unbinding will emit any required flushes */
while (!list_empty(&eviction_list)) {
- struct drm_i915_gem_object *obj;
-
vma = list_first_entry(&eviction_list,
struct i915_vma,
exec_list);
- obj = vma->obj;
list_del_init(&vma->exec_list);
+ vma->pin_count--;
if (ret == 0)
ret = i915_vma_unbind(vma);
-
- i915_gem_object_put(obj);
}
return ret;
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 72d2812bbbf5..db47b7588a83 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -3340,6 +3340,31 @@ i915_vma_retire(struct i915_gem_active *active,
return;
list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
+ if (unlikely(vma->closed && !vma->pin_count))
+ WARN_ON(i915_vma_unbind(vma));
+}
+
+void i915_vma_destroy(struct i915_vma *vma)
+{
+ GEM_BUG_ON(vma->node.allocated);
+ GEM_BUG_ON(i915_vma_is_active(vma));
+ GEM_BUG_ON(!vma->closed);
+
+ list_del(&vma->vm_link);
+ if (!vma->is_ggtt)
+ i915_ppgtt_put(i915_vm_to_ppgtt(vma->vm));
+
+ kmem_cache_free(to_i915(vma->obj->base.dev)->vmas, vma);
+}
+
+void i915_vma_close(struct i915_vma *vma)
+{
+ GEM_BUG_ON(vma->closed);
+ vma->closed = true;
+
+ list_del_init(&vma->obj_link);
+ if (!i915_vma_is_active(vma) && !vma->pin_count)
+ WARN_ON(i915_vma_unbind(vma));
}
static struct i915_vma *
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index 7efe135c0944..a4f491dc1dae 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -189,6 +189,7 @@ struct i915_vma {
unsigned int bound : 4;
unsigned int active : I915_NUM_ENGINES;
bool is_ggtt : 1;
+ bool closed : 1;
/**
* Support different GGTT views into the same object.
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 63/64] drm/i915: Mark the context and address space as closed
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (60 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 62/64] drm/i915: Release vma when the handle is closed Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 8:42 ` [PATCH 64/64] Revert "drm/i915: Clean up associated VMAs on context destruction" Chris Wilson
2016-07-07 9:12 ` ✗ Ro.CI.BAT: warning for series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Patchwork
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
When the user closes the context mark it and the dependent address space
as closed. As we use an asynchronous destruct method, this has two
purposes. First it allows us to flag the closed context and detect
internal errors if we to create any new objects for it (as it is removed
from the user's namespace, these should be internal bugs only). And
secondly, it allows us to immediately reap stale vma.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 1 +
drivers/gpu/drm/i915/i915_gem.c | 15 ++++++------
drivers/gpu/drm/i915/i915_gem_context.c | 43 ++++++++++++++++++++++++++++-----
drivers/gpu/drm/i915/i915_gem_gtt.c | 9 +++++--
drivers/gpu/drm/i915/i915_gem_gtt.h | 9 +++++++
drivers/gpu/drm/i915/i915_gem_stolen.c | 2 +-
6 files changed, 63 insertions(+), 16 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index d4165cb388de..b465fc4addb2 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -904,6 +904,7 @@ struct i915_gem_context {
struct list_head link;
u8 remap_slice;
+ bool closed:1;
};
enum fb_op_origin {
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index 0813ef41a1d1..d6ca6ecec37f 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2856,12 +2856,15 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
__i915_vma_iounmap(vma);
}
- trace_i915_vma_unbind(vma);
-
- vma->vm->unbind_vma(vma);
+ if (likely(!vma->vm->closed)) {
+ trace_i915_vma_unbind(vma);
+ vma->vm->unbind_vma(vma);
+ }
vma->bound = 0;
- list_del_init(&vma->vm_link);
+ drm_mm_remove_node(&vma->node);
+ list_move_tail(&vma->vm_link, &vma->vm->unbound_list);
+
if (vma->is_ggtt) {
if (vma->ggtt_view.type == I915_GGTT_VIEW_NORMAL) {
obj->map_and_fenceable = false;
@@ -2872,8 +2875,6 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
vma->ggtt_view.pages = NULL;
}
- drm_mm_remove_node(&vma->node);
-
/* Since the unbound list is global, only move to that list if
* no more VMAs exist. */
if (--obj->bind_count == 0)
@@ -3115,7 +3116,7 @@ search_free:
goto err_remove_node;
list_move_tail(&obj->global_list, &dev_priv->mm.bound_list);
- list_add_tail(&vma->vm_link, &vm->inactive_list);
+ list_move_tail(&vma->vm_link, &vm->inactive_list);
obj->bind_count++;
return vma;
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index ce3de01f4a6e..0e1c4a696c6c 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -156,6 +156,7 @@ void i915_gem_context_free(struct kref *ctx_ref)
lockdep_assert_held(&ctx->i915->drm.struct_mutex);
trace_i915_context_free(ctx);
+ GEM_BUG_ON(!ctx->closed);
/*
* This context is going away and we need to remove all VMAs still
@@ -224,6 +225,37 @@ i915_gem_alloc_context_obj(struct drm_device *dev, size_t size)
return obj;
}
+static void i915_ppgtt_close(struct i915_address_space *vm)
+{
+ struct list_head *phases[] = {
+ &vm->active_list,
+ &vm->inactive_list,
+ &vm->unbound_list,
+ NULL,
+ }, **phase;
+
+ GEM_BUG_ON(vm->closed);
+ vm->closed = true;
+
+ for (phase = phases; *phase; phase++) {
+ struct i915_vma *vma, *vn;
+
+ list_for_each_entry_safe(vma, vn, *phase, vm_link)
+ if (!vma->closed)
+ i915_vma_close(vma);
+ }
+}
+
+static void context_close(struct i915_gem_context *ctx)
+{
+ GEM_BUG_ON(ctx->closed);
+ ctx->closed = true;
+ if (ctx->ppgtt)
+ i915_ppgtt_close(&ctx->ppgtt->base);
+ ctx->file_priv = ERR_PTR(-EBADF);
+ i915_gem_context_put(ctx);
+}
+
static int assign_hw_id(struct drm_i915_private *dev_priv, unsigned *out)
{
int ret;
@@ -305,7 +337,7 @@ __create_hw_context(struct drm_device *dev,
return ctx;
err_out:
- i915_gem_context_put(ctx);
+ context_close(ctx);
return ERR_PTR(ret);
}
@@ -334,7 +366,7 @@ i915_gem_create_context(struct drm_device *dev,
DRM_DEBUG_DRIVER("PPGTT setup failed (%ld)\n",
PTR_ERR(ppgtt));
idr_remove(&file_priv->context_idr, ctx->user_handle);
- i915_gem_context_put(ctx);
+ context_close(ctx);
return ERR_CAST(ppgtt);
}
@@ -505,7 +537,7 @@ void i915_gem_context_fini(struct drm_device *dev)
lockdep_assert_held(&dev->struct_mutex);
- i915_gem_context_put(dctx);
+ context_close(dctx);
dev_priv->kernel_context = NULL;
ida_destroy(&dev_priv->context_hw_ida);
@@ -515,8 +547,7 @@ static int context_idr_cleanup(int id, void *p, void *data)
{
struct i915_gem_context *ctx = p;
- ctx->file_priv = ERR_PTR(-EBADF);
- i915_gem_context_put(ctx);
+ context_close(ctx);
return 0;
}
@@ -986,7 +1017,7 @@ int i915_gem_context_destroy_ioctl(struct drm_device *dev, void *data,
}
idr_remove(&file_priv->context_idr, ctx->user_handle);
- i915_gem_context_put(ctx);
+ context_close(ctx);
mutex_unlock(&dev->struct_mutex);
DRM_DEBUG_DRIVER("HW context %d destroyed\n", args->ctx_id);
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index db47b7588a83..ee239f8c567f 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -2125,6 +2125,7 @@ static void i915_address_space_init(struct i915_address_space *vm,
vm->dev = &dev_priv->drm;
INIT_LIST_HEAD(&vm->active_list);
INIT_LIST_HEAD(&vm->inactive_list);
+ INIT_LIST_HEAD(&vm->unbound_list);
list_add_tail(&vm->global_link, &dev_priv->vm_list);
}
@@ -2217,9 +2218,10 @@ void i915_ppgtt_release(struct kref *kref)
trace_i915_ppgtt_release(&ppgtt->base);
- /* vmas should already be unbound */
+ /* vmas should already be unbound and destroyed */
WARN_ON(!list_empty(&ppgtt->base.active_list));
WARN_ON(!list_empty(&ppgtt->base.inactive_list));
+ WARN_ON(!list_empty(&ppgtt->base.unbound_list));
list_del(&ppgtt->base.global_link);
drm_mm_takedown(&ppgtt->base.mm);
@@ -3375,6 +3377,8 @@ __i915_gem_vma_create(struct drm_i915_gem_object *obj,
struct i915_vma *vma;
int i;
+ GEM_BUG_ON(vm->closed);
+
if (WARN_ON(i915_is_ggtt(vm) != !!ggtt_view))
return ERR_PTR(-EINVAL);
@@ -3382,11 +3386,11 @@ __i915_gem_vma_create(struct drm_i915_gem_object *obj,
if (vma == NULL)
return ERR_PTR(-ENOMEM);
- INIT_LIST_HEAD(&vma->vm_link);
INIT_LIST_HEAD(&vma->obj_link);
INIT_LIST_HEAD(&vma->exec_list);
for (i = 0; i < ARRAY_SIZE(vma->last_read); i++)
init_request_active(&vma->last_read[i], i915_vma_retire);
+ list_add(&vma->vm_link, &vm->unbound_list);
vma->vm = vm;
vma->obj = obj;
vma->is_ggtt = i915_is_ggtt(vm);
@@ -3427,6 +3431,7 @@ i915_gem_obj_lookup_or_create_ggtt_vma(struct drm_i915_gem_object *obj,
if (!vma)
vma = __i915_gem_vma_create(obj, &ggtt->base, view);
+ GEM_BUG_ON(vma->closed);
return vma;
}
diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.h b/drivers/gpu/drm/i915/i915_gem_gtt.h
index a4f491dc1dae..fe0921c12c9b 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.h
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.h
@@ -314,6 +314,8 @@ struct i915_address_space {
u64 start; /* Start offset always 0 for dri2 */
u64 total; /* size addr space maps (ex. 2GB for ggtt) */
+ bool closed;
+
struct i915_page_scratch *scratch_page;
struct i915_page_table *scratch_pt;
struct i915_page_directory *scratch_pd;
@@ -342,6 +344,13 @@ struct i915_address_space {
*/
struct list_head inactive_list;
+ /**
+ * List of vma that have been unbound.
+ *
+ * A reference is not held on the buffer while on this list.
+ */
+ struct list_head unbound_list;
+
/* FIXME: Need a more generic return type */
gen6_pte_t (*pte_encode)(dma_addr_t addr,
enum i915_cache_level level,
diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c
index 2c321c8df7c3..bc91ffe614e2 100644
--- a/drivers/gpu/drm/i915/i915_gem_stolen.c
+++ b/drivers/gpu/drm/i915/i915_gem_stolen.c
@@ -707,7 +707,7 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_device *dev,
vma->bound |= GLOBAL_BIND;
__i915_vma_set_map_and_fenceable(vma);
- list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
+ list_move_tail(&vma->vm_link, &ggtt->base.inactive_list);
obj->bind_count++;
list_add_tail(&obj->global_list, &dev_priv->mm.bound_list);
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* [PATCH 64/64] Revert "drm/i915: Clean up associated VMAs on context destruction"
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (61 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 63/64] drm/i915: Mark the context and address space as closed Chris Wilson
@ 2016-07-07 8:42 ` Chris Wilson
2016-07-07 9:12 ` ✗ Ro.CI.BAT: warning for series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Patchwork
63 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 8:42 UTC (permalink / raw)
To: intel-gfx
This reverts commit e9f24d5fb7cf3628b195b18ff3ac4e37937ceeae.
The patch was only a stop-gap measure that fixed half the problem - the
leak of the fbcon when restarting X. A complete solution required
releasing the VMA when the object itself was closed rather than rely on
file/process exit. The previous patches add the VMA tracking necessary
to do close them along with the object, context or file, and so the time
has come to remove the partial fix.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
---
drivers/gpu/drm/i915/i915_drv.h | 5 -----
drivers/gpu/drm/i915/i915_gem.c | 14 ++------------
drivers/gpu/drm/i915/i915_gem_context.c | 22 ----------------------
3 files changed, 2 insertions(+), 39 deletions(-)
diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
index b465fc4addb2..d1827ad82476 100644
--- a/drivers/gpu/drm/i915/i915_drv.h
+++ b/drivers/gpu/drm/i915/i915_drv.h
@@ -3024,11 +3024,6 @@ int i915_vma_bind(struct i915_vma *vma, enum i915_cache_level cache_level,
u32 flags);
void __i915_vma_set_map_and_fenceable(struct i915_vma *vma);
int __must_check i915_vma_unbind(struct i915_vma *vma);
-/*
- * BEWARE: Do not use the function below unless you can _absolutely_
- * _guarantee_ VMA in question is _not in use_ anywhere.
- */
-int __must_check __i915_vma_unbind_no_wait(struct i915_vma *vma);
void i915_vma_close(struct i915_vma *vma);
void i915_vma_destroy(struct i915_vma *vma);
diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
index d6ca6ecec37f..64aecb726593 100644
--- a/drivers/gpu/drm/i915/i915_gem.c
+++ b/drivers/gpu/drm/i915/i915_gem.c
@@ -2802,7 +2802,7 @@ static void __i915_vma_iounmap(struct i915_vma *vma)
vma->iomap = NULL;
}
-static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
+int i915_vma_unbind(struct i915_vma *vma)
{
struct drm_i915_gem_object *obj = vma->obj;
unsigned long active;
@@ -2812,7 +2812,7 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
* have side-effects such as unpinning or even unbinding this vma.
*/
active = vma->active;
- if (active && wait) {
+ if (active) {
int idx;
/* When a closed VMA is retired, it is unbound - eek.
@@ -2894,16 +2894,6 @@ destroy:
return 0;
}
-int i915_vma_unbind(struct i915_vma *vma)
-{
- return __i915_vma_unbind(vma, true);
-}
-
-int __i915_vma_unbind_no_wait(struct i915_vma *vma)
-{
- return __i915_vma_unbind(vma, false);
-}
-
int i915_gem_wait_for_idle(struct drm_i915_private *dev_priv)
{
struct intel_engine_cs *engine;
diff --git a/drivers/gpu/drm/i915/i915_gem_context.c b/drivers/gpu/drm/i915/i915_gem_context.c
index 0e1c4a696c6c..6cb2619e83ba 100644
--- a/drivers/gpu/drm/i915/i915_gem_context.c
+++ b/drivers/gpu/drm/i915/i915_gem_context.c
@@ -134,21 +134,6 @@ static int get_context_size(struct drm_i915_private *dev_priv)
return ret;
}
-static void i915_gem_context_clean(struct i915_gem_context *ctx)
-{
- struct i915_hw_ppgtt *ppgtt = ctx->ppgtt;
- struct i915_vma *vma, *next;
-
- if (!ppgtt)
- return;
-
- list_for_each_entry_safe(vma, next, &ppgtt->base.inactive_list,
- vm_link) {
- if (WARN_ON(__i915_vma_unbind_no_wait(vma)))
- break;
- }
-}
-
void i915_gem_context_free(struct kref *ctx_ref)
{
struct i915_gem_context *ctx = container_of(ctx_ref, typeof(*ctx), ref);
@@ -158,13 +143,6 @@ void i915_gem_context_free(struct kref *ctx_ref)
trace_i915_context_free(ctx);
GEM_BUG_ON(!ctx->closed);
- /*
- * This context is going away and we need to remove all VMAs still
- * around. This is to handle imported shared objects for which
- * destructor did not run when their handles were closed.
- */
- i915_gem_context_clean(ctx);
-
i915_ppgtt_put(ctx->ppgtt);
for (i = 0; i < I915_NUM_ENGINES; i++) {
--
2.8.1
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply related [flat|nested] 93+ messages in thread
* ✗ Ro.CI.BAT: warning for series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
` (62 preceding siblings ...)
2016-07-07 8:42 ` [PATCH 64/64] Revert "drm/i915: Clean up associated VMAs on context destruction" Chris Wilson
@ 2016-07-07 9:12 ` Patchwork
2016-07-07 9:28 ` Chris Wilson
63 siblings, 1 reply; 93+ messages in thread
From: Patchwork @ 2016-07-07 9:12 UTC (permalink / raw)
To: Chris Wilson; +Cc: intel-gfx
== Series Details ==
Series: series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping
URL : https://patchwork.freedesktop.org/series/9603/
State : warning
== Summary ==
Series 9603v1 Series without cover letter
http://patchwork.freedesktop.org/api/1.0/series/9603/revisions/1/mbox
Test gem_exec_flush:
Subgroup basic-batch-kernel-default-cmd:
fail -> PASS (ro-byt-n2820)
Subgroup basic-uc-pro-default:
fail -> PASS (ro-byt-n2820)
Test kms_pipe_crc_basic:
Subgroup suspend-read-crc-pipe-a:
skip -> DMESG-WARN (ro-bdw-i7-5557U)
fi-skl-i5-6260u total:235 pass:207 dwarn:0 dfail:0 fail:2 skip:26
fi-skl-i7-6700k total:235 pass:193 dwarn:0 dfail:0 fail:2 skip:40
fi-snb-i7-2600 total:235 pass:179 dwarn:0 dfail:0 fail:2 skip:54
ro-bdw-i5-5250u total:229 pass:204 dwarn:1 dfail:1 fail:0 skip:23
ro-bdw-i7-5557U total:229 pass:204 dwarn:2 dfail:1 fail:0 skip:22
ro-bdw-i7-5600u total:229 pass:190 dwarn:0 dfail:1 fail:0 skip:38
ro-byt-n2820 total:229 pass:182 dwarn:0 dfail:1 fail:1 skip:45
ro-hsw-i3-4010u total:229 pass:197 dwarn:0 dfail:1 fail:0 skip:31
ro-hsw-i7-4770r total:229 pass:197 dwarn:0 dfail:1 fail:0 skip:31
ro-ilk-i7-620lm total:229 pass:157 dwarn:0 dfail:1 fail:1 skip:70
ro-ilk1-i5-650 total:224 pass:157 dwarn:0 dfail:1 fail:1 skip:65
ro-ivb-i7-3770 total:229 pass:188 dwarn:0 dfail:1 fail:0 skip:40
ro-skl3-i5-6260u total:229 pass:208 dwarn:1 dfail:1 fail:0 skip:19
ro-snb-i7-2620M total:229 pass:179 dwarn:0 dfail:1 fail:1 skip:48
fi-kbl-qkkr failed to connect after reboot
Results at /archive/results/CI_IGT_test/RO_Patchwork_1444/
eb8bb07 drm-intel-nightly: 2016y-07m-06d-15h-24m-53s UTC integration manifest
60b4309 Revert "drm/i915: Clean up associated VMAs on context destruction"
fb67695 drm/i915: Mark the context and address space as closed
c6f68f9 drm/i915: Release vma when the handle is closed
dad259a drm/i915: Track active vma requests
1d6f6d8 drm/i915: i915_vma_move_to_active prep patch
3091836 drm/i915: Move request list retirement to i915_gem_request.c
187b6e2 drm/i915: Double check activity before relocations
8dadc9e drm/i915: Move the special case wait-request handling to its one caller
79cb934 drm/i915: Convert intel_overlay to request tracking
a0c0650 drm/i915: Refactor activity tracking for requests
2c57225 drm/i915: Remove obsolete i915_gem_object_flush_active()
c5c30ad drm/i915: Rename request->list to link for consistency
99cc6f6 drm/i915: Refactor blocking waits
8e49329 drm/i915: Mark up i915_gem_active for locking annotation
b42246a drm/i915: Prepare i915_gem_active for annotations
9ced8cf drm/i915: Introduce i915_gem_active for request tracking
a3dabe6 drm/i915: Kill drop_pages()
d836905 drm/i915: Be more careful when unbinding vma
bbbb30f drm/i915: Count how many VMA are bound for an object
d9480d1 drm/i915: Store owning file on the i915_address_space
d9f31ed drm/i915: Split early global GTT initialisation
b5c4a1d drm/i915: Amalgamate GGTT/ppGTT vma debug list walkers
d420162 drm/i915: Simplify calling engine->sync_to
345b2d3 drm/i915: Unify legacy/execlists submit_execbuf callbacks
763bdf1 drm/i915: Remove duplicate golden render state init from execlists
0b15a5f drm/i915/ringbuffer: Specialise SNB+ request emission for semaphores
1bd68e5 drm/i915: Reuse legacy breadcrumbs + tail emission
0b870d3 drm/i915: Stop passing caller's num_dwords to engine->semaphore.signal()
b9be88d drm/i915: Unify request submission
b9edd1e drm/i915: Convert engine->write_tail to operate on a request
d78ab83 drm/i915: Unify legacy/execlists emission of MI_BATCHBUFFER_START
245f8e0 drm/i915: Simplify request_alloc by returning the allocated request
c3b4b08 drm/i915: Remove obsolete engine->gpu_caches_dirty
a67c86c drm/i915: Rename intel_pin_and_map_ring()
b0949da drm/i915: Rename residual ringbuf parameters
e18c7b4 drm/i915: Rename struct intel_ringbuffer to struct intel_ring
b3dffd6 drm/i915: Rename intel_context[engine].ringbuf
feef1b6 drm/i915: Rename backpointer from intel_ringbuffer to intel_engine_cs
37ec67d drm/i915: Rename request->ringbuf to request->ring
79dd9df drm/i915: Unify intel_logical_ring_emit and intel_ring_emit
532765f drm/i915: Convert i915_semaphores_is_enabled over to early sanitize
d28e0a7 drm/i915: Rename ring->virtual_start as ring->vaddr
a12aa01 drm/i915: Treat ringbuffer writes as write to normal memory
e8d35eb drm/i915: Rename drm_gem_object_unreference_unlocked in preparation for lockless free
be3ab6a drm/i915: Rename drm_gem_object_unreference in preparation for lockless free
5aef5d5 drm/i915: Wrap drm_gem_object_reference in i915_gem_object_get
d63e536 drm/i915: Wrap drm_gem_object_lookup in i915_gem_object_lookup
5cf2117 drm/i915: Rename i915_gem_context_reference/unreference()
2e85775 drm/i915: Rename request reference/unreference to get/put
4bcec96 drm/i915: Mark imported dma-buf objects as being coherent
013d21a drm/i915: Wait on external rendering for GEM objects
e878c15 drm/i915: Export our request as a dma-buf fence on the reservation object
37d11e5 drm/i915: Disable waitboosting for mmioflips/semaphores
535bdf6 drm/i915: Disable waitboosting for fence_wait()
5db5a38 drm/i915: Derive GEM requests from dma-fence
e1bdb73 drm/i915: Mark all current requests as complete before resetting them
d7140e6 drm/i915: Retire oldest completed request before allocating next
e1717e7 drm/i915: Move GEM request routines to i915_gem_request.c
984214f drm: Restore double clflush on the last partial cacheline
6f014f3 drm/i915: Remove temporary RPM wakeref assert disables
473d356 drm/i915: Defer enabling rc6 til after we submit the first batch/context
656d097 drm/i915: Remove superfluous powersave work flushing
0e75749 drm/i915: Preserve current RPS frequency across init
095531c drm/i915/breadcrumbs: Queue hangcheck before sleeping
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: ✗ Ro.CI.BAT: warning for series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping
2016-07-07 9:12 ` ✗ Ro.CI.BAT: warning for series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Patchwork
@ 2016-07-07 9:28 ` Chris Wilson
2016-07-08 8:44 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 9:28 UTC (permalink / raw)
To: intel-gfx
On Thu, Jul 07, 2016 at 09:12:01AM -0000, Patchwork wrote:
> == Series Details ==
>
> Series: series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping
> URL : https://patchwork.freedesktop.org/series/9603/
> State : warning
>
> == Summary ==
>
> Series 9603v1 Series without cover letter
> http://patchwork.freedesktop.org/api/1.0/series/9603/revisions/1/mbox
>
> Test gem_exec_flush:
> Subgroup basic-batch-kernel-default-cmd:
> fail -> PASS (ro-byt-n2820)
> Subgroup basic-uc-pro-default:
> fail -> PASS (ro-byt-n2820)
> Test kms_pipe_crc_basic:
> Subgroup suspend-read-crc-pipe-a:
> skip -> DMESG-WARN (ro-bdw-i7-5557U)
>
> fi-skl-i5-6260u total:235 pass:207 dwarn:0 dfail:0 fail:2 skip:26
> fi-skl-i7-6700k total:235 pass:193 dwarn:0 dfail:0 fail:2 skip:40
What happened to prime-busy? Should be fixed...
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next
2016-07-07 8:41 ` [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next Chris Wilson
@ 2016-07-07 9:41 ` Tvrtko Ursulin
2016-07-07 9:45 ` Chris Wilson
2016-07-08 9:21 ` Mika Kuoppala
1 sibling, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-07 9:41 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 07/07/16 09:41, Chris Wilson wrote:
> In order to keep the memory allocated for requests reasonably tight, try
> to reuse the oldest request (so long as it is completed and has no
> external references) for the next allocation.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
> drivers/gpu/drm/i915/i915_gem_request.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
> index 9e9aa6b725f7..ee1189c35509 100644
> --- a/drivers/gpu/drm/i915/i915_gem_request.c
> +++ b/drivers/gpu/drm/i915/i915_gem_request.c
> @@ -226,6 +226,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
> if (ret)
> return ret;
>
> + if (!list_empty(&engine->request_list)) {
> + req = list_first_entry(&engine->request_list,
> + typeof(*req), list);
> + if (i915_gem_request_completed(req))
> + i915_gem_request_retire(req);
> + }
> +
> req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
> if (!req)
> return -ENOMEM;
>
I am thinking that this does not play well with the execlists which is
holding references to requests for a little bit longer than they are on
the engine->request_list.
In fact I don't see how you can just steal it without looking at the
reference count.
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next
2016-07-07 9:41 ` Tvrtko Ursulin
@ 2016-07-07 9:45 ` Chris Wilson
2016-07-07 10:03 ` Tvrtko Ursulin
0 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 9:45 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Thu, Jul 07, 2016 at 10:41:03AM +0100, Tvrtko Ursulin wrote:
>
> On 07/07/16 09:41, Chris Wilson wrote:
> >In order to keep the memory allocated for requests reasonably tight, try
> >to reuse the oldest request (so long as it is completed and has no
> >external references) for the next allocation.
> >
> >Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> >---
> > drivers/gpu/drm/i915/i915_gem_request.c | 7 +++++++
> > 1 file changed, 7 insertions(+)
> >
> >diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
> >index 9e9aa6b725f7..ee1189c35509 100644
> >--- a/drivers/gpu/drm/i915/i915_gem_request.c
> >+++ b/drivers/gpu/drm/i915/i915_gem_request.c
> >@@ -226,6 +226,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
> > if (ret)
> > return ret;
> >
> >+ if (!list_empty(&engine->request_list)) {
> >+ req = list_first_entry(&engine->request_list,
> >+ typeof(*req), list);
> >+ if (i915_gem_request_completed(req))
> >+ i915_gem_request_retire(req);
> >+ }
> >+
> > req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
> > if (!req)
> > return -ENOMEM;
> >
>
> I am thinking that this does not play well with the execlists which
> is holding references to requests for a little bit longer than they
> are on the engine->request_list.
>
> In fact I don't see how you can just steal it without looking at the
> reference count.
?
There is no stealing, the request list and execlist are independently
referenced.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next
2016-07-07 9:45 ` Chris Wilson
@ 2016-07-07 10:03 ` Tvrtko Ursulin
2016-07-07 10:10 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-07 10:03 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 07/07/16 10:45, Chris Wilson wrote:
> On Thu, Jul 07, 2016 at 10:41:03AM +0100, Tvrtko Ursulin wrote:
>>
>> On 07/07/16 09:41, Chris Wilson wrote:
>>> In order to keep the memory allocated for requests reasonably tight, try
>>> to reuse the oldest request (so long as it is completed and has no
>>> external references) for the next allocation.
>>>
>>> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
>>> ---
>>> drivers/gpu/drm/i915/i915_gem_request.c | 7 +++++++
>>> 1 file changed, 7 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
>>> index 9e9aa6b725f7..ee1189c35509 100644
>>> --- a/drivers/gpu/drm/i915/i915_gem_request.c
>>> +++ b/drivers/gpu/drm/i915/i915_gem_request.c
>>> @@ -226,6 +226,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
>>> if (ret)
>>> return ret;
>>>
>>> + if (!list_empty(&engine->request_list)) {
>>> + req = list_first_entry(&engine->request_list,
>>> + typeof(*req), list);
>>> + if (i915_gem_request_completed(req))
>>> + i915_gem_request_retire(req);
>>> + }
>>> +
>>> req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
>>> if (!req)
>>> return -ENOMEM;
>>>
>>
>> I am thinking that this does not play well with the execlists which
>> is holding references to requests for a little bit longer than they
>> are on the engine->request_list.
>>
>> In fact I don't see how you can just steal it without looking at the
>> reference count.
>
> ?
>
> There is no stealing, the request list and execlist are independently
> referenced.
Oh my bad - I read the commit message as you will take the oldest
completed from engine->request_list instead of allocating a new one.
Code does not do that at all. You are just trying to free one so the
SLAB allocator might potentially reuse it.
Are you just trying to improve the retire worker? :))
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next
2016-07-07 10:03 ` Tvrtko Ursulin
@ 2016-07-07 10:10 ` Chris Wilson
0 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-07 10:10 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Thu, Jul 07, 2016 at 11:03:42AM +0100, Tvrtko Ursulin wrote:
>
> On 07/07/16 10:45, Chris Wilson wrote:
> >On Thu, Jul 07, 2016 at 10:41:03AM +0100, Tvrtko Ursulin wrote:
> >>
> >>On 07/07/16 09:41, Chris Wilson wrote:
> >>>In order to keep the memory allocated for requests reasonably tight, try
> >>>to reuse the oldest request (so long as it is completed and has no
> >>>external references) for the next allocation.
> >>>
> >>>Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> >>>---
> >>> drivers/gpu/drm/i915/i915_gem_request.c | 7 +++++++
> >>> 1 file changed, 7 insertions(+)
> >>>
> >>>diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
> >>>index 9e9aa6b725f7..ee1189c35509 100644
> >>>--- a/drivers/gpu/drm/i915/i915_gem_request.c
> >>>+++ b/drivers/gpu/drm/i915/i915_gem_request.c
> >>>@@ -226,6 +226,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
> >>> if (ret)
> >>> return ret;
> >>>
> >>>+ if (!list_empty(&engine->request_list)) {
> >>>+ req = list_first_entry(&engine->request_list,
> >>>+ typeof(*req), list);
> >>>+ if (i915_gem_request_completed(req))
> >>>+ i915_gem_request_retire(req);
> >>>+ }
> >>>+
> >>> req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
> >>> if (!req)
> >>> return -ENOMEM;
> >>>
> >>
> >>I am thinking that this does not play well with the execlists which
> >>is holding references to requests for a little bit longer than they
> >>are on the engine->request_list.
> >>
> >>In fact I don't see how you can just steal it without looking at the
> >>reference count.
> >
> >?
> >
> >There is no stealing, the request list and execlist are independently
> >referenced.
>
> Oh my bad - I read the commit message as you will take the oldest
> completed from engine->request_list instead of allocating a new one.
> Code does not do that at all. You are just trying to free one so the
> SLAB allocator might potentially reuse it.
>
> Are you just trying to improve the retire worker? :))
The issue down the line is that we increase the laziness of retirement,
principally to avoid long stalls as we free lots of requests, objects
and pages. The problem with going too lazy is then we starve the system
of memory (and just move the long stall to the extreme case of hitting
the direct reclaim). We curtail that (in order to protect ourselves from
provoking an oom) by handling the RCU at that moment. As alluded to in
that future patch, the other approach is throttling at the point of
consumption. This adds the lightweight version of that with the
advantage of trying to keep the slab cache ready for a new allocation.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 09/64] drm/i915: Mark all current requests as complete before resetting them
2016-07-07 8:41 ` [PATCH 09/64] drm/i915: Mark all current requests as complete before resetting them Chris Wilson
@ 2016-07-07 16:09 ` Mika Kuoppala
0 siblings, 0 replies; 93+ messages in thread
From: Mika Kuoppala @ 2016-07-07 16:09 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
Chris Wilson <chris@chris-wilson.co.uk> writes:
> Following a GPU reset upon hang, we retire all the requests and then
> mark them all as complete. If we mark them as complete first, we both
> keep the normal retirement order (completed first then retired) and
> provide a small optimisation for concurrent lookups.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 14cc87b916c6..1c05e4573173 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -2486,6 +2486,12 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
> i915_gem_object_retire__read(obj, engine->id);
> }
>
> + /* Mark all pending requests as complete so that any concurrent
> + * (lockless) lookup doesn't try and wait upon the request as we
> + * reset it.
> + */
> + intel_ring_init_seqno(engine, engine->last_submitted_seqno);
> +
> /*
> * Clear the execlists queue up before freeing the requests, as those
> * are the ones that keep the context and ringbuffer backing objects
> @@ -2527,8 +2533,6 @@ static void i915_gem_reset_engine_cleanup(struct intel_engine_cs *engine)
> buffer->last_retired_head = buffer->tail;
> intel_ring_update_space(buffer);
> }
> -
> - intel_ring_init_seqno(engine, engine->last_submitted_seqno);
> }
>
> void i915_gem_reset(struct drm_device *dev)
> --
> 2.8.1
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: ✗ Ro.CI.BAT: warning for series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping
2016-07-07 9:28 ` Chris Wilson
@ 2016-07-08 8:44 ` Chris Wilson
0 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-08 8:44 UTC (permalink / raw)
To: intel-gfx
On Thu, Jul 07, 2016 at 10:28:42AM +0100, Chris Wilson wrote:
> On Thu, Jul 07, 2016 at 09:12:01AM -0000, Patchwork wrote:
> > == Series Details ==
> >
> > Series: series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping
> > URL : https://patchwork.freedesktop.org/series/9603/
> > State : warning
> >
> > == Summary ==
> >
> > Series 9603v1 Series without cover letter
> > http://patchwork.freedesktop.org/api/1.0/series/9603/revisions/1/mbox
> >
> > Test gem_exec_flush:
> > Subgroup basic-batch-kernel-default-cmd:
> > fail -> PASS (ro-byt-n2820)
> > Subgroup basic-uc-pro-default:
> > fail -> PASS (ro-byt-n2820)
> > Test kms_pipe_crc_basic:
> > Subgroup suspend-read-crc-pipe-a:
> > skip -> DMESG-WARN (ro-bdw-i7-5557U)
> >
> > fi-skl-i5-6260u total:235 pass:207 dwarn:0 dfail:0 fail:2 skip:26
> > fi-skl-i7-6700k total:235 pass:193 dwarn:0 dfail:0 fail:2 skip:40
>
> What happened to prime-busy? Should be fixed...
Trybot:
eries 9649v1 Series without cover letter
http://patchwork.freedesktop.org/api/1.0/series/9649/revisions/1/mbox
Test gem_exec_flush:
Subgroup basic-batch-kernel-default-cmd:
fail -> PASS (ro-byt-n2820)
Subgroup basic-uc-pro-default:
fail -> PASS (ro-byt-n2820)
Test gem_exec_suspend:
Subgroup basic-s3:
dmesg-warn -> PASS (fi-skl-i5-6260u)
Test prime_busy:
Subgroup basic-after-default:
fail -> PASS (fi-skl-i7-6700k)
fail -> PASS (fi-skl-i5-6260u)
fail -> PASS (fi-snb-i7-2600)
Subgroup basic-before-default:
fail -> PASS (fi-skl-i7-6700k)
fail -> PASS (fi-skl-i5-6260u)
fail -> PASS (fi-snb-i7-2600)
Subgroup basic-wait-after-default:
fail -> PASS (fi-skl-i7-6700k)
fail -> PASS (fi-skl-i5-6260u)
fail -> PASS (fi-snb-i7-2600)
Subgroup basic-wait-before-default:
fail -> PASS (fi-skl-i7-6700k)
fail -> PASS (fi-skl-i5-6260u)
fail -> PASS (fi-snb-i7-2600)
But why only skl/snb? Is prime_busy not being run elsewhere?
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next
2016-07-07 8:41 ` [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next Chris Wilson
2016-07-07 9:41 ` Tvrtko Ursulin
@ 2016-07-08 9:21 ` Mika Kuoppala
2016-07-08 9:28 ` Chris Wilson
1 sibling, 1 reply; 93+ messages in thread
From: Mika Kuoppala @ 2016-07-08 9:21 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
Chris Wilson <chris@chris-wilson.co.uk> writes:
> In order to keep the memory allocated for requests reasonably tight, try
> to reuse the oldest request (so long as it is completed and has no
> external references) for the next allocation.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
> drivers/gpu/drm/i915/i915_gem_request.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
> index 9e9aa6b725f7..ee1189c35509 100644
> --- a/drivers/gpu/drm/i915/i915_gem_request.c
> +++ b/drivers/gpu/drm/i915/i915_gem_request.c
> @@ -226,6 +226,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
> if (ret)
> return ret;
>
> + if (!list_empty(&engine->request_list)) {
> + req = list_first_entry(&engine->request_list,
> + typeof(*req), list);
> + if (i915_gem_request_completed(req))
> + i915_gem_request_retire(req);
> + }
> +
With the last_submitted_seqno and req->seqno you could get rough
estimation of requests-in-flight and perhaps retire handful more
if the diff is very big?
But one-for-one makes sense.
Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
> req = kmem_cache_zalloc(dev_priv->requests, GFP_KERNEL);
> if (!req)
> return -ENOMEM;
> --
> 2.8.1
>
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next
2016-07-08 9:21 ` Mika Kuoppala
@ 2016-07-08 9:28 ` Chris Wilson
0 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-08 9:28 UTC (permalink / raw)
To: Mika Kuoppala; +Cc: intel-gfx
On Fri, Jul 08, 2016 at 12:21:18PM +0300, Mika Kuoppala wrote:
> Chris Wilson <chris@chris-wilson.co.uk> writes:
>
> > In order to keep the memory allocated for requests reasonably tight, try
> > to reuse the oldest request (so long as it is completed and has no
> > external references) for the next allocation.
> >
> > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> > ---
> > drivers/gpu/drm/i915/i915_gem_request.c | 7 +++++++
> > 1 file changed, 7 insertions(+)
> >
> > diff --git a/drivers/gpu/drm/i915/i915_gem_request.c b/drivers/gpu/drm/i915/i915_gem_request.c
> > index 9e9aa6b725f7..ee1189c35509 100644
> > --- a/drivers/gpu/drm/i915/i915_gem_request.c
> > +++ b/drivers/gpu/drm/i915/i915_gem_request.c
> > @@ -226,6 +226,13 @@ __i915_gem_request_alloc(struct intel_engine_cs *engine,
> > if (ret)
> > return ret;
> >
> > + if (!list_empty(&engine->request_list)) {
> > + req = list_first_entry(&engine->request_list,
> > + typeof(*req), list);
> > + if (i915_gem_request_completed(req))
> > + i915_gem_request_retire(req);
> > + }
> > +
>
> With the last_submitted_seqno and req->seqno you could get rough
> estimation of requests-in-flight and perhaps retire handful more
> if the diff is very big?
That's the problem we want to avoid. We want to be cautious in how many
we retire as that leads to huge latency spikes here.
The observation that lead to trying to retire one was simply to improve
the slab-cache hit rate whilst minimising the time we spend in freeing
objects before submitting new work.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 06/64] drm: Restore double clflush on the last partial cacheline
2016-07-07 8:41 ` [PATCH 06/64] drm: Restore double clflush on the last partial cacheline Chris Wilson
@ 2016-07-12 13:57 ` Daniel Vetter
0 siblings, 0 replies; 93+ messages in thread
From: Daniel Vetter @ 2016-07-12 13:57 UTC (permalink / raw)
To: Chris Wilson
Cc: intel-gfx, dri-devel, Akash Goel, Imre Deak, Daniel Vetter,
Jason Ekstrand, stable
On Thu, Jul 07, 2016 at 09:41:12AM +0100, Chris Wilson wrote:
> This effectively reverts
>
> commit afcd950cafea6e27b739fe7772cbbeed37d05b8b
> Author: Chris Wilson <chris@chris-wilson.co.uk>
> Date: Wed Jun 10 15:58:01 2015 +0100
>
> drm: Avoid the double clflush on the last cache line in drm_clflush_virt_range()
>
> as we have observed issues with serialisation of the clflush operations
> on Baytrail+ Atoms with partial updates. Applying the double flush on the
> last cacheline forces that clflush to be ordered with respect to the
> previous clflush, and the mfence then protects against prefetches crossing
> the clflush boundary.
>
> The same issue can be demonstrated in userspace with igt/gem_exec_flush.
>
> Fixes: afcd950cafea6 (drm: Avoid the double clflush on the last cache...)
> Testcase: igt/gem_concurrent_blit
> Testcase: igt/gem_partial_pread_pwrite
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92845
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: dri-devel@lists.freedesktop.org
> Cc: Akash Goel <akash.goel@intel.com>
> Cc: Imre Deak <imre.deak@intel.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Jason Ekstrand <jason.ekstrand@intel.com>
> Cc: stable@vger.kernel.org
> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
It's duct-tape, but oh well. Applied to drm-misc.
-Daniel
> ---
> drivers/gpu/drm/drm_cache.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c
> index 059f7c39c582..a7916e5f8864 100644
> --- a/drivers/gpu/drm/drm_cache.c
> +++ b/drivers/gpu/drm/drm_cache.c
> @@ -136,6 +136,7 @@ drm_clflush_virt_range(void *addr, unsigned long length)
> mb();
> for (; addr < end; addr += size)
> clflushopt(addr);
> + clflushopt(end - 1); /* force serialisation */
> mb();
> return;
> }
> --
> 2.8.1
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 06/64] drm: Restore double clflush on the last partial cacheline
@ 2016-07-12 13:57 ` Daniel Vetter
0 siblings, 0 replies; 93+ messages in thread
From: Daniel Vetter @ 2016-07-12 13:57 UTC (permalink / raw)
To: Chris Wilson
Cc: Daniel Vetter, dri-devel, Jason Ekstrand, Akash Goel, stable, intel-gfx
On Thu, Jul 07, 2016 at 09:41:12AM +0100, Chris Wilson wrote:
> This effectively reverts
>
> commit afcd950cafea6e27b739fe7772cbbeed37d05b8b
> Author: Chris Wilson <chris@chris-wilson.co.uk>
> Date: Wed Jun 10 15:58:01 2015 +0100
>
> drm: Avoid the double clflush on the last cache line in drm_clflush_virt_range()
>
> as we have observed issues with serialisation of the clflush operations
> on Baytrail+ Atoms with partial updates. Applying the double flush on the
> last cacheline forces that clflush to be ordered with respect to the
> previous clflush, and the mfence then protects against prefetches crossing
> the clflush boundary.
>
> The same issue can be demonstrated in userspace with igt/gem_exec_flush.
>
> Fixes: afcd950cafea6 (drm: Avoid the double clflush on the last cache...)
> Testcase: igt/gem_concurrent_blit
> Testcase: igt/gem_partial_pread_pwrite
> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92845
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> Cc: dri-devel@lists.freedesktop.org
> Cc: Akash Goel <akash.goel@intel.com>
> Cc: Imre Deak <imre.deak@intel.com>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Cc: Jason Ekstrand <jason.ekstrand@intel.com>
> Cc: stable@vger.kernel.org
> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
It's duct-tape, but oh well. Applied to drm-misc.
-Daniel
> ---
> drivers/gpu/drm/drm_cache.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c
> index 059f7c39c582..a7916e5f8864 100644
> --- a/drivers/gpu/drm/drm_cache.c
> +++ b/drivers/gpu/drm/drm_cache.c
> @@ -136,6 +136,7 @@ drm_clflush_virt_range(void *addr, unsigned long length)
> mb();
> for (; addr < end; addr += size)
> clflushopt(addr);
> + clflushopt(end - 1); /* force serialisation */
> mb();
> return;
> }
> --
> 2.8.1
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 46/64] drm/i915: Count how many VMA are bound for an object
2016-07-07 8:41 ` [PATCH 46/64] drm/i915: Count how many VMA are bound for an object Chris Wilson
@ 2016-07-12 14:30 ` Tvrtko Ursulin
2016-07-12 14:38 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-12 14:30 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 07/07/16 09:41, Chris Wilson wrote:
> Since we may have VMA allocated for an object, but we interrupted their
> binding, there is a disparity between have elements on the obj->vma_list
> and being bound. i915_gem_obj_bound_any() does this check, but this is
> not rigorously observed - add an explicit count to make it easier.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
> drivers/gpu/drm/i915/i915_debugfs.c | 12 +++++------
> drivers/gpu/drm/i915/i915_drv.h | 3 ++-
> drivers/gpu/drm/i915/i915_gem.c | 34 +++++++++++++-------------------
> drivers/gpu/drm/i915/i915_gem_shrinker.c | 17 +---------------
> drivers/gpu/drm/i915/i915_gem_stolen.c | 1 +
> 5 files changed, 23 insertions(+), 44 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
> index 327064823bb0..d3852828878f 100644
> --- a/drivers/gpu/drm/i915/i915_debugfs.c
> +++ b/drivers/gpu/drm/i915/i915_debugfs.c
> @@ -174,6 +174,9 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
> if (obj->fence_reg != I915_FENCE_REG_NONE)
> seq_printf(m, " (fence: %d)", obj->fence_reg);
> list_for_each_entry(vma, &obj->vma_list, obj_link) {
> + if (!drm_mm_node_allocated(&vma->node))
> + continue;
> +
> seq_printf(m, " (%sgtt offset: %08llx, size: %08llx",
> vma->is_ggtt ? "g" : "pp",
> vma->node.start, vma->node.size);
> @@ -335,11 +338,11 @@ static int per_file_stats(int id, void *ptr, void *data)
> struct drm_i915_gem_object *obj = ptr;
> struct file_stats *stats = data;
> struct i915_vma *vma;
> - int bound = 0;
>
> stats->count++;
> stats->total += obj->base.size;
> -
> + if (!obj->bind_count)
> + stats->unbound += obj->base.size;
> if (obj->base.name || obj->base.dma_buf)
> stats->shared += obj->base.size;
>
> @@ -347,8 +350,6 @@ static int per_file_stats(int id, void *ptr, void *data)
> if (!drm_mm_node_allocated(&vma->node))
> continue;
>
> - bound++;
> -
> if (vma->is_ggtt) {
> stats->global += vma->node.size;
> } else {
> @@ -366,9 +367,6 @@ static int per_file_stats(int id, void *ptr, void *data)
> stats->inactive += vma->node.size;
> }
>
> - if (!bound)
> - stats->unbound += obj->base.size;
> -
> return 0;
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index a19d18ea074e..633585054669 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -2217,6 +2217,8 @@ struct drm_i915_gem_object {
> unsigned int frontbuffer_bits:INTEL_FRONTBUFFER_BITS;
>
> unsigned int has_wc_mmap;
> + /** Count of VMA actually bound by this object */
> + unsigned int bind_count;
> unsigned int pin_display;
>
> struct sg_table *pages;
> @@ -3246,7 +3248,6 @@ i915_gem_obj_ggtt_offset(struct drm_i915_gem_object *o)
> return i915_gem_obj_ggtt_offset_view(o, &i915_ggtt_view_normal);
> }
>
> -bool i915_gem_obj_bound_any(struct drm_i915_gem_object *o);
> bool i915_gem_obj_ggtt_bound_view(struct drm_i915_gem_object *o,
> const struct i915_ggtt_view *view);
> bool i915_gem_obj_bound(struct drm_i915_gem_object *o,
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 98f3945fc50f..c6816f9969d5 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -2107,7 +2107,7 @@ i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
> if (obj->pages_pin_count)
> return -EBUSY;
>
> - BUG_ON(i915_gem_obj_bound_any(obj));
> + GEM_BUG_ON(obj->bind_count);
>
> /* ->put_pages might need to allocate memory for the bit17 swizzle
> * array, hence protect them from being reaped by removing them from gtt
> @@ -2957,7 +2957,6 @@ static void __i915_vma_iounmap(struct i915_vma *vma)
> static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
> {
> struct drm_i915_gem_object *obj = vma->obj;
> - struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
> int ret;
>
> if (list_empty(&vma->obj_link))
> @@ -2971,7 +2970,8 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
> if (vma->pin_count)
> return -EBUSY;
>
> - BUG_ON(obj->pages == NULL);
> + GEM_BUG_ON(obj->bind_count == 0);
> + GEM_BUG_ON(!obj->pages);
>
> if (wait) {
> ret = i915_gem_object_wait_rendering(obj, false);
> @@ -3011,8 +3011,9 @@ static int __i915_vma_unbind(struct i915_vma *vma, bool wait)
>
> /* Since the unbound list is global, only move to that list if
> * no more VMAs exist. */
> - if (list_empty(&obj->vma_list))
> - list_move_tail(&obj->global_list, &dev_priv->mm.unbound_list);
> + if (--obj->bind_count == 0)
> + list_move_tail(&obj->global_list,
> + &to_i915(obj->base.dev)->mm.unbound_list);
>
> /* And finally now the object is completely decoupled from this vma,
> * we can drop its hold on the backing storage and allow it to be
> @@ -3247,6 +3248,7 @@ search_free:
>
> list_move_tail(&obj->global_list, &dev_priv->mm.bound_list);
> list_add_tail(&vma->vm_link, &vm->inactive_list);
> + obj->bind_count++;
>
> return vma;
>
> @@ -3442,7 +3444,6 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> {
> struct drm_device *dev = obj->base.dev;
> struct i915_vma *vma, *next;
> - bool bound = false;
> int ret = 0;
>
> if (obj->cache_level == cache_level)
> @@ -3466,8 +3467,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> ret = i915_vma_unbind(vma);
> if (ret)
> return ret;
> - } else
> - bound = true;
> + }
> }
>
> /* We can reuse the existing drm_mm nodes but need to change the
> @@ -3477,7 +3477,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> * rewrite the PTE in the belief that doing so tramples upon less
> * state and so involves less work.
> */
> - if (bound) {
> + if (obj->bind_count) {
> /* Before we change the PTE, the GPU must not be accessing it.
> * If we wait upon the object, we know that all the bound
> * VMA are no longer active.
> @@ -3684,6 +3684,9 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
> old_read_domains,
> old_write_domain);
>
> + /* Increment the pages_pin_count to guard against the shrinker */
> + obj->pages_pin_count++;
Would it be clearer to look at obj->pin_display in the shrinker?
Although it looks like special casing out of the cleanliness of the
design in both case so I am not sure.
> +
> return 0;
>
> err_unpin_display:
> @@ -3700,6 +3703,7 @@ i915_gem_object_unpin_from_display_plane(struct drm_i915_gem_object *obj,
>
> i915_gem_object_ggtt_unpin_view(obj, view);
>
> + obj->pages_pin_count--;
> obj->pin_display--;
> }
>
> @@ -4215,6 +4219,7 @@ void i915_gem_free_object(struct drm_gem_object *gem_obj)
> dev_priv->mm.interruptible = was_interruptible;
> }
> }
> + GEM_BUG_ON(obj->bind_count);
>
> /* Stolen objects don't hold a ref, but do hold pin count. Fix that up
> * before progressing. */
> @@ -4851,17 +4856,6 @@ bool i915_gem_obj_ggtt_bound_view(struct drm_i915_gem_object *o,
> return false;
> }
>
> -bool i915_gem_obj_bound_any(struct drm_i915_gem_object *o)
> -{
> - struct i915_vma *vma;
> -
> - list_for_each_entry(vma, &o->vma_list, obj_link)
> - if (drm_mm_node_allocated(&vma->node))
> - return true;
> -
> - return false;
> -}
> -
> unsigned long i915_gem_obj_ggtt_size(struct drm_i915_gem_object *o)
> {
> struct i915_vma *vma;
> diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
> index e93a837cf2a1..725a8c894517 100644
> --- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
> +++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
> @@ -48,21 +48,6 @@ static bool mutex_is_locked_by(struct mutex *mutex, struct task_struct *task)
> #endif
> }
>
> -static int num_vma_bound(struct drm_i915_gem_object *obj)
> -{
> - struct i915_vma *vma;
> - int count = 0;
> -
> - list_for_each_entry(vma, &obj->vma_list, obj_link) {
> - if (drm_mm_node_allocated(&vma->node))
> - count++;
> - if (vma->pin_count)
> - count++;
> - }
> -
> - return count;
> -}
> -
> static bool swap_available(void)
> {
> return get_nr_swap_pages() > 0;
> @@ -82,7 +67,7 @@ static bool can_release_pages(struct drm_i915_gem_object *obj)
> * to the GPU, simply unbinding from the GPU is not going to succeed
> * in releasing our pin count on the pages themselves.
> */
> - if (obj->pages_pin_count != num_vma_bound(obj))
> + if (obj->pages_pin_count != obj->bind_count)
Would this be clearer as obj->pages_pin_count > obj->bind_count ?
> return false;
>
> /* We can only return physical pages to the system if we can either
> diff --git a/drivers/gpu/drm/i915/i915_gem_stolen.c b/drivers/gpu/drm/i915/i915_gem_stolen.c
> index 9a8cc8c51077..2c321c8df7c3 100644
> --- a/drivers/gpu/drm/i915/i915_gem_stolen.c
> +++ b/drivers/gpu/drm/i915/i915_gem_stolen.c
> @@ -708,6 +708,7 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_device *dev,
> vma->bound |= GLOBAL_BIND;
> __i915_vma_set_map_and_fenceable(vma);
> list_add_tail(&vma->vm_link, &ggtt->base.inactive_list);
> + obj->bind_count++;
>
> list_add_tail(&obj->global_list, &dev_priv->mm.bound_list);
> i915_gem_object_pin_pages(obj);
>
Looks fine to me.
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 46/64] drm/i915: Count how many VMA are bound for an object
2016-07-12 14:30 ` Tvrtko Ursulin
@ 2016-07-12 14:38 ` Chris Wilson
2016-07-12 15:12 ` Tvrtko Ursulin
0 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-12 14:38 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Tue, Jul 12, 2016 at 03:30:25PM +0100, Tvrtko Ursulin wrote:
> On 07/07/16 09:41, Chris Wilson wrote:
> >@@ -3684,6 +3684,9 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
> > old_read_domains,
> > old_write_domain);
> >
> >+ /* Increment the pages_pin_count to guard against the shrinker */
> >+ obj->pages_pin_count++;
>
> Would it be clearer to look at obj->pin_display in the shrinker?
> Although it looks like special casing out of the cleanliness of the
> design in both case so I am not sure.
Yeah. I liked the mechanism of telling the shrinker to avoid certain
pages by only having to control the pages_pin_count. It feels easier to
explain to others "the shrinker may reap any object that hasn't pinned
its pages" (explicitly called i915_gem_object_pin_pages for its own use)
rather than that + plus a list of exceptions known by the shrinker.
> >@@ -82,7 +67,7 @@ static bool can_release_pages(struct drm_i915_gem_object *obj)
> > * to the GPU, simply unbinding from the GPU is not going to succeed
> > * in releasing our pin count on the pages themselves.
> > */
> >- if (obj->pages_pin_count != num_vma_bound(obj))
> >+ if (obj->pages_pin_count != obj->bind_count)
>
> Would this be clearer as obj->pages_pin_count > obj->bind_count ?
Ok.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 47/64] drm/i915: Be more careful when unbinding vma
2016-07-07 8:41 ` [PATCH 47/64] drm/i915: Be more careful when unbinding vma Chris Wilson
@ 2016-07-12 15:04 ` Tvrtko Ursulin
2016-07-12 16:42 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-12 15:04 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 07/07/16 09:41, Chris Wilson wrote:
> When we call i915_vma_unbind(), we will wait upon outstanding rendering.
> This will also trigger a retirement phase, which may update the object
> lists. If, we extend request tracking to the VMA itself (rather than
> keep it at the encompassing object), then there is a potential that the
> obj->vma_list be modified for other elements upon i915_vma_unbind(). As
> a result, if we walk over the object list and call i915_vma_unbind(), we
> need to be prepared for that list to change.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
> drivers/gpu/drm/i915/i915_drv.h | 2 ++
> drivers/gpu/drm/i915/i915_gem.c | 57 +++++++++++++++++++++++---------
> drivers/gpu/drm/i915/i915_gem_shrinker.c | 7 +---
> drivers/gpu/drm/i915/i915_gem_userptr.c | 4 +--
> 4 files changed, 46 insertions(+), 24 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 633585054669..27e1182544a2 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -3032,6 +3032,8 @@ int __must_check i915_vma_unbind(struct i915_vma *vma);
> * _guarantee_ VMA in question is _not in use_ anywhere.
> */
> int __must_check __i915_vma_unbind_no_wait(struct i915_vma *vma);
> +
> +int i915_gem_object_unbind(struct drm_i915_gem_object *obj);
> int i915_gem_object_put_pages(struct drm_i915_gem_object *obj);
> void i915_gem_release_all_mmaps(struct drm_i915_private *dev_priv);
> void i915_gem_release_mmap(struct drm_i915_gem_object *obj);
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index c6816f9969d5..28a3079a7892 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -283,18 +283,38 @@ static const struct drm_i915_gem_object_ops i915_gem_phys_ops = {
> .release = i915_gem_object_release_phys,
> };
>
> +int
> +i915_gem_object_unbind(struct drm_i915_gem_object *obj)
> +{
> + struct i915_vma *vma;
> + LIST_HEAD(still_in_list);
> + int ret;
> +
> + /* The vma will only be freed if it is marked as closed, and if we wait
> + * upon rendering to the vma, we may unbind anything in the list.
> + */
> + while ((vma = list_first_entry_or_null(&obj->vma_list,
> + struct i915_vma,
> + obj_link))) {
> + list_move_tail(&vma->obj_link, &still_in_list);
> + ret = i915_vma_unbind(vma);
> + if (ret)
> + break;
> + }
> + list_splice(&still_in_list, &obj->vma_list);
> +
> + return ret;
> +}
> +
> static int
> drop_pages(struct drm_i915_gem_object *obj)
> {
> - struct i915_vma *vma, *next;
> int ret;
>
> i915_gem_object_get(obj);
> - list_for_each_entry_safe(vma, next, &obj->vma_list, obj_link)
> - if (i915_vma_unbind(vma))
> - break;
> -
> - ret = i915_gem_object_put_pages(obj);
> + ret = i915_gem_object_unbind(obj);
> + if (ret == 0)
> + ret = i915_gem_object_put_pages(obj);
> i915_gem_object_put(obj);
>
> return ret;
> @@ -3442,8 +3462,7 @@ i915_gem_object_set_to_gtt_domain(struct drm_i915_gem_object *obj, bool write)
> int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> enum i915_cache_level cache_level)
> {
> - struct drm_device *dev = obj->base.dev;
> - struct i915_vma *vma, *next;
> + struct i915_vma *vma;
> int ret = 0;
>
> if (obj->cache_level == cache_level)
> @@ -3454,7 +3473,8 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> * catch the issue of the CS prefetch crossing page boundaries and
> * reading an invalid PTE on older architectures.
> */
> - list_for_each_entry_safe(vma, next, &obj->vma_list, obj_link) {
> +restart:
> + list_for_each_entry(vma, &obj->vma_list, obj_link) {
> if (!drm_mm_node_allocated(&vma->node))
> continue;
>
> @@ -3463,11 +3483,18 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> return -EBUSY;
> }
>
> - if (!i915_gem_valid_gtt_space(vma, cache_level)) {
> - ret = i915_vma_unbind(vma);
> - if (ret)
> - return ret;
> - }
> + if (i915_gem_valid_gtt_space(vma, cache_level))
> + continue;
> +
> + ret = i915_vma_unbind(vma);
> + if (ret)
> + return ret;
> +
> + /* As unbinding may affect other elements in the
> + * obj->vma_list (due to side-effects from retiring
> + * an active vma), play safe and restart the iterator.
> + */
> + goto restart;
> }
Does not look efficient for long lists but I don't see a solution right
now. Any chance of this O(N^2) iteration hurting us in the real world?
>
> /* We can reuse the existing drm_mm nodes but need to change the
> @@ -3486,7 +3513,7 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> if (ret)
> return ret;
>
> - if (!HAS_LLC(dev) && cache_level != I915_CACHE_NONE) {
> + if (!HAS_LLC(obj->base.dev) && cache_level != I915_CACHE_NONE) {
> /* Access to snoopable pages through the GTT is
> * incoherent and on some machines causes a hard
> * lockup. Relinquish the CPU mmaping to force
> diff --git a/drivers/gpu/drm/i915/i915_gem_shrinker.c b/drivers/gpu/drm/i915/i915_gem_shrinker.c
> index 725a8c894517..8ad95695f80c 100644
> --- a/drivers/gpu/drm/i915/i915_gem_shrinker.c
> +++ b/drivers/gpu/drm/i915/i915_gem_shrinker.c
> @@ -155,7 +155,6 @@ i915_gem_shrink(struct drm_i915_private *dev_priv,
> INIT_LIST_HEAD(&still_in_list);
> while (count < target && !list_empty(phase->list)) {
> struct drm_i915_gem_object *obj;
> - struct i915_vma *vma, *v;
>
> obj = list_first_entry(phase->list,
> typeof(*obj), global_list);
> @@ -178,11 +177,7 @@ i915_gem_shrink(struct drm_i915_private *dev_priv,
> i915_gem_object_get(obj);
>
> /* For the unbound phase, this should be a no-op! */
> - list_for_each_entry_safe(vma, v,
> - &obj->vma_list, obj_link)
> - if (i915_vma_unbind(vma))
> - break;
> -
> + i915_gem_object_unbind(obj);
> if (i915_gem_object_put_pages(obj) == 0)
> count += obj->base.size >> PAGE_SHIFT;
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
> index ca8b82ab93d6..e935b327f3f9 100644
> --- a/drivers/gpu/drm/i915/i915_gem_userptr.c
> +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
> @@ -104,7 +104,6 @@ static void cancel_userptr(struct work_struct *work)
>
> if (obj->pages != NULL) {
> struct drm_i915_private *dev_priv = to_i915(dev);
> - struct i915_vma *vma, *tmp;
> bool was_interruptible;
>
> wait_rendering(obj);
> @@ -112,8 +111,7 @@ static void cancel_userptr(struct work_struct *work)
> was_interruptible = dev_priv->mm.interruptible;
> dev_priv->mm.interruptible = false;
>
> - list_for_each_entry_safe(vma, tmp, &obj->vma_list, obj_link)
> - WARN_ON(i915_vma_unbind(vma));
> + WARN_ON(i915_gem_object_unbind(obj));
> WARN_ON(i915_gem_object_put_pages(obj));
>
> dev_priv->mm.interruptible = was_interruptible;
>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 46/64] drm/i915: Count how many VMA are bound for an object
2016-07-12 14:38 ` Chris Wilson
@ 2016-07-12 15:12 ` Tvrtko Ursulin
2016-07-12 16:08 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-12 15:12 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 12/07/16 15:38, Chris Wilson wrote:
> On Tue, Jul 12, 2016 at 03:30:25PM +0100, Tvrtko Ursulin wrote:
>> On 07/07/16 09:41, Chris Wilson wrote:
>>> @@ -3684,6 +3684,9 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
>>> old_read_domains,
>>> old_write_domain);
>>>
>>> + /* Increment the pages_pin_count to guard against the shrinker */
>>> + obj->pages_pin_count++;
>>
>> Would it be clearer to look at obj->pin_display in the shrinker?
>> Although it looks like special casing out of the cleanliness of the
>> design in both case so I am not sure.
>
> Yeah. I liked the mechanism of telling the shrinker to avoid certain
> pages by only having to control the pages_pin_count. It feels easier to
> explain to others "the shrinker may reap any object that hasn't pinned
> its pages" (explicitly called i915_gem_object_pin_pages for its own use)
> rather than that + plus a list of exceptions known by the shrinker.
Hm but wait a minute, framebuffer already has the VMA pinned. So how
will the shrinker reap it?
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 48/64] drm/i915: Kill drop_pages()
2016-07-07 8:41 ` [PATCH 48/64] drm/i915: Kill drop_pages() Chris Wilson
@ 2016-07-12 15:14 ` Tvrtko Ursulin
0 siblings, 0 replies; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-12 15:14 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 07/07/16 09:41, Chris Wilson wrote:
> The drop_pages() function is a dangerous trap in that it can release the
> passed in object pointer and so unless the caller is aware, it can
> easily trick us into using the stale object afterwards. Move it into its
> solitary callsite where we know it is safe.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
> drivers/gpu/drm/i915/i915_gem.c | 20 +++++---------------
> 1 file changed, 5 insertions(+), 15 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 28a3079a7892..b6497e9961ab 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -306,20 +306,6 @@ i915_gem_object_unbind(struct drm_i915_gem_object *obj)
> return ret;
> }
>
> -static int
> -drop_pages(struct drm_i915_gem_object *obj)
> -{
> - int ret;
> -
> - i915_gem_object_get(obj);
> - ret = i915_gem_object_unbind(obj);
> - if (ret == 0)
> - ret = i915_gem_object_put_pages(obj);
> - i915_gem_object_put(obj);
> -
> - return ret;
> -}
> -
> int
> i915_gem_object_attach_phys(struct drm_i915_gem_object *obj,
> int align)
> @@ -340,7 +326,11 @@ i915_gem_object_attach_phys(struct drm_i915_gem_object *obj,
> if (obj->base.filp == NULL)
> return -EINVAL;
>
> - ret = drop_pages(obj);
> + ret = i915_gem_object_unbind(obj);
> + if (ret)
> + return ret;
> +
> + ret = i915_gem_object_put_pages(obj);
> if (ret)
> return ret;
>
>
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking
2016-07-07 8:41 ` [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking Chris Wilson
@ 2016-07-12 16:05 ` Tvrtko Ursulin
2016-07-12 16:30 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-12 16:05 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 07/07/16 09:41, Chris Wilson wrote:
> In the next patch, request tracking is made more generic and for that we
> need a new expanded struct and to separate out the logic changes from
> the mechanical churn, we split out the structure renaming into this
> patch.
>
> v2: Writer's block. Add some spiel about why we track requests.
> v3: Now i915_gem_active.
> v4: Now with i915_gem_active_set() for attaching to the active request.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
> drivers/gpu/drm/i915/i915_debugfs.c | 11 +++---
> drivers/gpu/drm/i915/i915_drv.h | 9 +++--
> drivers/gpu/drm/i915/i915_gem.c | 58 +++++++++++++++---------------
> drivers/gpu/drm/i915/i915_gem_dmabuf.c | 8 +++--
> drivers/gpu/drm/i915/i915_gem_execbuffer.c | 4 +--
> drivers/gpu/drm/i915/i915_gem_fence.c | 6 ++--
> drivers/gpu/drm/i915/i915_gem_request.h | 41 +++++++++++++++++++++
> drivers/gpu/drm/i915/i915_gem_tiling.c | 2 +-
> drivers/gpu/drm/i915/i915_gem_userptr.c | 2 +-
> drivers/gpu/drm/i915/i915_gpu_error.c | 7 ++--
> drivers/gpu/drm/i915/intel_display.c | 8 ++---
> 11 files changed, 98 insertions(+), 58 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
> index d3852828878f..dd832eace487 100644
> --- a/drivers/gpu/drm/i915/i915_debugfs.c
> +++ b/drivers/gpu/drm/i915/i915_debugfs.c
> @@ -155,10 +155,10 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
> obj->base.write_domain);
> for_each_engine_id(engine, dev_priv, id)
> seq_printf(m, "%x ",
> - i915_gem_request_get_seqno(obj->last_read_req[id]));
> + i915_gem_request_get_seqno(obj->last_read[id].request));
> seq_printf(m, "] %x %x%s%s%s",
> - i915_gem_request_get_seqno(obj->last_write_req),
> - i915_gem_request_get_seqno(obj->last_fenced_req),
> + i915_gem_request_get_seqno(obj->last_write.request),
> + i915_gem_request_get_seqno(obj->last_fence.request),
> i915_cache_level_str(to_i915(obj->base.dev), obj->cache_level),
> obj->dirty ? " dirty" : "",
> obj->madv == I915_MADV_DONTNEED ? " purgeable" : "");
> @@ -195,9 +195,8 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
> *t = '\0';
> seq_printf(m, " (%s mappable)", s);
> }
> - if (obj->last_write_req != NULL)
> - seq_printf(m, " (%s)",
> - i915_gem_request_get_engine(obj->last_write_req)->name);
> + if (obj->last_write.request)
> + seq_printf(m, " (%s)", obj->last_write.request->engine->name);
> if (obj->frontbuffer_bits)
> seq_printf(m, " (frontbuffer: 0x%03x)", obj->frontbuffer_bits);
> }
> diff --git a/drivers/gpu/drm/i915/i915_drv.h b/drivers/gpu/drm/i915/i915_drv.h
> index 27e1182544a2..5b096c65646c 100644
> --- a/drivers/gpu/drm/i915/i915_drv.h
> +++ b/drivers/gpu/drm/i915/i915_drv.h
> @@ -2238,11 +2238,10 @@ struct drm_i915_gem_object {
> * requests on one ring where the write request is older than the
> * read request. This allows for the CPU to read from an active
> * buffer by only waiting for the write to complete.
> - * */
> - struct drm_i915_gem_request *last_read_req[I915_NUM_ENGINES];
> - struct drm_i915_gem_request *last_write_req;
> - /** Breadcrumb of last fenced GPU access to the buffer. */
> - struct drm_i915_gem_request *last_fenced_req;
> + */
> + struct i915_gem_active last_read[I915_NUM_ENGINES];
> + struct i915_gem_active last_write;
> + struct i915_gem_active last_fence;
>
> /** Current tiling stride for the object, if it's tiled. */
> uint32_t stride;
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index b6497e9961ab..5f302faf86e7 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -1353,23 +1353,23 @@ i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
> int ret, i;
>
> if (readonly) {
> - if (obj->last_write_req != NULL) {
> - ret = i915_wait_request(obj->last_write_req);
> + if (obj->last_write.request) {
> + ret = i915_wait_request(obj->last_write.request);
> if (ret)
> return ret;
>
> - i = obj->last_write_req->engine->id;
> - if (obj->last_read_req[i] == obj->last_write_req)
> + i = obj->last_write.request->engine->id;
> + if (obj->last_read[i].request == obj->last_write.request)
> i915_gem_object_retire__read(obj, i);
> else
> i915_gem_object_retire__write(obj);
> }
> } else {
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> - if (obj->last_read_req[i] == NULL)
> + if (!obj->last_read[i].request)
> continue;
>
> - ret = i915_wait_request(obj->last_read_req[i]);
> + ret = i915_wait_request(obj->last_read[i].request);
> if (ret)
> return ret;
>
> @@ -1397,9 +1397,9 @@ i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
> {
> int ring = req->engine->id;
>
> - if (obj->last_read_req[ring] == req)
> + if (obj->last_read[ring].request == req)
> i915_gem_object_retire__read(obj, ring);
> - else if (obj->last_write_req == req)
> + else if (obj->last_write.request == req)
> i915_gem_object_retire__write(obj);
>
> if (!i915_reset_in_progress(&req->i915->gpu_error))
> @@ -1428,7 +1428,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
> if (readonly) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_write_req;
> + req = obj->last_write.request;
> if (req == NULL)
> return 0;
>
> @@ -1437,7 +1437,7 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_read_req[i];
> + req = obj->last_read[i].request;
> if (req == NULL)
> continue;
>
> @@ -2375,7 +2375,7 @@ void i915_vma_move_to_active(struct i915_vma *vma,
> obj->active |= intel_engine_flag(engine);
>
> list_move_tail(&obj->engine_list[engine->id], &engine->active_list);
> - i915_gem_request_assign(&obj->last_read_req[engine->id], req);
> + i915_gem_active_set(&obj->last_read[engine->id], req);
>
> list_move_tail(&vma->vm_link, &vma->vm->active_list);
> }
> @@ -2383,10 +2383,10 @@ void i915_vma_move_to_active(struct i915_vma *vma,
> static void
> i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
> {
> - GEM_BUG_ON(obj->last_write_req == NULL);
> - GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write_req->engine)));
> + GEM_BUG_ON(!obj->last_write.request);
> + GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
>
> - i915_gem_request_assign(&obj->last_write_req, NULL);
> + i915_gem_request_assign(&obj->last_write.request, NULL);
Why not use i915_gem_active_set here? It will be strange to have a mix.
> intel_fb_obj_flush(obj, true, ORIGIN_CS);
> }
>
> @@ -2395,13 +2395,13 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
> {
> struct i915_vma *vma;
>
> - GEM_BUG_ON(obj->last_read_req[ring] == NULL);
> + GEM_BUG_ON(!obj->last_read[ring].request);
> GEM_BUG_ON(!(obj->active & (1 << ring)));
>
> list_del_init(&obj->engine_list[ring]);
> - i915_gem_request_assign(&obj->last_read_req[ring], NULL);
> + i915_gem_request_assign(&obj->last_read[ring].request, NULL);
>
> - if (obj->last_write_req && obj->last_write_req->engine->id == ring)
> + if (obj->last_write.request && obj->last_write.request->engine->id == ring)
> i915_gem_object_retire__write(obj);
>
> obj->active &= ~(1 << ring);
> @@ -2420,7 +2420,7 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
> list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
> }
>
> - i915_gem_request_assign(&obj->last_fenced_req, NULL);
> + i915_gem_request_assign(&obj->last_fence.request, NULL);
> i915_gem_object_put(obj);
> }
>
> @@ -2618,7 +2618,7 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
> struct drm_i915_gem_object,
> engine_list[engine->id]);
>
> - if (!list_empty(&obj->last_read_req[engine->id]->list))
> + if (!list_empty(&obj->last_read[engine->id].request->list))
> break;
>
> i915_gem_object_retire__read(obj, engine->id);
> @@ -2746,7 +2746,7 @@ i915_gem_object_flush_active(struct drm_i915_gem_object *obj)
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_read_req[i];
> + req = obj->last_read[i].request;
> if (req == NULL)
> continue;
>
> @@ -2822,10 +2822,10 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
> i915_gem_object_put(obj);
>
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> - if (obj->last_read_req[i] == NULL)
> + if (!obj->last_read[i].request)
> continue;
>
> - req[n++] = i915_gem_request_get(obj->last_read_req[i]);
> + req[n++] = i915_gem_request_get(obj->last_read[i].request);
> }
>
> mutex_unlock(&dev->struct_mutex);
> @@ -2916,12 +2916,12 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
>
> n = 0;
> if (readonly) {
> - if (obj->last_write_req)
> - req[n++] = obj->last_write_req;
> + if (obj->last_write.request)
> + req[n++] = obj->last_write.request;
> } else {
> for (i = 0; i < I915_NUM_ENGINES; i++)
> - if (obj->last_read_req[i])
> - req[n++] = obj->last_read_req[i];
> + if (obj->last_read[i].request)
> + req[n++] = obj->last_read[i].request;
> }
> for (i = 0; i < n; i++) {
> ret = __i915_gem_object_sync(obj, to, req[i]);
> @@ -4022,12 +4022,12 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_read_req[i];
> + req = obj->last_read[i].request;
> if (req)
> args->busy |= 1 << (16 + req->engine->exec_id);
> }
> - if (obj->last_write_req)
> - args->busy |= obj->last_write_req->engine->exec_id;
> + if (obj->last_write.request)
> + args->busy |= obj->last_write.request->engine->exec_id;
> }
>
> unref:
> diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
> index 85a5c86ba80a..aa767ca28532 100644
> --- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
> @@ -227,13 +227,15 @@ static void export_fences(struct drm_i915_gem_object *obj,
> {
> struct reservation_object *resv = dma_buf->resv;
> struct drm_i915_gem_request *req;
> + unsigned long active;
> int idx;
>
> mutex_lock(&obj->base.dev->struct_mutex);
> mutex_lock(&resv->lock.base);
>
> - for (idx = 0; idx < ARRAY_SIZE(obj->last_read_req); idx++) {
> - req = obj->last_read_req[idx];
> + active = obj->active;
> + for_each_active(active, idx) {
> + req = obj->last_read[idx].request;
> if (!req)
> continue;
>
> @@ -241,7 +243,7 @@ static void export_fences(struct drm_i915_gem_object *obj,
> reservation_object_add_shared_fence(resv, &req->fence);
> }
>
> - req = obj->last_write_req;
> + req = obj->last_write.request;
> if (req)
> reservation_object_add_excl_fence(resv, &req->fence);
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> index 5bfb2e2f3e67..22261a511d7d 100644
> --- a/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/i915_gem_execbuffer.c
> @@ -1143,7 +1143,7 @@ i915_gem_execbuffer_move_to_active(struct list_head *vmas,
> i915_vma_move_to_active(vma, req);
> eb_export_fence(obj, req);
> if (obj->base.write_domain) {
> - i915_gem_request_assign(&obj->last_write_req, req);
> + i915_gem_active_set(&obj->last_write, req);
>
> intel_fb_obj_invalidate(obj, ORIGIN_CS);
>
> @@ -1151,7 +1151,7 @@ i915_gem_execbuffer_move_to_active(struct list_head *vmas,
> obj->base.write_domain &= ~I915_GEM_GPU_DOMAINS;
> }
> if (entry->flags & EXEC_OBJECT_NEEDS_FENCE) {
> - i915_gem_request_assign(&obj->last_fenced_req, req);
> + i915_gem_active_set(&obj->last_fence, req);
> if (entry->flags & __EXEC_OBJECT_HAS_FENCE) {
> struct drm_i915_private *dev_priv = engine->i915;
> list_move_tail(&dev_priv->fence_regs[obj->fence_reg].lru_list,
> diff --git a/drivers/gpu/drm/i915/i915_gem_fence.c b/drivers/gpu/drm/i915/i915_gem_fence.c
> index 251d7a95af89..9838046801bd 100644
> --- a/drivers/gpu/drm/i915/i915_gem_fence.c
> +++ b/drivers/gpu/drm/i915/i915_gem_fence.c
> @@ -261,12 +261,12 @@ static inline void i915_gem_object_fence_lost(struct drm_i915_gem_object *obj)
> static int
> i915_gem_object_wait_fence(struct drm_i915_gem_object *obj)
> {
> - if (obj->last_fenced_req) {
> - int ret = i915_wait_request(obj->last_fenced_req);
> + if (obj->last_fence.request) {
> + int ret = i915_wait_request(obj->last_fence.request);
> if (ret)
> return ret;
>
> - i915_gem_request_assign(&obj->last_fenced_req, NULL);
> + i915_gem_request_assign(&obj->last_fence.request, NULL);
> }
>
> return 0;
> diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
> index 928fc8baf435..ff8c54fa955f 100644
> --- a/drivers/gpu/drm/i915/i915_gem_request.h
> +++ b/drivers/gpu/drm/i915/i915_gem_request.h
> @@ -247,4 +247,45 @@ static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
> __i915_spin_request(request, state, timeout_us));
> }
>
> +/* We treat requests as fences. This is not be to confused with our
> + * "fence registers" but pipeline synchronisation objects ala GL_ARB_sync.
> + * We use the fences to synchronize access from the CPU with activity on the
> + * GPU, for example, we should not rewrite an object's PTE whilst the GPU
> + * is reading them. We also track fences at a higher level to provide
> + * implicit synchronisation around GEM objects, e.g. set-domain will wait
> + * for outstanding GPU rendering before marking the object ready for CPU
> + * access, or a pageflip will wait until the GPU is complete before showing
> + * the frame on the scanout.
> + *
> + * In order to use a fence, the object must track the fence it needs to
> + * serialise with. For example, GEM objects want to track both read and
> + * write access so that we can perform concurrent read operations between
> + * the CPU and GPU engines, as well as waiting for all rendering to
> + * complete, or waiting for the last GPU user of a "fence register". The
> + * object then embeds a @i915_gem_active to track the most recent (in
> + * retirment order) request relevant for the desired mode of access.
> + * The @i915_gem_active is updated with i915_gem_request_mark_active() to
i915_gem_active_set
> + * track the most recent fence request, typically this is done as part of
> + * i915_vma_move_to_active().
> + *
> + * When the @i915_gem_active completes (is retired), it will
> + * signal its completion to the owner through a callback as well as mark
> + * itself as idle (i915_gem_active.request == NULL). The owner
> + * can then perform any action, such as delayed freeing of an active
> + * resource including itself.
> + */
> +struct i915_gem_active {
> + struct drm_i915_gem_request *request;
> +};
> +
> +static inline void
> +i915_gem_active_set(struct i915_gem_active *active,
> + struct drm_i915_gem_request *request)
> +{
> + i915_gem_request_assign(&active->request, request);
> +}
> +
> +#define for_each_active(mask, idx) \
> + for (; mask ? idx = ffs(mask) - 1, 1 : 0; mask &= ~(1 << idx))
> +
> #endif /* I915_GEM_REQUEST_H */
> diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
> index e83fc2d0433c..00d796da65fb 100644
> --- a/drivers/gpu/drm/i915/i915_gem_tiling.c
> +++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
> @@ -242,7 +242,7 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
> }
>
> obj->fence_dirty =
> - obj->last_fenced_req ||
> + obj->last_fence.request ||
> obj->fence_reg != I915_FENCE_REG_NONE;
>
> obj->tiling_mode = args->tiling_mode;
> diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
> index e935b327f3f9..32f50a70ea42 100644
> --- a/drivers/gpu/drm/i915/i915_gem_userptr.c
> +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
> @@ -74,7 +74,7 @@ static void wait_rendering(struct drm_i915_gem_object *obj)
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_read_req[i];
> + req = obj->last_read[i].request;
> if (req == NULL)
> continue;
>
> diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
> index 2fbe81d51af1..5e12b8ee49d2 100644
> --- a/drivers/gpu/drm/i915/i915_gpu_error.c
> +++ b/drivers/gpu/drm/i915/i915_gpu_error.c
> @@ -749,8 +749,8 @@ static void capture_bo(struct drm_i915_error_buffer *err,
> err->size = obj->base.size;
> err->name = obj->base.name;
> for (i = 0; i < I915_NUM_ENGINES; i++)
> - err->rseqno[i] = i915_gem_request_get_seqno(obj->last_read_req[i]);
> - err->wseqno = i915_gem_request_get_seqno(obj->last_write_req);
> + err->rseqno[i] = i915_gem_request_get_seqno(obj->last_read[i].request);
> + err->wseqno = i915_gem_request_get_seqno(obj->last_write.request);
> err->gtt_offset = vma->node.start;
> err->read_domains = obj->base.read_domains;
> err->write_domain = obj->base.write_domain;
> @@ -762,8 +762,7 @@ static void capture_bo(struct drm_i915_error_buffer *err,
> err->dirty = obj->dirty;
> err->purgeable = obj->madv != I915_MADV_WILLNEED;
> err->userptr = obj->userptr.mm != NULL;
> - err->ring = obj->last_write_req ?
> - i915_gem_request_get_engine(obj->last_write_req)->id : -1;
> + err->ring = obj->last_write.request ? obj->last_write.request->engine->id : -1;
> err->cache_level = obj->cache_level;
> }
>
> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
> index 5f4bdc3d4972..27ac6db4e26a 100644
> --- a/drivers/gpu/drm/i915/intel_display.c
> +++ b/drivers/gpu/drm/i915/intel_display.c
> @@ -11465,7 +11465,7 @@ static bool use_mmio_flip(struct intel_engine_cs *engine,
> if (resv && !reservation_object_test_signaled_rcu(resv, false))
> return true;
>
> - return engine != i915_gem_request_get_engine(obj->last_write_req);
> + return engine != i915_gem_request_get_engine(obj->last_write.request);
> }
>
> static void skl_do_mmio_flip(struct intel_crtc *intel_crtc,
> @@ -11768,7 +11768,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
> } else if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev)) {
> engine = &dev_priv->engine[BCS];
> } else if (INTEL_INFO(dev)->gen >= 7) {
> - engine = i915_gem_request_get_engine(obj->last_write_req);
> + engine = i915_gem_request_get_engine(obj->last_write.request);
> if (engine == NULL || engine->id != RCS)
> engine = &dev_priv->engine[BCS];
> } else {
> @@ -11790,7 +11790,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
> INIT_WORK(&work->mmio_work, intel_mmio_flip_work_func);
>
> i915_gem_request_assign(&work->flip_queued_req,
> - obj->last_write_req);
> + obj->last_write.request);
>
> schedule_work(&work->mmio_work);
> } else {
> @@ -14096,7 +14096,7 @@ intel_prepare_plane_fb(struct drm_plane *plane,
> to_intel_plane_state(new_state);
>
> i915_gem_request_assign(&plane_state->wait_req,
> - obj->last_write_req);
> + obj->last_write.request);
> }
>
> return ret;
>
Otherwise looks fine.
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 46/64] drm/i915: Count how many VMA are bound for an object
2016-07-12 15:12 ` Tvrtko Ursulin
@ 2016-07-12 16:08 ` Chris Wilson
0 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-12 16:08 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Tue, Jul 12, 2016 at 04:12:43PM +0100, Tvrtko Ursulin wrote:
>
> On 12/07/16 15:38, Chris Wilson wrote:
> >On Tue, Jul 12, 2016 at 03:30:25PM +0100, Tvrtko Ursulin wrote:
> >>On 07/07/16 09:41, Chris Wilson wrote:
> >>>@@ -3684,6 +3684,9 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
> >>> old_read_domains,
> >>> old_write_domain);
> >>>
> >>>+ /* Increment the pages_pin_count to guard against the shrinker */
> >>>+ obj->pages_pin_count++;
> >>
> >>Would it be clearer to look at obj->pin_display in the shrinker?
> >>Although it looks like special casing out of the cleanliness of the
> >>design in both case so I am not sure.
> >
> >Yeah. I liked the mechanism of telling the shrinker to avoid certain
> >pages by only having to control the pages_pin_count. It feels easier to
> >explain to others "the shrinker may reap any object that hasn't pinned
> >its pages" (explicitly called i915_gem_object_pin_pages for its own use)
> >rather than that + plus a list of exceptions known by the shrinker.
>
> Hm but wait a minute, framebuffer already has the VMA pinned. So how
> will the shrinker reap it?
It won't, but it would try. Worse it would include the object in its
estimates for shrinkable pages.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking
2016-07-12 16:05 ` Tvrtko Ursulin
@ 2016-07-12 16:30 ` Chris Wilson
2016-07-13 8:54 ` Tvrtko Ursulin
0 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-12 16:30 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Tue, Jul 12, 2016 at 05:05:44PM +0100, Tvrtko Ursulin wrote:
>
> On 07/07/16 09:41, Chris Wilson wrote:
> >@@ -2383,10 +2383,10 @@ void i915_vma_move_to_active(struct i915_vma *vma,
> > static void
> > i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
> > {
> >- GEM_BUG_ON(obj->last_write_req == NULL);
> >- GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write_req->engine)));
> >+ GEM_BUG_ON(!obj->last_write.request);
> >+ GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
> >
> >- i915_gem_request_assign(&obj->last_write_req, NULL);
> >+ i915_gem_request_assign(&obj->last_write.request, NULL);
>
> Why not use i915_gem_active_set here? It will be strange to have a mix.
That would be strange imo. This is only a staging patch, but setting the
active from inside the retirement handler isn't clean and would look odd
later.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 47/64] drm/i915: Be more careful when unbinding vma
2016-07-12 15:04 ` Tvrtko Ursulin
@ 2016-07-12 16:42 ` Chris Wilson
2016-07-13 8:53 ` Tvrtko Ursulin
0 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-12 16:42 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Tue, Jul 12, 2016 at 04:04:57PM +0100, Tvrtko Ursulin wrote:
> >@@ -3463,11 +3483,18 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
> > return -EBUSY;
> > }
> >
> >- if (!i915_gem_valid_gtt_space(vma, cache_level)) {
> >- ret = i915_vma_unbind(vma);
> >- if (ret)
> >- return ret;
> >- }
> >+ if (i915_gem_valid_gtt_space(vma, cache_level))
> >+ continue;
> >+
> >+ ret = i915_vma_unbind(vma);
> >+ if (ret)
> >+ return ret;
> >+
> >+ /* As unbinding may affect other elements in the
> >+ * obj->vma_list (due to side-effects from retiring
> >+ * an active vma), play safe and restart the iterator.
> >+ */
> >+ goto restart;
> > }
>
> Does not look efficient for long lists but I don't see a solution
> right now. Any chance of this O(N^2) iteration hurting us in the
> real world?
Defintely not pretty, but couldn't think of a clean & safe alternative,
so went with simple for a change.
Fortunately, it is more or less a one-time conversion, operating on a
shared buffer between normally 2 clients. However, they will full-ppgtt
or we would not have multple vma, and so not affected.
Single client multiple vma case would be partial vma. That could be a
much larger list... But on the afffected machines, such vma would
already be in the correct cache regime and so not need rebinding.
I don't forsee (or have seen) anybody spinning here, so I am quite happy
to punt the problem until someone complains. At least I think I can
write a test case to have lots of partial vma, but not necessarily able
to trigger the restart. That would require careful contrl of
fragmentation with the ggtt... But mixing gtt faults of a bunch of small
objects and different partials of a large should be enough...
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [Intel-gfx] [PATCH 06/64] drm: Restore double clflush on the last partial cacheline
2016-07-12 13:57 ` Daniel Vetter
@ 2016-07-13 8:07 ` Mika Kuoppala
-1 siblings, 0 replies; 93+ messages in thread
From: Mika Kuoppala @ 2016-07-13 8:07 UTC (permalink / raw)
To: Daniel Vetter, Chris Wilson
Cc: Daniel Vetter, dri-devel, Jason Ekstrand, Akash Goel, stable, intel-gfx
Daniel Vetter <daniel@ffwll.ch> writes:
> On Thu, Jul 07, 2016 at 09:41:12AM +0100, Chris Wilson wrote:
>> This effectively reverts
>>
>> commit afcd950cafea6e27b739fe7772cbbeed37d05b8b
>> Author: Chris Wilson <chris@chris-wilson.co.uk>
>> Date: Wed Jun 10 15:58:01 2015 +0100
>>
>> drm: Avoid the double clflush on the last cache line in drm_clflush_virt_range()
>>
>> as we have observed issues with serialisation of the clflush operations
>> on Baytrail+ Atoms with partial updates. Applying the double flush on the
>> last cacheline forces that clflush to be ordered with respect to the
>> previous clflush, and the mfence then protects against prefetches crossing
>> the clflush boundary.
>>
>> The same issue can be demonstrated in userspace with igt/gem_exec_flush.
>>
>> Fixes: afcd950cafea6 (drm: Avoid the double clflush on the last cache...)
>> Testcase: igt/gem_concurrent_blit
>> Testcase: igt/gem_partial_pread_pwrite
>> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92845
>> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
>> Cc: dri-devel@lists.freedesktop.org
>> Cc: Akash Goel <akash.goel@intel.com>
>> Cc: Imre Deak <imre.deak@intel.com>
>> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
>> Cc: Jason Ekstrand <jason.ekstrand@intel.com>
>> Cc: stable@vger.kernel.org
>> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
>
> It's duct-tape, but oh well. Applied to drm-misc.
We were confused how many layers we need.
-Mika
> -Daniel
>
>> ---
>> drivers/gpu/drm/drm_cache.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c
>> index 059f7c39c582..a7916e5f8864 100644
>> --- a/drivers/gpu/drm/drm_cache.c
>> +++ b/drivers/gpu/drm/drm_cache.c
>> @@ -136,6 +136,7 @@ drm_clflush_virt_range(void *addr, unsigned long length)
>> mb();
>> for (; addr < end; addr += size)
>> clflushopt(addr);
>> + clflushopt(end - 1); /* force serialisation */
>> mb();
>> return;
>> }
>> --
>> 2.8.1
>>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 06/64] drm: Restore double clflush on the last partial cacheline
@ 2016-07-13 8:07 ` Mika Kuoppala
0 siblings, 0 replies; 93+ messages in thread
From: Mika Kuoppala @ 2016-07-13 8:07 UTC (permalink / raw)
To: Daniel Vetter, Chris Wilson
Cc: Daniel Vetter, intel-gfx, stable, Jason Ekstrand, Akash Goel, dri-devel
Daniel Vetter <daniel@ffwll.ch> writes:
> On Thu, Jul 07, 2016 at 09:41:12AM +0100, Chris Wilson wrote:
>> This effectively reverts
>>
>> commit afcd950cafea6e27b739fe7772cbbeed37d05b8b
>> Author: Chris Wilson <chris@chris-wilson.co.uk>
>> Date: Wed Jun 10 15:58:01 2015 +0100
>>
>> drm: Avoid the double clflush on the last cache line in drm_clflush_virt_range()
>>
>> as we have observed issues with serialisation of the clflush operations
>> on Baytrail+ Atoms with partial updates. Applying the double flush on the
>> last cacheline forces that clflush to be ordered with respect to the
>> previous clflush, and the mfence then protects against prefetches crossing
>> the clflush boundary.
>>
>> The same issue can be demonstrated in userspace with igt/gem_exec_flush.
>>
>> Fixes: afcd950cafea6 (drm: Avoid the double clflush on the last cache...)
>> Testcase: igt/gem_concurrent_blit
>> Testcase: igt/gem_partial_pread_pwrite
>> Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=92845
>> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
>> Cc: dri-devel@lists.freedesktop.org
>> Cc: Akash Goel <akash.goel@intel.com>
>> Cc: Imre Deak <imre.deak@intel.com>
>> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
>> Cc: Jason Ekstrand <jason.ekstrand@intel.com>
>> Cc: stable@vger.kernel.org
>> Reviewed-by: Mika Kuoppala <mika.kuoppala@intel.com>
>
> It's duct-tape, but oh well. Applied to drm-misc.
We were confused how many layers we need.
-Mika
> -Daniel
>
>> ---
>> drivers/gpu/drm/drm_cache.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/drivers/gpu/drm/drm_cache.c b/drivers/gpu/drm/drm_cache.c
>> index 059f7c39c582..a7916e5f8864 100644
>> --- a/drivers/gpu/drm/drm_cache.c
>> +++ b/drivers/gpu/drm/drm_cache.c
>> @@ -136,6 +136,7 @@ drm_clflush_virt_range(void *addr, unsigned long length)
>> mb();
>> for (; addr < end; addr += size)
>> clflushopt(addr);
>> + clflushopt(end - 1); /* force serialisation */
>> mb();
>> return;
>> }
>> --
>> 2.8.1
>>
>
> --
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch
> _______________________________________________
> Intel-gfx mailing list
> Intel-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 47/64] drm/i915: Be more careful when unbinding vma
2016-07-12 16:42 ` Chris Wilson
@ 2016-07-13 8:53 ` Tvrtko Ursulin
0 siblings, 0 replies; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-13 8:53 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 12/07/16 17:42, Chris Wilson wrote:
> On Tue, Jul 12, 2016 at 04:04:57PM +0100, Tvrtko Ursulin wrote:
>>> @@ -3463,11 +3483,18 @@ int i915_gem_object_set_cache_level(struct drm_i915_gem_object *obj,
>>> return -EBUSY;
>>> }
>>>
>>> - if (!i915_gem_valid_gtt_space(vma, cache_level)) {
>>> - ret = i915_vma_unbind(vma);
>>> - if (ret)
>>> - return ret;
>>> - }
>>> + if (i915_gem_valid_gtt_space(vma, cache_level))
>>> + continue;
>>> +
>>> + ret = i915_vma_unbind(vma);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + /* As unbinding may affect other elements in the
>>> + * obj->vma_list (due to side-effects from retiring
>>> + * an active vma), play safe and restart the iterator.
>>> + */
>>> + goto restart;
>>> }
>>
>> Does not look efficient for long lists but I don't see a solution
>> right now. Any chance of this O(N^2) iteration hurting us in the
>> real world?
>
> Defintely not pretty, but couldn't think of a clean & safe alternative,
> so went with simple for a change.
>
> Fortunately, it is more or less a one-time conversion, operating on a
> shared buffer between normally 2 clients. However, they will full-ppgtt
> or we would not have multple vma, and so not affected.
>
> Single client multiple vma case would be partial vma. That could be a
> much larger list... But on the afffected machines, such vma would
> already be in the correct cache regime and so not need rebinding.
>
> I don't forsee (or have seen) anybody spinning here, so I am quite happy
> to punt the problem until someone complains. At least I think I can
> write a test case to have lots of partial vma, but not necessarily able
> to trigger the restart. That would require careful contrl of
> fragmentation with the ggtt... But mixing gtt faults of a bunch of small
> objects and different partials of a large should be enough...
Or object sharing. But yeah, I agree that it feels OK to leave that for
later if it ever surfaces.
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking
2016-07-12 16:30 ` Chris Wilson
@ 2016-07-13 8:54 ` Tvrtko Ursulin
0 siblings, 0 replies; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-13 8:54 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 12/07/16 17:30, Chris Wilson wrote:
> On Tue, Jul 12, 2016 at 05:05:44PM +0100, Tvrtko Ursulin wrote:
>>
>> On 07/07/16 09:41, Chris Wilson wrote:
>>> @@ -2383,10 +2383,10 @@ void i915_vma_move_to_active(struct i915_vma *vma,
>>> static void
>>> i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
>>> {
>>> - GEM_BUG_ON(obj->last_write_req == NULL);
>>> - GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write_req->engine)));
>>> + GEM_BUG_ON(!obj->last_write.request);
>>> + GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
>>>
>>> - i915_gem_request_assign(&obj->last_write_req, NULL);
>>> + i915_gem_request_assign(&obj->last_write.request, NULL);
>>
>> Why not use i915_gem_active_set here? It will be strange to have a mix.
>
> That would be strange imo. This is only a staging patch, but setting the
> active from inside the retirement handler isn't clean and would look odd
> later.
Yes you are right. In that case;
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations
2016-07-07 8:41 ` [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations Chris Wilson
@ 2016-07-13 15:40 ` Tvrtko Ursulin
2016-07-13 15:58 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-13 15:40 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 07/07/16 09:41, Chris Wilson wrote:
> In the future, we will want to add annotations to the i915_gem_active
> struct. The API is thus expanded to hide direct access to the contents
> of i915_gem_active and mediated instead through a number of helpers.
>
> Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> ---
> drivers/gpu/drm/i915/i915_debugfs.c | 13 ++--
> drivers/gpu/drm/i915/i915_gem.c | 91 +++++++++++++----------
> drivers/gpu/drm/i915/i915_gem_dmabuf.c | 4 +-
> drivers/gpu/drm/i915/i915_gem_fence.c | 11 ++-
> drivers/gpu/drm/i915/i915_gem_request.h | 128 +++++++++++++++++++++++++++++++-
> drivers/gpu/drm/i915/i915_gem_tiling.c | 2 +-
> drivers/gpu/drm/i915/i915_gem_userptr.c | 8 +-
> drivers/gpu/drm/i915/i915_gpu_error.c | 9 ++-
> drivers/gpu/drm/i915/intel_display.c | 15 ++--
> 9 files changed, 208 insertions(+), 73 deletions(-)
>
> diff --git a/drivers/gpu/drm/i915/i915_debugfs.c b/drivers/gpu/drm/i915/i915_debugfs.c
> index dd832eace487..ae1c640fc1c8 100644
> --- a/drivers/gpu/drm/i915/i915_debugfs.c
> +++ b/drivers/gpu/drm/i915/i915_debugfs.c
> @@ -155,10 +155,10 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
> obj->base.write_domain);
> for_each_engine_id(engine, dev_priv, id)
> seq_printf(m, "%x ",
> - i915_gem_request_get_seqno(obj->last_read[id].request));
> + i915_gem_active_get_seqno(&obj->last_read[id]));
> seq_printf(m, "] %x %x%s%s%s",
> - i915_gem_request_get_seqno(obj->last_write.request),
> - i915_gem_request_get_seqno(obj->last_fence.request),
> + i915_gem_active_get_seqno(&obj->last_write),
> + i915_gem_active_get_seqno(&obj->last_fence),
> i915_cache_level_str(to_i915(obj->base.dev), obj->cache_level),
> obj->dirty ? " dirty" : "",
> obj->madv == I915_MADV_DONTNEED ? " purgeable" : "");
> @@ -195,8 +195,11 @@ describe_obj(struct seq_file *m, struct drm_i915_gem_object *obj)
> *t = '\0';
> seq_printf(m, " (%s mappable)", s);
> }
> - if (obj->last_write.request)
> - seq_printf(m, " (%s)", obj->last_write.request->engine->name);
> +
> + engine = i915_gem_active_get_engine(&obj->last_write);
> + if (engine)
> + seq_printf(m, " (%s)", engine->name);
> +
> if (obj->frontbuffer_bits)
> seq_printf(m, " (frontbuffer: 0x%03x)", obj->frontbuffer_bits);
> }
> diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> index 5f302faf86e7..9c371e84b1bb 100644
> --- a/drivers/gpu/drm/i915/i915_gem.c
> +++ b/drivers/gpu/drm/i915/i915_gem.c
> @@ -1349,27 +1349,30 @@ int
> i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
> bool readonly)
> {
> + struct drm_i915_gem_request *request;
> struct reservation_object *resv;
> int ret, i;
>
> if (readonly) {
> - if (obj->last_write.request) {
> - ret = i915_wait_request(obj->last_write.request);
> + request = i915_gem_active_peek(&obj->last_write);
Why not get_request since you have get_engine? Or later there will be
get_request with different semantics?
> + if (request) {
> + ret = i915_wait_request(request);
> if (ret)
> return ret;
>
> - i = obj->last_write.request->engine->id;
> - if (obj->last_read[i].request == obj->last_write.request)
> + i = request->engine->id;
> + if (i915_gem_active_peek(&obj->last_read[i]) == request)
> i915_gem_object_retire__read(obj, i);
> else
> i915_gem_object_retire__write(obj);
> }
> } else {
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> - if (!obj->last_read[i].request)
> + request = i915_gem_active_peek(&obj->last_read[i]);
> + if (!request)
> continue;
>
> - ret = i915_wait_request(obj->last_read[i].request);
> + ret = i915_wait_request(request);
> if (ret)
> return ret;
>
> @@ -1397,9 +1400,9 @@ i915_gem_object_retire_request(struct drm_i915_gem_object *obj,
> {
> int ring = req->engine->id;
>
> - if (obj->last_read[ring].request == req)
> + if (i915_gem_active_peek(&obj->last_read[ring]) == req)
> i915_gem_object_retire__read(obj, ring);
> - else if (obj->last_write.request == req)
> + else if (i915_gem_active_peek(&obj->last_write) == req)
> i915_gem_object_retire__write(obj);
>
> if (!i915_reset_in_progress(&req->i915->gpu_error))
> @@ -1428,20 +1431,20 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
> if (readonly) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_write.request;
> + req = i915_gem_active_peek(&obj->last_write);
> if (req == NULL)
> return 0;
>
> - requests[n++] = i915_gem_request_get(req);
> + requests[n++] = req;
It used to take a reference and now it doesn't.
> } else {
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_read[i].request;
> + req = i915_gem_active_peek(&obj->last_read[i]);
> if (req == NULL)
> continue;
>
> - requests[n++] = i915_gem_request_get(req);
> + requests[n++] = req;
> }
> }
>
> @@ -2383,25 +2386,27 @@ void i915_vma_move_to_active(struct i915_vma *vma,
> static void
> i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
> {
> - GEM_BUG_ON(!obj->last_write.request);
> - GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
> + GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_write));
> + GEM_BUG_ON(!(obj->active & intel_engine_flag(i915_gem_active_get_engine(&obj->last_write))));
>
> - i915_gem_request_assign(&obj->last_write.request, NULL);
> + i915_gem_active_set(&obj->last_write, NULL);
Aha!
> intel_fb_obj_flush(obj, true, ORIGIN_CS);
> }
>
> static void
> i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
> {
> + struct intel_engine_cs *engine;
> struct i915_vma *vma;
>
> - GEM_BUG_ON(!obj->last_read[ring].request);
> + GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_read[ring]));
> GEM_BUG_ON(!(obj->active & (1 << ring)));
>
> list_del_init(&obj->engine_list[ring]);
> - i915_gem_request_assign(&obj->last_read[ring].request, NULL);
> + i915_gem_active_set(&obj->last_read[ring], NULL);
>
> - if (obj->last_write.request && obj->last_write.request->engine->id == ring)
> + engine = i915_gem_active_get_engine(&obj->last_write);
> + if (engine && engine->id == ring)
> i915_gem_object_retire__write(obj);
>
> obj->active &= ~(1 << ring);
> @@ -2420,7 +2425,7 @@ i915_gem_object_retire__read(struct drm_i915_gem_object *obj, int ring)
> list_move_tail(&vma->vm_link, &vma->vm->inactive_list);
> }
>
> - i915_gem_request_assign(&obj->last_fence.request, NULL);
> + i915_gem_active_set(&obj->last_fence, NULL);
> i915_gem_object_put(obj);
> }
>
> @@ -2618,7 +2623,7 @@ i915_gem_retire_requests_ring(struct intel_engine_cs *engine)
> struct drm_i915_gem_object,
> engine_list[engine->id]);
>
> - if (!list_empty(&obj->last_read[engine->id].request->list))
> + if (!list_empty(&i915_gem_active_peek(&obj->last_read[engine->id])->list))
> break;
>
> i915_gem_object_retire__read(obj, engine->id);
> @@ -2746,7 +2751,7 @@ i915_gem_object_flush_active(struct drm_i915_gem_object *obj)
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_read[i].request;
> + req = i915_gem_active_peek(&obj->last_read[i]);
> if (req == NULL)
> continue;
>
> @@ -2786,7 +2791,7 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
> {
> struct drm_i915_gem_wait *args = data;
> struct drm_i915_gem_object *obj;
> - struct drm_i915_gem_request *req[I915_NUM_ENGINES];
> + struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
> int i, n = 0;
> int ret;
>
> @@ -2822,20 +2827,21 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
> i915_gem_object_put(obj);
>
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> - if (!obj->last_read[i].request)
> - continue;
> + struct drm_i915_gem_request *req;
>
> - req[n++] = i915_gem_request_get(obj->last_read[i].request);
> + req = i915_gem_active_get(&obj->last_read[i]);
Oh right there is a get request one, this time preserving the reference
taking behaviour.
> + if (req)
> + requests[n++] = req;
> }
>
> mutex_unlock(&dev->struct_mutex);
>
> for (i = 0; i < n; i++) {
> if (ret == 0)
> - ret = __i915_wait_request(req[i], true,
> + ret = __i915_wait_request(requests[i], true,
> args->timeout_ns > 0 ? &args->timeout_ns : NULL,
> to_rps_client(file));
> - i915_gem_request_put(req[i]);
> + i915_gem_request_put(requests[i]);
> }
> return ret;
>
> @@ -2908,7 +2914,7 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
> struct drm_i915_gem_request *to)
> {
> const bool readonly = obj->base.pending_write_domain == 0;
> - struct drm_i915_gem_request *req[I915_NUM_ENGINES];
> + struct drm_i915_gem_request *requests[I915_NUM_ENGINES];
> int ret, i, n;
>
> if (!obj->active)
> @@ -2916,15 +2922,22 @@ i915_gem_object_sync(struct drm_i915_gem_object *obj,
>
> n = 0;
> if (readonly) {
> - if (obj->last_write.request)
> - req[n++] = obj->last_write.request;
> + struct drm_i915_gem_request *req;
> +
> + req = i915_gem_active_peek(&obj->last_write);
> + if (req)
> + requests[n++] = req;
> } else {
> - for (i = 0; i < I915_NUM_ENGINES; i++)
> - if (obj->last_read[i].request)
> - req[n++] = obj->last_read[i].request;
> + for (i = 0; i < I915_NUM_ENGINES; i++) {
> + struct drm_i915_gem_request *req;
> +
> + req = i915_gem_active_peek(&obj->last_read[i]);
> + if (req)
> + requests[n++] = req;
> + }
> }
> for (i = 0; i < n; i++) {
> - ret = __i915_gem_object_sync(obj, to, req[i]);
> + ret = __i915_gem_object_sync(obj, to, requests[i]);
> if (ret)
> return ret;
> }
> @@ -4017,17 +4030,17 @@ i915_gem_busy_ioctl(struct drm_device *dev, void *data,
>
> args->busy = 0;
> if (obj->active) {
> + struct drm_i915_gem_request *req;
> int i;
>
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> - struct drm_i915_gem_request *req;
> -
> - req = obj->last_read[i].request;
> + req = i915_gem_active_peek(&obj->last_read[i]);
> if (req)
> args->busy |= 1 << (16 + req->engine->exec_id);
> }
> - if (obj->last_write.request)
> - args->busy |= obj->last_write.request->engine->exec_id;
> + req = i915_gem_active_peek(&obj->last_write);
> + if (req)
> + args->busy |= req->engine->exec_id;
> }
>
> unref:
> diff --git a/drivers/gpu/drm/i915/i915_gem_dmabuf.c b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
> index aa767ca28532..38000c59d456 100644
> --- a/drivers/gpu/drm/i915/i915_gem_dmabuf.c
> +++ b/drivers/gpu/drm/i915/i915_gem_dmabuf.c
> @@ -235,7 +235,7 @@ static void export_fences(struct drm_i915_gem_object *obj,
>
> active = obj->active;
> for_each_active(active, idx) {
> - req = obj->last_read[idx].request;
> + req = i915_gem_active_peek(&obj->last_read[idx]);
> if (!req)
> continue;
>
> @@ -243,7 +243,7 @@ static void export_fences(struct drm_i915_gem_object *obj,
> reservation_object_add_shared_fence(resv, &req->fence);
> }
>
> - req = obj->last_write.request;
> + req = i915_gem_active_peek(&obj->last_write);
> if (req)
> reservation_object_add_excl_fence(resv, &req->fence);
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_fence.c b/drivers/gpu/drm/i915/i915_gem_fence.c
> index 9838046801bd..9fdbd66128a6 100644
> --- a/drivers/gpu/drm/i915/i915_gem_fence.c
> +++ b/drivers/gpu/drm/i915/i915_gem_fence.c
> @@ -261,14 +261,13 @@ static inline void i915_gem_object_fence_lost(struct drm_i915_gem_object *obj)
> static int
> i915_gem_object_wait_fence(struct drm_i915_gem_object *obj)
> {
> - if (obj->last_fence.request) {
> - int ret = i915_wait_request(obj->last_fence.request);
> - if (ret)
> - return ret;
> + int ret;
>
> - i915_gem_request_assign(&obj->last_fence.request, NULL);
> - }
> + ret = i915_gem_active_wait(&obj->last_fence);
> + if (ret)
> + return ret;
>
> + i915_gem_active_set(&obj->last_fence, NULL);
> return 0;
> }
>
> diff --git a/drivers/gpu/drm/i915/i915_gem_request.h b/drivers/gpu/drm/i915/i915_gem_request.h
> index ff8c54fa955f..46d9b00a28d7 100644
> --- a/drivers/gpu/drm/i915/i915_gem_request.h
> +++ b/drivers/gpu/drm/i915/i915_gem_request.h
> @@ -275,14 +275,138 @@ static inline bool i915_spin_request(const struct drm_i915_gem_request *request,
> * resource including itself.
> */
> struct i915_gem_active {
> - struct drm_i915_gem_request *request;
> + struct drm_i915_gem_request *__request;
> };
>
> +/**
> + * i915_gem_active_set - updates the tracker to watch the current request
> + * @active - the active tracker
> + * @request - the request to watch
> + *
> + * i915_gem_active_set() watches the given @request for completion. Whilst
> + * that @request is busy, the @active reports busy. When that @request is
> + * retired, the @active tracker is updated to report idle.
> + */
> static inline void
> i915_gem_active_set(struct i915_gem_active *active,
> struct drm_i915_gem_request *request)
> {
> - i915_gem_request_assign(&active->request, request);
> + i915_gem_request_assign(&active->__request, request);
> +}
> +
> +/**
> + * i915_gem_active_peek - report the request being monitored
> + * @active - the active tracker
> + *
> + * i915_gem_active_peek() returns the current request being tracked, or NULL.
> + * It does not obtain a reference on the request for the caller, so the
> + * caller must hold struct_mutex.
> + */
> +static inline struct drm_i915_gem_request *
> +i915_gem_active_peek(const struct i915_gem_active *active)
> +{
> + return active->__request;
> +}
> +
> +/**
> + * i915_gem_active_get - return a reference to the active request
> + * @active - the active tracker
> + *
> + * i915_gem_active_get() returns a reference to the active request, or NULL
> + * if the active tracker is idle. The caller must hold struct_mutex.
> + */
> +static inline struct drm_i915_gem_request *
> +i915_gem_active_get(const struct i915_gem_active *active)
> +{
> + struct drm_i915_gem_request *request;
> +
> + request = i915_gem_active_peek(active);
> + if (!request || i915_gem_request_completed(request))
The check for request_completed feels like a hack - why it is needed?
> + return NULL;
> +
> + return i915_gem_request_get(request);
> +}
> +
> +/**
> + * __i915_gem_active_is_busy - report whether the active tracker is assigned
> + * @active - the active tracker
> + *
> + * __i915_gem_active_is_busy() returns true if the active tracker is currently
> + * assigned to a request. Due to the lazy retiring, that request may be idle
> + * and this may report stale information.
> + */
> +static inline bool
> +__i915_gem_active_is_busy(const struct i915_gem_active *active)
> +{
> + return i915_gem_active_peek(active);
> +}
> +
> +/**
> + * i915_gem_active_is_idle - report whether the active tracker is idle
> + * @active - the active tracker
> + *
> + * i915_gem_active_is_idle() returns true if the active tracker is currently
> + * unassigned or if the request is complete (but not yet retired). Requires
> + * the caller to hold struct_mutex (but that can be relaxed if desired).
> + */
> +static inline bool
> +i915_gem_active_is_idle(const struct i915_gem_active *active)
> +{
> + struct drm_i915_gem_request *request;
> +
> + request = i915_gem_active_peek(active);
> + if (!request || i915_gem_request_completed(request))
> + return true;
> +
> + return false;
> +}
> +
> +/**
> + * i915_gem_active_wait - waits until the request is completed
> + * @active - the active request on which to wait
> + *
> + * i915_gem_active_wait() waits until the request is completed before
> + * returning.
> + */
> +static inline int __must_check
> +i915_gem_active_wait(const struct i915_gem_active *active)
> +{
> + struct drm_i915_gem_request *request;
> +
> + request = i915_gem_active_peek(active);
> + if (!request)
> + return 0;
> +
> + return i915_wait_request(request);
> +}
> +
> +/**
> + * i915_gem_active_retire - waits until the request is retired
> + * @active - the active request on which to wait
> + *
> + * Unlike i915_gem_active_eait(), this i915_gem_active_retire() will
s/eait/wait/
> + * make sure the request is retired before returning.
> + */
> +static inline int __must_check
> +i915_gem_active_retire(const struct i915_gem_active *active)
> +{
> + return i915_gem_active_wait(active);
> +}
But how does it ensure anything different than i915_gem_active_wait when
it calls the very function? Maybe in future patches there will be a
difference?
> +
> +/* Convenience functions for peeking at state inside active's request whilst
> + * guarded by the struct_mutex.
> + */
> +
> +static inline uint32_t
> +i915_gem_active_get_seqno(const struct i915_gem_active *active)
> +{
> + return i915_gem_request_get_seqno(i915_gem_active_peek(active));
> +}
> +
> +static inline struct intel_engine_cs *
> +i915_gem_active_get_engine(const struct i915_gem_active *active)
> +{
> + return i915_gem_request_get_engine(i915_gem_active_peek(active));
> }
>
> #define for_each_active(mask, idx) \
> diff --git a/drivers/gpu/drm/i915/i915_gem_tiling.c b/drivers/gpu/drm/i915/i915_gem_tiling.c
> index 00d796da65fb..8cef2d6b291a 100644
> --- a/drivers/gpu/drm/i915/i915_gem_tiling.c
> +++ b/drivers/gpu/drm/i915/i915_gem_tiling.c
> @@ -242,7 +242,7 @@ i915_gem_set_tiling(struct drm_device *dev, void *data,
> }
>
> obj->fence_dirty =
> - obj->last_fence.request ||
> + !i915_gem_active_is_idle(&obj->last_fence) ||
> obj->fence_reg != I915_FENCE_REG_NONE;
>
> obj->tiling_mode = args->tiling_mode;
> diff --git a/drivers/gpu/drm/i915/i915_gem_userptr.c b/drivers/gpu/drm/i915/i915_gem_userptr.c
> index 32f50a70ea42..00ab5e9d2eb7 100644
> --- a/drivers/gpu/drm/i915/i915_gem_userptr.c
> +++ b/drivers/gpu/drm/i915/i915_gem_userptr.c
> @@ -74,11 +74,9 @@ static void wait_rendering(struct drm_i915_gem_object *obj)
> for (i = 0; i < I915_NUM_ENGINES; i++) {
> struct drm_i915_gem_request *req;
>
> - req = obj->last_read[i].request;
> - if (req == NULL)
> - continue;
> -
> - requests[n++] = i915_gem_request_get(req);
> + req = i915_gem_active_get(&obj->last_read[i]);
> + if (req)
> + requests[n++] = req;
> }
>
> mutex_unlock(&dev->struct_mutex);
> diff --git a/drivers/gpu/drm/i915/i915_gpu_error.c b/drivers/gpu/drm/i915/i915_gpu_error.c
> index 5e12b8ee49d2..9e1949f2f4dd 100644
> --- a/drivers/gpu/drm/i915/i915_gpu_error.c
> +++ b/drivers/gpu/drm/i915/i915_gpu_error.c
> @@ -744,13 +744,14 @@ static void capture_bo(struct drm_i915_error_buffer *err,
> struct i915_vma *vma)
> {
> struct drm_i915_gem_object *obj = vma->obj;
> + struct intel_engine_cs *engine;
> int i;
>
> err->size = obj->base.size;
> err->name = obj->base.name;
> for (i = 0; i < I915_NUM_ENGINES; i++)
> - err->rseqno[i] = i915_gem_request_get_seqno(obj->last_read[i].request);
> - err->wseqno = i915_gem_request_get_seqno(obj->last_write.request);
> + err->rseqno[i] = i915_gem_active_get_seqno(&obj->last_read[i]);
> + err->wseqno = i915_gem_active_get_seqno(&obj->last_write);
> err->gtt_offset = vma->node.start;
> err->read_domains = obj->base.read_domains;
> err->write_domain = obj->base.write_domain;
> @@ -762,8 +763,10 @@ static void capture_bo(struct drm_i915_error_buffer *err,
> err->dirty = obj->dirty;
> err->purgeable = obj->madv != I915_MADV_WILLNEED;
> err->userptr = obj->userptr.mm != NULL;
> - err->ring = obj->last_write.request ? obj->last_write.request->engine->id : -1;
> err->cache_level = obj->cache_level;
> +
> + engine = i915_gem_active_get_engine(&obj->last_write);
> + err->ring = engine ? engine->id : -1;
> }
>
> static u32 capture_active_bo(struct drm_i915_error_buffer *err,
> diff --git a/drivers/gpu/drm/i915/intel_display.c b/drivers/gpu/drm/i915/intel_display.c
> index 27ac6db4e26a..a0abe5588d17 100644
> --- a/drivers/gpu/drm/i915/intel_display.c
> +++ b/drivers/gpu/drm/i915/intel_display.c
> @@ -11465,7 +11465,7 @@ static bool use_mmio_flip(struct intel_engine_cs *engine,
> if (resv && !reservation_object_test_signaled_rcu(resv, false))
> return true;
>
> - return engine != i915_gem_request_get_engine(obj->last_write.request);
> + return engine != i915_gem_active_get_engine(&obj->last_write);
> }
>
> static void skl_do_mmio_flip(struct intel_crtc *intel_crtc,
> @@ -11768,7 +11768,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
> } else if (IS_IVYBRIDGE(dev) || IS_HASWELL(dev)) {
> engine = &dev_priv->engine[BCS];
> } else if (INTEL_INFO(dev)->gen >= 7) {
> - engine = i915_gem_request_get_engine(obj->last_write.request);
> + engine = i915_gem_active_get_engine(&obj->last_write);
> if (engine == NULL || engine->id != RCS)
> engine = &dev_priv->engine[BCS];
> } else {
> @@ -11789,9 +11789,7 @@ static int intel_crtc_page_flip(struct drm_crtc *crtc,
> if (mmio_flip) {
> INIT_WORK(&work->mmio_work, intel_mmio_flip_work_func);
>
> - i915_gem_request_assign(&work->flip_queued_req,
> - obj->last_write.request);
> -
> + work->flip_queued_req = i915_gem_active_get(&obj->last_write);
> schedule_work(&work->mmio_work);
> } else {
> request = i915_gem_request_alloc(engine, engine->last_context);
> @@ -14092,11 +14090,8 @@ intel_prepare_plane_fb(struct drm_plane *plane,
> }
>
> if (ret == 0) {
> - struct intel_plane_state *plane_state =
> - to_intel_plane_state(new_state);
> -
> - i915_gem_request_assign(&plane_state->wait_req,
> - obj->last_write.request);
> + to_intel_plane_state(new_state)->wait_req =
> + i915_gem_active_get(&obj->last_write);
> }
>
> return ret;
>
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations
2016-07-13 15:40 ` Tvrtko Ursulin
@ 2016-07-13 15:58 ` Chris Wilson
2016-07-14 9:32 ` Tvrtko Ursulin
0 siblings, 1 reply; 93+ messages in thread
From: Chris Wilson @ 2016-07-13 15:58 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Wed, Jul 13, 2016 at 04:40:03PM +0100, Tvrtko Ursulin wrote:
>
> On 07/07/16 09:41, Chris Wilson wrote:
> >In the future, we will want to add annotations to the i915_gem_active
> >struct. The API is thus expanded to hide direct access to the contents
> >of i915_gem_active and mediated instead through a number of helpers.
> >
> >Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
> >---
> >diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c
> >index 5f302faf86e7..9c371e84b1bb 100644
> >--- a/drivers/gpu/drm/i915/i915_gem.c
> >+++ b/drivers/gpu/drm/i915/i915_gem.c
> >@@ -1349,27 +1349,30 @@ int
> > i915_gem_object_wait_rendering(struct drm_i915_gem_object *obj,
> > bool readonly)
> > {
> >+ struct drm_i915_gem_request *request;
> > struct reservation_object *resv;
> > int ret, i;
> >
> > if (readonly) {
> >- if (obj->last_write.request) {
> >- ret = i915_wait_request(obj->last_write.request);
> >+ request = i915_gem_active_peek(&obj->last_write);
>
> Why not get_request since you have get_engine? Or later there will
> be get_request with different semantics?
It wasn't meant to be adding the reference...
> >@@ -1428,20 +1431,20 @@ i915_gem_object_wait_rendering__nonblocking(struct drm_i915_gem_object *obj,
> > if (readonly) {
> > struct drm_i915_gem_request *req;
> >
> >- req = obj->last_write.request;
> >+ req = i915_gem_active_peek(&obj->last_write);
> > if (req == NULL)
> > return 0;
> >
> >- requests[n++] = i915_gem_request_get(req);
> >+ requests[n++] = req;
>
> It used to take a reference and now it doesn't.
and this should still take the reference...
This patch wasn't meant to be making such changes, only translating
between functions.
> > } else {
> > for (i = 0; i < I915_NUM_ENGINES; i++) {
> > struct drm_i915_gem_request *req;
> >
> >- req = obj->last_read[i].request;
> >+ req = i915_gem_active_peek(&obj->last_read[i]);
> > if (req == NULL)
> > continue;
> >
> >- requests[n++] = i915_gem_request_get(req);
> >+ requests[n++] = req;
> > }
> > }
> >
> >@@ -2383,25 +2386,27 @@ void i915_vma_move_to_active(struct i915_vma *vma,
> > static void
> > i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
> > {
> >- GEM_BUG_ON(!obj->last_write.request);
> >- GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
> >+ GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_write));
> >+ GEM_BUG_ON(!(obj->active & intel_engine_flag(i915_gem_active_get_engine(&obj->last_write))));
> >
> >- i915_gem_request_assign(&obj->last_write.request, NULL);
> >+ i915_gem_active_set(&obj->last_write, NULL);
>
> Aha!
Drat. Didn't think I did that...
Oh well, no excuses now but to go back in time and make the change
earlier. It does get removed eventually!
> >@@ -2822,20 +2827,21 @@ i915_gem_wait_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
> > i915_gem_object_put(obj);
> >
> > for (i = 0; i < I915_NUM_ENGINES; i++) {
> >- if (!obj->last_read[i].request)
> >- continue;
> >+ struct drm_i915_gem_request *req;
> >
> >- req[n++] = i915_gem_request_get(obj->last_read[i].request);
> >+ req = i915_gem_active_get(&obj->last_read[i]);
>
> Oh right there is a get request one, this time preserving the
> reference taking behaviour.
Yup, you spotted my mix up earlier.
> >+/**
> >+ * i915_gem_active_get - return a reference to the active request
> >+ * @active - the active tracker
> >+ *
> >+ * i915_gem_active_get() returns a reference to the active request, or NULL
> >+ * if the active tracker is idle. The caller must hold struct_mutex.
> >+ */
> >+static inline struct drm_i915_gem_request *
> >+i915_gem_active_get(const struct i915_gem_active *active)
> >+{
> >+ struct drm_i915_gem_request *request;
> >+
> >+ request = i915_gem_active_peek(active);
> >+ if (!request || i915_gem_request_completed(request))
>
> The check for request_completed feels like a hack - why it is needed?
Because retirement is lazy and the GPU runs asynchronously. We want to
whilst building an execbuf check whether or not the wait is required. If
we detect a wait we have to backout of an atomic section inside execbuf.
Whilst execbuf is more demanding, the shortcircuit applies equally in
many places. This becomes especially apparent when looking at the
request locklessly, and I wanted to keep this path equivalent.
> >+/**
> >+ * i915_gem_active_retire - waits until the request is retired
> >+ * @active - the active request on which to wait
> >+ *
> >+ * Unlike i915_gem_active_eait(), this i915_gem_active_retire() will
>
> s/eait/wait/
>
> >+ * make sure the request is retired before returning.
> >+ */
> >+static inline int __must_check
> >+i915_gem_active_retire(const struct i915_gem_active *active)
> >+{
> >+ return i915_gem_active_wait(active);
> >+}
>
> But how does it ensure anything different than i915_gem_active_wait
> when it calls the very function? Maybe in future patches there will
> be a difference?
Right. It changes in the future, at the moment it is just trying to
illustrate the semantic difference - in some places the retirement is
required, but only of this active tracker, in others we need to retire
all requests up to this point, others simply need to know the request is
passed.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations
2016-07-13 15:58 ` Chris Wilson
@ 2016-07-14 9:32 ` Tvrtko Ursulin
2016-07-14 10:04 ` Chris Wilson
0 siblings, 1 reply; 93+ messages in thread
From: Tvrtko Ursulin @ 2016-07-14 9:32 UTC (permalink / raw)
To: Chris Wilson, intel-gfx
On 13/07/16 16:58, Chris Wilson wrote:
> On Wed, Jul 13, 2016 at 04:40:03PM +0100, Tvrtko Ursulin wrote:
[snip]
>>> } else {
>>> for (i = 0; i < I915_NUM_ENGINES; i++) {
>>> struct drm_i915_gem_request *req;
>>>
>>> - req = obj->last_read[i].request;
>>> + req = i915_gem_active_peek(&obj->last_read[i]);
>>> if (req == NULL)
>>> continue;
>>>
>>> - requests[n++] = i915_gem_request_get(req);
>>> + requests[n++] = req;
>>> }
>>> }
>>>
>>> @@ -2383,25 +2386,27 @@ void i915_vma_move_to_active(struct i915_vma *vma,
>>> static void
>>> i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
>>> {
>>> - GEM_BUG_ON(!obj->last_write.request);
>>> - GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
>>> + GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_write));
>>> + GEM_BUG_ON(!(obj->active & intel_engine_flag(i915_gem_active_get_engine(&obj->last_write))));
>>>
>>> - i915_gem_request_assign(&obj->last_write.request, NULL);
>>> + i915_gem_active_set(&obj->last_write, NULL);
>>
>> Aha!
>
> Drat. Didn't think I did that...
>
> Oh well, no excuses now but to go back in time and make the change
> earlier. It does get removed eventually!
Probably not worth it. You can have a special dispensation since I am
reviewing all the same lines of code patch after patch anyway. :)
Regards,
Tvrtko
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
* Re: [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations
2016-07-14 9:32 ` Tvrtko Ursulin
@ 2016-07-14 10:04 ` Chris Wilson
0 siblings, 0 replies; 93+ messages in thread
From: Chris Wilson @ 2016-07-14 10:04 UTC (permalink / raw)
To: Tvrtko Ursulin; +Cc: intel-gfx
On Thu, Jul 14, 2016 at 10:32:05AM +0100, Tvrtko Ursulin wrote:
>
> On 13/07/16 16:58, Chris Wilson wrote:
> >On Wed, Jul 13, 2016 at 04:40:03PM +0100, Tvrtko Ursulin wrote:
>
> [snip]
>
> >>> } else {
> >>> for (i = 0; i < I915_NUM_ENGINES; i++) {
> >>> struct drm_i915_gem_request *req;
> >>>
> >>>- req = obj->last_read[i].request;
> >>>+ req = i915_gem_active_peek(&obj->last_read[i]);
> >>> if (req == NULL)
> >>> continue;
> >>>
> >>>- requests[n++] = i915_gem_request_get(req);
> >>>+ requests[n++] = req;
> >>> }
> >>> }
> >>>
> >>>@@ -2383,25 +2386,27 @@ void i915_vma_move_to_active(struct i915_vma *vma,
> >>> static void
> >>> i915_gem_object_retire__write(struct drm_i915_gem_object *obj)
> >>> {
> >>>- GEM_BUG_ON(!obj->last_write.request);
> >>>- GEM_BUG_ON(!(obj->active & intel_engine_flag(obj->last_write.request->engine)));
> >>>+ GEM_BUG_ON(!__i915_gem_active_is_busy(&obj->last_write));
> >>>+ GEM_BUG_ON(!(obj->active & intel_engine_flag(i915_gem_active_get_engine(&obj->last_write))));
> >>>
> >>>- i915_gem_request_assign(&obj->last_write.request, NULL);
> >>>+ i915_gem_active_set(&obj->last_write, NULL);
> >>
> >>Aha!
> >
> >Drat. Didn't think I did that...
> >
> >Oh well, no excuses now but to go back in time and make the change
> >earlier. It does get removed eventually!
>
> Probably not worth it. You can have a special dispensation since I
> am reviewing all the same lines of code patch after patch anyway. :)
Too late, since this patch had to be fixed, I did the earlier fixup as
well.
-Chris
--
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply [flat|nested] 93+ messages in thread
end of thread, other threads:[~2016-07-14 10:04 UTC | newest]
Thread overview: 93+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-07-07 8:41 [PATCH 01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Chris Wilson
2016-07-07 8:41 ` [PATCH 02/64] drm/i915: Preserve current RPS frequency across init Chris Wilson
2016-07-07 8:41 ` [PATCH 03/64] drm/i915: Remove superfluous powersave work flushing Chris Wilson
2016-07-07 8:41 ` [PATCH 04/64] drm/i915: Defer enabling rc6 til after we submit the first batch/context Chris Wilson
2016-07-07 8:41 ` [PATCH 05/64] drm/i915: Remove temporary RPM wakeref assert disables Chris Wilson
2016-07-07 8:41 ` [PATCH 06/64] drm: Restore double clflush on the last partial cacheline Chris Wilson
2016-07-12 13:57 ` Daniel Vetter
2016-07-12 13:57 ` Daniel Vetter
2016-07-13 8:07 ` [Intel-gfx] " Mika Kuoppala
2016-07-13 8:07 ` Mika Kuoppala
2016-07-07 8:41 ` [PATCH 07/64] drm/i915: Move GEM request routines to i915_gem_request.c Chris Wilson
2016-07-07 8:41 ` [PATCH 08/64] drm/i915: Retire oldest completed request before allocating next Chris Wilson
2016-07-07 9:41 ` Tvrtko Ursulin
2016-07-07 9:45 ` Chris Wilson
2016-07-07 10:03 ` Tvrtko Ursulin
2016-07-07 10:10 ` Chris Wilson
2016-07-08 9:21 ` Mika Kuoppala
2016-07-08 9:28 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 09/64] drm/i915: Mark all current requests as complete before resetting them Chris Wilson
2016-07-07 16:09 ` Mika Kuoppala
2016-07-07 8:41 ` [PATCH 10/64] drm/i915: Derive GEM requests from dma-fence Chris Wilson
2016-07-07 8:41 ` [PATCH 11/64] drm/i915: Disable waitboosting for fence_wait() Chris Wilson
2016-07-07 8:41 ` [PATCH 12/64] drm/i915: Disable waitboosting for mmioflips/semaphores Chris Wilson
2016-07-07 8:41 ` [PATCH 13/64] drm/i915: Export our request as a dma-buf fence on the reservation object Chris Wilson
2016-07-07 8:41 ` [PATCH 14/64] drm/i915: Wait on external rendering for GEM objects Chris Wilson
2016-07-07 8:41 ` [PATCH 15/64] drm/i915: Mark imported dma-buf objects as being coherent Chris Wilson
2016-07-07 8:41 ` [PATCH 16/64] drm/i915: Rename request reference/unreference to get/put Chris Wilson
2016-07-07 8:41 ` [PATCH 17/64] drm/i915: Rename i915_gem_context_reference/unreference() Chris Wilson
2016-07-07 8:41 ` [PATCH 18/64] drm/i915: Wrap drm_gem_object_lookup in i915_gem_object_lookup Chris Wilson
2016-07-07 8:41 ` [PATCH 19/64] drm/i915: Wrap drm_gem_object_reference in i915_gem_object_get Chris Wilson
2016-07-07 8:41 ` [PATCH 20/64] drm/i915: Rename drm_gem_object_unreference in preparation for lockless free Chris Wilson
2016-07-07 8:41 ` [PATCH 21/64] drm/i915: Rename drm_gem_object_unreference_unlocked " Chris Wilson
2016-07-07 8:41 ` [PATCH 22/64] drm/i915: Treat ringbuffer writes as write to normal memory Chris Wilson
2016-07-07 8:41 ` [PATCH 23/64] drm/i915: Rename ring->virtual_start as ring->vaddr Chris Wilson
2016-07-07 8:41 ` [PATCH 24/64] drm/i915: Convert i915_semaphores_is_enabled over to early sanitize Chris Wilson
2016-07-07 8:41 ` [PATCH 25/64] drm/i915: Unify intel_logical_ring_emit and intel_ring_emit Chris Wilson
2016-07-07 8:41 ` [PATCH 26/64] drm/i915: Rename request->ringbuf to request->ring Chris Wilson
2016-07-07 8:41 ` [PATCH 27/64] drm/i915: Rename backpointer from intel_ringbuffer to intel_engine_cs Chris Wilson
2016-07-07 8:41 ` [PATCH 28/64] drm/i915: Rename intel_context[engine].ringbuf Chris Wilson
2016-07-07 8:41 ` [PATCH 29/64] drm/i915: Rename struct intel_ringbuffer to struct intel_ring Chris Wilson
2016-07-07 8:41 ` [PATCH 30/64] drm/i915: Rename residual ringbuf parameters Chris Wilson
2016-07-07 8:41 ` [PATCH 31/64] drm/i915: Rename intel_pin_and_map_ring() Chris Wilson
2016-07-07 8:41 ` [PATCH 32/64] drm/i915: Remove obsolete engine->gpu_caches_dirty Chris Wilson
2016-07-07 8:41 ` [PATCH 33/64] drm/i915: Simplify request_alloc by returning the allocated request Chris Wilson
2016-07-07 8:41 ` [PATCH 34/64] drm/i915: Unify legacy/execlists emission of MI_BATCHBUFFER_START Chris Wilson
2016-07-07 8:41 ` [PATCH 35/64] drm/i915: Convert engine->write_tail to operate on a request Chris Wilson
2016-07-07 8:41 ` [PATCH 36/64] drm/i915: Unify request submission Chris Wilson
2016-07-07 8:41 ` [PATCH 37/64] drm/i915: Stop passing caller's num_dwords to engine->semaphore.signal() Chris Wilson
2016-07-07 8:41 ` [PATCH 38/64] drm/i915: Reuse legacy breadcrumbs + tail emission Chris Wilson
2016-07-07 8:41 ` [PATCH 39/64] drm/i915/ringbuffer: Specialise SNB+ request emission for semaphores Chris Wilson
2016-07-07 8:41 ` [PATCH 40/64] drm/i915: Remove duplicate golden render state init from execlists Chris Wilson
2016-07-07 8:41 ` [PATCH 41/64] drm/i915: Unify legacy/execlists submit_execbuf callbacks Chris Wilson
2016-07-07 8:41 ` [PATCH 42/64] drm/i915: Simplify calling engine->sync_to Chris Wilson
2016-07-07 8:41 ` [PATCH 43/64] drm/i915: Amalgamate GGTT/ppGTT vma debug list walkers Chris Wilson
2016-07-07 8:41 ` [PATCH 44/64] drm/i915: Split early global GTT initialisation Chris Wilson
2016-07-07 8:41 ` [PATCH 45/64] drm/i915: Store owning file on the i915_address_space Chris Wilson
2016-07-07 8:41 ` [PATCH 46/64] drm/i915: Count how many VMA are bound for an object Chris Wilson
2016-07-12 14:30 ` Tvrtko Ursulin
2016-07-12 14:38 ` Chris Wilson
2016-07-12 15:12 ` Tvrtko Ursulin
2016-07-12 16:08 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 47/64] drm/i915: Be more careful when unbinding vma Chris Wilson
2016-07-12 15:04 ` Tvrtko Ursulin
2016-07-12 16:42 ` Chris Wilson
2016-07-13 8:53 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 48/64] drm/i915: Kill drop_pages() Chris Wilson
2016-07-12 15:14 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 49/64] drm/i915: Introduce i915_gem_active for request tracking Chris Wilson
2016-07-12 16:05 ` Tvrtko Ursulin
2016-07-12 16:30 ` Chris Wilson
2016-07-13 8:54 ` Tvrtko Ursulin
2016-07-07 8:41 ` [PATCH 50/64] drm/i915: Prepare i915_gem_active for annotations Chris Wilson
2016-07-13 15:40 ` Tvrtko Ursulin
2016-07-13 15:58 ` Chris Wilson
2016-07-14 9:32 ` Tvrtko Ursulin
2016-07-14 10:04 ` Chris Wilson
2016-07-07 8:41 ` [PATCH 51/64] drm/i915: Mark up i915_gem_active for locking annotation Chris Wilson
2016-07-07 8:41 ` [PATCH 52/64] drm/i915: Refactor blocking waits Chris Wilson
2016-07-07 8:41 ` [PATCH 53/64] drm/i915: Rename request->list to link for consistency Chris Wilson
2016-07-07 8:42 ` [PATCH 54/64] drm/i915: Remove obsolete i915_gem_object_flush_active() Chris Wilson
2016-07-07 8:42 ` [PATCH 55/64] drm/i915: Refactor activity tracking for requests Chris Wilson
2016-07-07 8:42 ` [PATCH 56/64] drm/i915: Convert intel_overlay to request tracking Chris Wilson
2016-07-07 8:42 ` [PATCH 57/64] drm/i915: Move the special case wait-request handling to its one caller Chris Wilson
2016-07-07 8:42 ` [PATCH 58/64] drm/i915: Double check activity before relocations Chris Wilson
2016-07-07 8:42 ` [PATCH 59/64] drm/i915: Move request list retirement to i915_gem_request.c Chris Wilson
2016-07-07 8:42 ` [PATCH 60/64] drm/i915: i915_vma_move_to_active prep patch Chris Wilson
2016-07-07 8:42 ` [PATCH 61/64] drm/i915: Track active vma requests Chris Wilson
2016-07-07 8:42 ` [PATCH 62/64] drm/i915: Release vma when the handle is closed Chris Wilson
2016-07-07 8:42 ` [PATCH 63/64] drm/i915: Mark the context and address space as closed Chris Wilson
2016-07-07 8:42 ` [PATCH 64/64] Revert "drm/i915: Clean up associated VMAs on context destruction" Chris Wilson
2016-07-07 9:12 ` ✗ Ro.CI.BAT: warning for series starting with [01/64] drm/i915/breadcrumbs: Queue hangcheck before sleeping Patchwork
2016-07-07 9:28 ` Chris Wilson
2016-07-08 8:44 ` Chris Wilson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.