[PATCH V2 00/23] drm/etnaviv: support performance counters

[PATCH V2 00/23] drm/etnaviv: support performance counters

[Christian Gmeiner]

In a perfect world we would be able to read GPU registers of interest
via the command stream with a 'read-register' command/package. For perf
counters it is a must to read them synchronized with the GPU to put the
values in relation to a draw command. As Vivante GPUs do not provide this
functionality we need to emulate it in software.


We need to support three different kind of perf register types:

1) normal register
  This is the easierst case where we can simply read the register and we
  are done.

2) debug register
  We need to configure the mux register and then read the debug register value.

3) pipeline register
  We need to 'iterate' over all pixel pipes and sum up the values. The 'iteration'
  is done by select the pipe of interest via HI_CLOCK_CONTROL_DEBUG_PIXEL_PIPE.
  There is also need to configure the mux register.


Allowing the userspace to do it all by its own feels quite error prone and not
future-proof. Thats why the kernel exports all performance domains and their
signals to the userspace via two new ioctls. So the kernel knows all performance
counters and how to sample them.

At the moment all performacne domains and signals get exported to all gpu pipe types,
but that can be changed in follow-up patches.

struct drm_etnaviv_gem_submit was extended to include so-called performance monitor
requests (pmrs). A request defines what domain and signal should be sampled (pre/post
draw cmdbuffer) and where to store the result.

The whole series can be found here:
https://github.com/austriancoder/linux/tree/perfmon-v2

The used libdrm and mesa branches to test this feature can be found here:
https://github.com/austriancoder/libdrm/commits/perfmon-v2
https://github.com/austriancoder/mesa/commits/perfmon-v2

Changes v1 -> v2:
 - reworked events
 - reworked uapi
 - reworked enumeration of domains and signals
 - process sync point with a work item to keep irq as fast as possible
 - prevent GPU hang when reading pixel pipeline perf values
 - all SH perf counters are accessed via perf_reg_read(..)

Happy reviewing!

Christian Gmeiner (23):
  drm/etnaviv: use bitmap to keep track of events
  drm/etnaviv: make it possible to allocate multiple events
  drm/etnaviv: add infrastructure to query perf counter
  drm/etnaviv: add uapi for perfmon feature
  drm/etnaviv: add internal representation of perfmon_request
  drm/etnaviv: extend etnaviv_gpu_cmdbuf_new(..) with nr_pmrs
  drm/etnaviv: add performance monitor request validation
  drm/etnaviv: copy pmrs from userspace
  drm/etnaviv: add performance monitor request processing
  drm/etnaviv: add 'sync point' support
  drm/etnaviv: clear alloced event
  drm/etnaviv: use 'sync points' for performance monitor requests
  drm/etnaviv: add HI perf domain
  drm/etnaviv: add PE perf domain
  drm/etnaviv: add SH perf domain
  drm/etnaviv: add PA perf domain
  drm/etnaviv: add SE perf domain
  drm/etnaviv: add RA perf domain
  drm/etnaviv: add TX perf domain
  drm/etnaviv: add MC perf domain
  drm/etnaviv: need to disable clock gating when doing profiling
  drm/etnaviv: enable debug registers on demand
  drm/etnaviv: submit supports performance monitor requests

 drivers/gpu/drm/etnaviv/Makefile             |   3 +-
 drivers/gpu/drm/etnaviv/etnaviv_buffer.c     |  36 +++
 drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.c     |  15 +-
 drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h     |   6 +-
 drivers/gpu/drm/etnaviv/etnaviv_drv.c        |  39 ++-
 drivers/gpu/drm/etnaviv/etnaviv_drv.h        |   1 +
 drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c |  69 +++-
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c        | 202 +++++++++---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.h        |  13 +-
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c    | 456 +++++++++++++++++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.h    |  48 +++
 include/uapi/drm/etnaviv_drm.h               |  43 ++-
 12 files changed, 879 insertions(+), 52 deletions(-)
 create mode 100644 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
 create mode 100644 drivers/gpu/drm/etnaviv/etnaviv_perfmon.h

-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 01/23] drm/etnaviv: use bitmap to keep track of events

[Christian Gmeiner]

This is prep work to be able to allocate multiple events in one go.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 31 +++++++++++++------------------
 drivers/gpu/drm/etnaviv/etnaviv_gpu.h |  6 ++++--
 2 files changed, 17 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index ada45fdd0eae..fa9c7bd98e9c 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -739,10 +739,9 @@ int etnaviv_gpu_init(struct etnaviv_gpu *gpu)
 	/* Setup event management */
 	spin_lock_init(&gpu->event_spinlock);
 	init_completion(&gpu->event_free);
-	for (i = 0; i < ARRAY_SIZE(gpu->event); i++) {
-		gpu->event[i].used = false;
+	bitmap_zero(gpu->event_bitmap, ETNA_NR_EVENTS);
+	for (i = 0; i < ARRAY_SIZE(gpu->event); i++)
 		complete(&gpu->event_free);
-	}
 
 	/* Now program the hardware */
 	mutex_lock(&gpu->lock);
@@ -926,7 +925,7 @@ static void recover_worker(struct work_struct *work)
 	struct etnaviv_gpu *gpu = container_of(work, struct etnaviv_gpu,
 					       recover_work);
 	unsigned long flags;
-	unsigned int i;
+	unsigned int i = 0;
 
 	dev_err(gpu->dev, "hangcheck recover!\n");
 
@@ -945,14 +944,12 @@ static void recover_worker(struct work_struct *work)
 
 	/* complete all events, the GPU won't do it after the reset */
 	spin_lock_irqsave(&gpu->event_spinlock, flags);
-	for (i = 0; i < ARRAY_SIZE(gpu->event); i++) {
-		if (!gpu->event[i].used)
-			continue;
+	for_each_set_bit_from(i, gpu->event_bitmap, ETNA_NR_EVENTS) {
 		dma_fence_signal(gpu->event[i].fence);
 		gpu->event[i].fence = NULL;
-		gpu->event[i].used = false;
 		complete(&gpu->event_free);
 	}
+	bitmap_zero(gpu->event_bitmap, ETNA_NR_EVENTS);
 	spin_unlock_irqrestore(&gpu->event_spinlock, flags);
 	gpu->completed_fence = gpu->active_fence;
 
@@ -1143,7 +1140,7 @@ int etnaviv_gpu_fence_sync_obj(struct etnaviv_gem_object *etnaviv_obj,
 static unsigned int event_alloc(struct etnaviv_gpu *gpu)
 {
 	unsigned long ret, flags;
-	unsigned int i, event = ~0U;
+	unsigned int event;
 
 	ret = wait_for_completion_timeout(&gpu->event_free,
 					  msecs_to_jiffies(10 * 10000));
@@ -1153,13 +1150,11 @@ static unsigned int event_alloc(struct etnaviv_gpu *gpu)
 	spin_lock_irqsave(&gpu->event_spinlock, flags);
 
 	/* find first free event */
-	for (i = 0; i < ARRAY_SIZE(gpu->event); i++) {
-		if (gpu->event[i].used == false) {
-			gpu->event[i].used = true;
-			event = i;
-			break;
-		}
-	}
+	event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
+	if (event < ETNA_NR_EVENTS)
+		set_bit(event, gpu->event_bitmap);
+	else
+		event = ~0U;
 
 	spin_unlock_irqrestore(&gpu->event_spinlock, flags);
 
@@ -1172,12 +1167,12 @@ static void event_free(struct etnaviv_gpu *gpu, unsigned int event)
 
 	spin_lock_irqsave(&gpu->event_spinlock, flags);
 
-	if (gpu->event[event].used == false) {
+	if (!test_bit(event, gpu->event_bitmap)) {
 		dev_warn(gpu->dev, "event %u is already marked as free",
 			 event);
 		spin_unlock_irqrestore(&gpu->event_spinlock, flags);
 	} else {
-		gpu->event[event].used = false;
+		clear_bit(event, gpu->event_bitmap);
 		spin_unlock_irqrestore(&gpu->event_spinlock, flags);
 
 		complete(&gpu->event_free);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
index 689cb8f3680c..70e6590aacdf 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
@@ -88,13 +88,14 @@ struct etnaviv_chip_identity {
 };
 
 struct etnaviv_event {
-	bool used;
 	struct dma_fence *fence;
 };
 
 struct etnaviv_cmdbuf_suballoc;
 struct etnaviv_cmdbuf;
 
+#define ETNA_NR_EVENTS 30
+
 struct etnaviv_gpu {
 	struct drm_device *drm;
 	struct thermal_cooling_device *cooling;
@@ -112,7 +113,8 @@ struct etnaviv_gpu {
 	u32 memory_base;
 
 	/* event management: */
-	struct etnaviv_event event[30];
+	DECLARE_BITMAP(event_bitmap, ETNA_NR_EVENTS);
+	struct etnaviv_event event[ETNA_NR_EVENTS];
 	struct completion event_free;
 	spinlock_t event_spinlock;
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 02/23] drm/etnaviv: make it possible to allocate multiple events

[Christian Gmeiner]

This makes it possible to allocate multiple events under the event
spinlock. This change is needed to support 'sync'-points.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 31 ++++++++++++++++++++-----------
 1 file changed, 20 insertions(+), 11 deletions(-)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index fa9c7bd98e9c..ab108b0ed573 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -1137,10 +1137,12 @@ int etnaviv_gpu_fence_sync_obj(struct etnaviv_gem_object *etnaviv_obj,
  * event management:
  */
 
-static unsigned int event_alloc(struct etnaviv_gpu *gpu)
+static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
+	unsigned int *events)
 {
 	unsigned long ret, flags;
-	unsigned int event;
+	unsigned used, i;
+	int err = 0;
 
 	ret = wait_for_completion_timeout(&gpu->event_free,
 					  msecs_to_jiffies(10 * 10000));
@@ -1149,16 +1151,24 @@ static unsigned int event_alloc(struct etnaviv_gpu *gpu)
 
 	spin_lock_irqsave(&gpu->event_spinlock, flags);
 
-	/* find first free event */
-	event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
-	if (event < ETNA_NR_EVENTS)
+	/* are there enough free events? */
+	used = bitmap_weight(gpu->event_bitmap, ETNA_NR_EVENTS);
+	if (used + nr_events > ETNA_NR_EVENTS) {
+		err = -EBUSY;
+		goto out;
+	}
+
+	for (i = 0; i < nr_events; i++) {
+		int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
+
+		events[i] = event;
 		set_bit(event, gpu->event_bitmap);
-	else
-		event = ~0U;
+	}
 
+out:
 	spin_unlock_irqrestore(&gpu->event_spinlock, flags);
 
-	return event;
+	return err;
 }
 
 static void event_free(struct etnaviv_gpu *gpu, unsigned int event)
@@ -1327,10 +1337,9 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 	 *
 	 */
 
-	event = event_alloc(gpu);
-	if (unlikely(event == ~0U)) {
+	ret = event_alloc(gpu, 1, &event);
+	if (!ret) {
 		DRM_ERROR("no free event\n");
-		ret = -EBUSY;
 		goto out_pm_put;
 	}
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 03/23] drm/etnaviv: add infrastructure to query perf counter

[Christian Gmeiner]

Make it possible that userspace can query all performance domains and
its signals. This information is needed to sample those signals via
submit ioctl.

At the moment no performance domain is available.

Changes from v1 -> v2:
 - use a 16 bit value for signals
 - fix padding issues
 - add id member to domain and signal struct

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/Makefile          |  3 +-
 drivers/gpu/drm/etnaviv/etnaviv_drv.c     | 37 ++++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 85 +++++++++++++++++++++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.h | 31 +++++++++++
 include/uapi/drm/etnaviv_drm.h            | 27 +++++++++-
 5 files changed, 181 insertions(+), 2 deletions(-)
 create mode 100644 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
 create mode 100644 drivers/gpu/drm/etnaviv/etnaviv_perfmon.h

diff --git a/drivers/gpu/drm/etnaviv/Makefile b/drivers/gpu/drm/etnaviv/Makefile
index 4f76c992043f..15c3bfa89a79 100644
--- a/drivers/gpu/drm/etnaviv/Makefile
+++ b/drivers/gpu/drm/etnaviv/Makefile
@@ -10,6 +10,7 @@ etnaviv-y := \
 	etnaviv_gpu.o \
 	etnaviv_iommu_v2.o \
 	etnaviv_iommu.o \
-	etnaviv_mmu.o
+	etnaviv_mmu.o \
+	etnaviv_perfmon.o
 
 obj-$(CONFIG_DRM_ETNAVIV)	+= etnaviv.o
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
index 91e17aeee1da..536760a9fef3 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
@@ -23,6 +23,7 @@
 #include "etnaviv_gpu.h"
 #include "etnaviv_gem.h"
 #include "etnaviv_mmu.h"
+#include "etnaviv_perfmon.h"
 
 #ifdef CONFIG_DRM_ETNAVIV_REGISTER_LOGGING
 static bool reglog;
@@ -451,6 +452,40 @@ static int etnaviv_ioctl_gem_wait(struct drm_device *dev, void *data,
 	return ret;
 }
 
+static int etnaviv_ioctl_pm_query_dom(struct drm_device *dev, void *data,
+	struct drm_file *file)
+{
+	struct etnaviv_drm_private *priv = dev->dev_private;
+	struct drm_etnaviv_pm_domain *args = data;
+	struct etnaviv_gpu *gpu;
+
+	if (args->pipe >= ETNA_MAX_PIPES)
+		return -EINVAL;
+
+	gpu = priv->gpu[args->pipe];
+	if (!gpu)
+		return -ENXIO;
+
+	return etnaviv_pm_query_dom(gpu, args);
+}
+
+static int etnaviv_ioctl_pm_query_sig(struct drm_device *dev, void *data,
+	struct drm_file *file)
+{
+	struct etnaviv_drm_private *priv = dev->dev_private;
+	struct drm_etnaviv_pm_signal *args = data;
+	struct etnaviv_gpu *gpu;
+
+	if (args->pipe >= ETNA_MAX_PIPES)
+		return -EINVAL;
+
+	gpu = priv->gpu[args->pipe];
+	if (!gpu)
+		return -ENXIO;
+
+	return etnaviv_pm_query_sig(gpu, args);
+}
+
 static const struct drm_ioctl_desc etnaviv_ioctls[] = {
 #define ETNA_IOCTL(n, func, flags) \
 	DRM_IOCTL_DEF_DRV(ETNAVIV_##n, etnaviv_ioctl_##func, flags)
@@ -463,6 +498,8 @@ static const struct drm_ioctl_desc etnaviv_ioctls[] = {
 	ETNA_IOCTL(WAIT_FENCE,   wait_fence,   DRM_AUTH|DRM_RENDER_ALLOW),
 	ETNA_IOCTL(GEM_USERPTR,  gem_userptr,  DRM_AUTH|DRM_RENDER_ALLOW),
 	ETNA_IOCTL(GEM_WAIT,     gem_wait,     DRM_AUTH|DRM_RENDER_ALLOW),
+	ETNA_IOCTL(PM_QUERY_DOM, pm_query_dom, DRM_AUTH|DRM_RENDER_ALLOW),
+	ETNA_IOCTL(PM_QUERY_SIG, pm_query_sig, DRM_AUTH|DRM_RENDER_ALLOW),
 };
 
 static const struct vm_operations_struct vm_ops = {
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
new file mode 100644
index 000000000000..5bc015194791
--- /dev/null
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -0,0 +1,85 @@
+/*
+ * Copyright (C) 2017 Etnaviv Project
+ * Copyright (C) 2017 Zodiac Inflight Innovations
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "etnaviv_gpu.h"
+
+struct etnaviv_pm_domain;
+
+struct etnaviv_pm_signal {
+	char name[64];
+	u32 data;
+
+	u32 (*sample)(struct etnaviv_gpu *gpu,
+	              const struct etnaviv_pm_domain *domain,
+	              const struct etnaviv_pm_signal *signal);
+};
+
+struct etnaviv_pm_domain {
+	char name[64];
+	u8 nr_signals;
+	const struct etnaviv_pm_signal *signal;
+};
+
+static const struct etnaviv_pm_domain doms[] = {
+};
+
+int etnaviv_pm_query_dom(struct etnaviv_gpu *gpu,
+	struct drm_etnaviv_pm_domain *domain)
+{
+	const struct etnaviv_pm_domain *dom;
+
+	if (domain->iter >= ARRAY_SIZE(doms))
+		return -EINVAL;
+
+	dom = &doms[domain->iter];
+
+	domain->id = domain->iter;
+	domain->nr_signals = dom->nr_signals;
+	strncpy(domain->name, dom->name, sizeof(domain->name));
+
+	domain->iter++;
+	if (domain->iter == ARRAY_SIZE(doms))
+		domain->iter = 0xff;
+
+	return 0;
+}
+
+int etnaviv_pm_query_sig(struct etnaviv_gpu *gpu,
+	struct drm_etnaviv_pm_signal *signal)
+{
+	const struct etnaviv_pm_domain *dom;
+	const struct etnaviv_pm_signal *sig;
+
+	if (signal->domain >= ARRAY_SIZE(doms))
+		return -EINVAL;
+
+	dom = &doms[signal->domain];
+
+	if (signal->iter > dom->nr_signals)
+		return -EINVAL;
+
+	sig = &dom->signal[signal->iter];
+
+	signal->id = signal->iter;
+	strncpy(signal->name, sig->name, sizeof(signal->name));
+
+	signal->iter++;
+	if (signal->iter == dom->nr_signals)
+		signal->iter = 0xffff;
+
+	return 0;
+}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
new file mode 100644
index 000000000000..4589cadc3a4f
--- /dev/null
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright (C) 2017 Etnaviv Project
+ * Copyright (C) 2017 Zodiac Inflight Innovations
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published by
+ * the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along with
+ * this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ETNAVIV_PERFMON_H__
+#define __ETNAVIV_PERFMON_H__
+
+struct etnaviv_gpu;
+struct drm_etnaviv_pm_domain;
+struct drm_etnaviv_pm_signal;
+
+int etnaviv_pm_query_dom(struct etnaviv_gpu *gpu,
+	struct drm_etnaviv_pm_domain *domain);
+
+int etnaviv_pm_query_sig(struct etnaviv_gpu *gpu,
+	struct drm_etnaviv_pm_signal *signal);
+
+#endif /* __ETNAVIV_PERFMON_H__ */
diff --git a/include/uapi/drm/etnaviv_drm.h b/include/uapi/drm/etnaviv_drm.h
index 76f6f78a352b..c065c8308bbd 100644
--- a/include/uapi/drm/etnaviv_drm.h
+++ b/include/uapi/drm/etnaviv_drm.h
@@ -210,6 +210,27 @@ struct drm_etnaviv_gem_wait {
 	struct drm_etnaviv_timespec timeout;	/* in */
 };
 
+/*
+ * Performance Monitor (PM):
+ */
+
+struct drm_etnaviv_pm_domain {
+	__u32 pipe;       /* in */
+	__u8  iter;       /* in/out, select pm domain at index iter */
+	__u8  id;         /* out, id of domain */
+	__u16 nr_signals; /* out, how many signals does this domain provide */
+	char  name[64];   /* out, name of domain */
+};
+
+struct drm_etnaviv_pm_signal {
+	__u32 pipe;       /* in */
+	__u8  domain;     /* in, pm domain index */
+	__u8  pad;
+	__u16 iter;       /* in/out, select pm source at index iter */
+	__u16 id;         /* out, id of signal */
+	char  name[64];   /* out, name of domain */
+};
+
 #define DRM_ETNAVIV_GET_PARAM          0x00
 /* placeholder:
 #define DRM_ETNAVIV_SET_PARAM          0x01
@@ -222,7 +243,9 @@ struct drm_etnaviv_gem_wait {
 #define DRM_ETNAVIV_WAIT_FENCE         0x07
 #define DRM_ETNAVIV_GEM_USERPTR        0x08
 #define DRM_ETNAVIV_GEM_WAIT           0x09
-#define DRM_ETNAVIV_NUM_IOCTLS         0x0a
+#define DRM_ETNAVIV_PM_QUERY_DOM       0x0a
+#define DRM_ETNAVIV_PM_QUERY_SIG       0x0b
+#define DRM_ETNAVIV_NUM_IOCTLS         0x0c
 
 #define DRM_IOCTL_ETNAVIV_GET_PARAM    DRM_IOWR(DRM_COMMAND_BASE + DRM_ETNAVIV_GET_PARAM, struct drm_etnaviv_param)
 #define DRM_IOCTL_ETNAVIV_GEM_NEW      DRM_IOWR(DRM_COMMAND_BASE + DRM_ETNAVIV_GEM_NEW, struct drm_etnaviv_gem_new)
@@ -233,6 +256,8 @@ struct drm_etnaviv_gem_wait {
 #define DRM_IOCTL_ETNAVIV_WAIT_FENCE   DRM_IOW(DRM_COMMAND_BASE + DRM_ETNAVIV_WAIT_FENCE, struct drm_etnaviv_wait_fence)
 #define DRM_IOCTL_ETNAVIV_GEM_USERPTR  DRM_IOWR(DRM_COMMAND_BASE + DRM_ETNAVIV_GEM_USERPTR, struct drm_etnaviv_gem_userptr)
 #define DRM_IOCTL_ETNAVIV_GEM_WAIT     DRM_IOW(DRM_COMMAND_BASE + DRM_ETNAVIV_GEM_WAIT, struct drm_etnaviv_gem_wait)
+#define DRM_IOCTL_ETNAVIV_PM_QUERY_DOM DRM_IOWR(DRM_COMMAND_BASE + DRM_ETNAVIV_PM_QUERY_DOM, struct drm_etnaviv_pm_domain)
+#define DRM_IOCTL_ETNAVIV_PM_QUERY_SIG DRM_IOWR(DRM_COMMAND_BASE + DRM_ETNAVIV_PM_QUERY_SIG, struct drm_etnaviv_pm_signal)
 
 #if defined(__cplusplus)
 }
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 04/23] drm/etnaviv: add uapi for perfmon feature

[Christian Gmeiner]

Sadly we can not read any registers via command stream so we need
to extend the drm_etnaviv_gem_submit struct with performance monitor
requests. Those requests gets process before or after the actual
submitted command stream.

The Vivante kernel driver has a special ioctl to read all perfmon
registers at once and return it.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 include/uapi/drm/etnaviv_drm.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/uapi/drm/etnaviv_drm.h b/include/uapi/drm/etnaviv_drm.h
index c065c8308bbd..110cc73bf549 100644
--- a/include/uapi/drm/etnaviv_drm.h
+++ b/include/uapi/drm/etnaviv_drm.h
@@ -150,6 +150,19 @@ struct drm_etnaviv_gem_submit_bo {
 	__u64 presumed;       /* in/out, presumed buffer address */
 };
 
+/* performance monitor request (pmr) */
+#define ETNA_PM_PROCESS_PRE             0x0001
+#define ETNA_PM_PROCESS_POST            0x0002
+struct drm_etnaviv_gem_submit_pmr {
+	__u32 flags;          /* in, when to process request (ETNA_PM_PROCESS_x) */
+	__u8  domain;         /* in, pm domain */
+	__u8  pad;
+	__u16 signal;         /* in, pm signal */
+	__u32 sequence;       /* in, sequence number */
+	__u32 read_offset;    /* in, offset from read_bo */
+	__u32 read_idx;       /* in, index of read_bo buffer */
+};
+
 /* Each cmdstream submit consists of a table of buffers involved, and
  * one or more cmdstream buffers.  This allows for conditional execution
  * (context-restore), and IB buffers needed for per tile/bin draw cmds.
@@ -175,6 +188,9 @@ struct drm_etnaviv_gem_submit {
 	__u64 stream;         /* in, ptr to cmdstream */
 	__u32 flags;          /* in, mask of ETNA_SUBMIT_x */
 	__s32 fence_fd;       /* in/out, fence fd (see ETNA_SUBMIT_FENCE_FD_x) */
+	__u64 pmrs;           /* in, ptr to array of submit_pmr's */
+	__u32 nr_pmrs;        /* in, number of submit_pmr's */
+	__u32 pad;
 };
 
 /* The normal way to synchronize with the GPU is just to CPU_PREP on
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 05/23] drm/etnaviv: add internal representation of perfmon_request

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h  |  4 ++++
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.h | 12 ++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h b/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h
index 80d78076c679..1b549f0d59a0 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h
@@ -21,6 +21,7 @@
 
 struct etnaviv_gpu;
 struct etnaviv_cmdbuf_suballoc;
+struct etnaviv_perfmon_request;
 
 struct etnaviv_cmdbuf {
 	/* suballocator this cmdbuf is allocated from */
@@ -38,6 +39,9 @@ struct etnaviv_cmdbuf {
 	u32 exec_state;
 	/* per GPU in-flight list */
 	struct list_head node;
+	/* perfmon requests */
+	unsigned int nr_pmrs;
+	struct etnaviv_perfmon_request *pmrs;
 	/* BOs attached to this command buffer */
 	unsigned int nr_bos;
 	struct etnaviv_vram_mapping *bo_map[0];
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
index 4589cadc3a4f..4b2b518cfd16 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
@@ -22,6 +22,18 @@ struct etnaviv_gpu;
 struct drm_etnaviv_pm_domain;
 struct drm_etnaviv_pm_signal;
 
+struct etnaviv_perfmon_request
+{
+	u32 flags;
+	u8 domain;
+	u8 signal;
+	u32 sequence;
+
+	/* bo to store a value */
+	u32 *bo_vma;
+	u32 offset;
+};
+
 int etnaviv_pm_query_dom(struct etnaviv_gpu *gpu,
 	struct drm_etnaviv_pm_domain *domain);
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 06/23] drm/etnaviv: extend etnaviv_gpu_cmdbuf_new(..) with nr_pmrs

[Christian Gmeiner]

This commits extends etnaviv_gpu_cmdbuf_new(..) to define the number
of struct etnaviv_perfmon elements gets used.

Changes from v1 -> v2:
- make use of goto as requested by Lucas

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.c     | 15 ++++++++++++++-
 drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h     |  2 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c |  2 +-
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c        |  2 +-
 4 files changed, 17 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.c b/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.c
index 633e0f07cbac..66ac79558bbd 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.c
@@ -19,6 +19,7 @@
 #include "etnaviv_cmdbuf.h"
 #include "etnaviv_gpu.h"
 #include "etnaviv_mmu.h"
+#include "etnaviv_perfmon.h"
 
 #define SUBALLOC_SIZE		SZ_256K
 #define SUBALLOC_GRANULE	SZ_4K
@@ -87,9 +88,10 @@ void etnaviv_cmdbuf_suballoc_destroy(struct etnaviv_cmdbuf_suballoc *suballoc)
 
 struct etnaviv_cmdbuf *
 etnaviv_cmdbuf_new(struct etnaviv_cmdbuf_suballoc *suballoc, u32 size,
-		   size_t nr_bos)
+		   size_t nr_bos, size_t nr_pmrs)
 {
 	struct etnaviv_cmdbuf *cmdbuf;
+	struct etnaviv_perfmon_request *pmrs;
 	size_t sz = size_vstruct(nr_bos, sizeof(cmdbuf->bo_map[0]),
 				 sizeof(*cmdbuf));
 	int granule_offs, order, ret;
@@ -98,6 +100,12 @@ etnaviv_cmdbuf_new(struct etnaviv_cmdbuf_suballoc *suballoc, u32 size,
 	if (!cmdbuf)
 		return NULL;
 
+	sz = sizeof(*pmrs) * nr_pmrs;
+	pmrs = kzalloc(sz, GFP_KERNEL);
+	if (!pmrs)
+		goto out_free_cmdbuf;
+
+	cmdbuf->pmrs = pmrs;
 	cmdbuf->suballoc = suballoc;
 	cmdbuf->size = size;
 
@@ -124,6 +132,10 @@ etnaviv_cmdbuf_new(struct etnaviv_cmdbuf_suballoc *suballoc, u32 size,
 	cmdbuf->vaddr = suballoc->vaddr + cmdbuf->suballoc_offset;
 
 	return cmdbuf;
+
+out_free_cmdbuf:
+	kfree(cmdbuf);
+	return NULL;
 }
 
 void etnaviv_cmdbuf_free(struct etnaviv_cmdbuf *cmdbuf)
@@ -139,6 +151,7 @@ void etnaviv_cmdbuf_free(struct etnaviv_cmdbuf *cmdbuf)
 	suballoc->free_space = 1;
 	mutex_unlock(&suballoc->lock);
 	wake_up_all(&suballoc->free_event);
+	kfree(cmdbuf->pmrs);
 	kfree(cmdbuf);
 }
 
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h b/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h
index 1b549f0d59a0..b6348b9f2a9d 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_cmdbuf.h
@@ -53,7 +53,7 @@ void etnaviv_cmdbuf_suballoc_destroy(struct etnaviv_cmdbuf_suballoc *suballoc);
 
 struct etnaviv_cmdbuf *
 etnaviv_cmdbuf_new(struct etnaviv_cmdbuf_suballoc *suballoc, u32 size,
-		   size_t nr_bos);
+		   size_t nr_bos, size_t nr_pmrs);
 void etnaviv_cmdbuf_free(struct etnaviv_cmdbuf *cmdbuf);
 
 u32 etnaviv_cmdbuf_get_va(struct etnaviv_cmdbuf *buf);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index ee7069e93eda..9c57b14dfcbf 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -350,7 +350,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
 	stream = kvmalloc_array(1, args->stream_size, GFP_KERNEL);
 	cmdbuf = etnaviv_cmdbuf_new(gpu->cmdbuf_suballoc,
 				    ALIGN(args->stream_size, 8) + 8,
-				    args->nr_bos);
+				    args->nr_bos, 0);
 	if (!bos || !relocs || !stream || !cmdbuf) {
 		ret = -ENOMEM;
 		goto err_submit_cmds;
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index ab108b0ed573..ce6c869e214f 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -721,7 +721,7 @@ int etnaviv_gpu_init(struct etnaviv_gpu *gpu)
 	}
 
 	/* Create buffer: */
-	gpu->buffer = etnaviv_cmdbuf_new(gpu->cmdbuf_suballoc, PAGE_SIZE, 0);
+	gpu->buffer = etnaviv_cmdbuf_new(gpu->cmdbuf_suballoc, PAGE_SIZE, 0, 0);
 	if (!gpu->buffer) {
 		ret = -ENOMEM;
 		dev_err(gpu->dev, "could not create command buffer\n");
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 07/23] drm/etnaviv: add performance monitor request validation

[Christian Gmeiner]

Check if the selected domain and signal combination exists.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 15 +++++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.h |  2 ++
 2 files changed, 17 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 5bc015194791..3ecedeed87bd 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -83,3 +83,18 @@ int etnaviv_pm_query_sig(struct etnaviv_gpu *gpu,
 
 	return 0;
 }
+
+int etnaviv_pm_req_validate(const struct drm_etnaviv_gem_submit_pmr *r)
+{
+	const struct etnaviv_pm_domain *dom;
+
+	if (r->domain >= ARRAY_SIZE(doms))
+		return -EINVAL;
+
+	dom = &doms[r->domain];
+
+	if (r->signal > dom->nr_signals)
+		return -EINVAL;
+
+	return 0;
+}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
index 4b2b518cfd16..f20b69c942fc 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
@@ -40,4 +40,6 @@ int etnaviv_pm_query_dom(struct etnaviv_gpu *gpu,
 int etnaviv_pm_query_sig(struct etnaviv_gpu *gpu,
 	struct drm_etnaviv_pm_signal *signal);
 
+int etnaviv_pm_req_validate(const struct drm_etnaviv_gem_submit_pmr *r);
+
 #endif /* __ETNAVIV_PERFMON_H__ */
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 08/23] drm/etnaviv: copy pmrs from userspace

[Christian Gmeiner]

Changes from v1 -> v2:
- renamed submit_perfmon_request() to submit_perfmon_validate()
- extended flags validation
- added comment about offset 0
- moved assigment of cmdbuf->nr_pmrs below the copy_from_user of the pmrs.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 69 +++++++++++++++++++++++++++-
 1 file changed, 67 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
index 9c57b14dfcbf..91e35114d25c 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
@@ -21,6 +21,7 @@
 #include "etnaviv_drv.h"
 #include "etnaviv_gpu.h"
 #include "etnaviv_gem.h"
+#include "etnaviv_perfmon.h"
 
 /*
  * Cmdstream submission:
@@ -283,6 +284,54 @@ static int submit_reloc(struct etnaviv_gem_submit *submit, void *stream,
 	return 0;
 }
 
+static int submit_perfmon_validate(struct etnaviv_gem_submit *submit,
+		struct etnaviv_cmdbuf *cmdbuf,
+		const struct drm_etnaviv_gem_submit_pmr *pmrs,
+		u32 nr_pms)
+{
+	u32 i;
+
+	for (i = 0; i < nr_pms; i++) {
+		const struct drm_etnaviv_gem_submit_pmr *r = pmrs + i;
+		struct etnaviv_gem_submit_bo *bo;
+		int ret;
+
+		ret = submit_bo(submit, r->read_idx, &bo);
+		if (ret)
+			return ret;
+
+		/* at offset 0 a sequence number gets stored used for userspace sync */
+		if (r->read_offset == 0) {
+			DRM_ERROR("perfmon request: offset is 0");
+			return -EINVAL;
+		}
+
+		if (r->read_offset >= bo->obj->base.size - sizeof(u32)) {
+			DRM_ERROR("perfmon request: offset %u outside object", i);
+			return -EINVAL;
+		}
+
+		if (!(r->flags & (ETNA_PM_PROCESS_PRE | ETNA_PM_PROCESS_POST))) {
+			DRM_ERROR("perfmon request: flags are not valid");
+			return -EINVAL;
+		}
+
+		if (etnaviv_pm_req_validate(r)) {
+			DRM_ERROR("perfmon request: domain or signal not valid");
+			return -EINVAL;
+		}
+
+		cmdbuf->pmrs[i].flags = r->flags;
+		cmdbuf->pmrs[i].domain = r->domain;
+		cmdbuf->pmrs[i].signal = r->signal;
+		cmdbuf->pmrs[i].sequence = r->sequence;
+		cmdbuf->pmrs[i].offset = r->read_offset;
+		cmdbuf->pmrs[i].bo_vma = etnaviv_gem_vmap(&bo->obj->base);
+	}
+
+	return 0;
+}
+
 static void submit_cleanup(struct etnaviv_gem_submit *submit)
 {
 	unsigned i;
@@ -306,6 +355,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
 	struct etnaviv_drm_private *priv = dev->dev_private;
 	struct drm_etnaviv_gem_submit *args = data;
 	struct drm_etnaviv_gem_submit_reloc *relocs;
+	struct drm_etnaviv_gem_submit_pmr *pmrs;
 	struct drm_etnaviv_gem_submit_bo *bos;
 	struct etnaviv_gem_submit *submit;
 	struct etnaviv_cmdbuf *cmdbuf;
@@ -347,11 +397,12 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
 	 */
 	bos = kvmalloc_array(args->nr_bos, sizeof(*bos), GFP_KERNEL);
 	relocs = kvmalloc_array(args->nr_relocs, sizeof(*relocs), GFP_KERNEL);
+	pmrs = kvmalloc_array(args->nr_pmrs, sizeof(*pmrs), GFP_KERNEL);
 	stream = kvmalloc_array(1, args->stream_size, GFP_KERNEL);
 	cmdbuf = etnaviv_cmdbuf_new(gpu->cmdbuf_suballoc,
 				    ALIGN(args->stream_size, 8) + 8,
-				    args->nr_bos, 0);
-	if (!bos || !relocs || !stream || !cmdbuf) {
+				    args->nr_bos, args->nr_pmrs);
+	if (!bos || !relocs || !pmrs || !stream || !cmdbuf) {
 		ret = -ENOMEM;
 		goto err_submit_cmds;
 	}
@@ -373,6 +424,14 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
 		goto err_submit_cmds;
 	}
 
+	ret = copy_from_user(pmrs, u64_to_user_ptr(args->pmrs),
+			     args->nr_pmrs * sizeof(*pmrs));
+	if (ret) {
+		ret = -EFAULT;
+		goto err_submit_cmds;
+	}
+	cmdbuf->nr_pmrs = args->nr_pmrs;
+
 	ret = copy_from_user(stream, u64_to_user_ptr(args->stream),
 			     args->stream_size);
 	if (ret) {
@@ -441,6 +500,10 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
 	if (ret)
 		goto out;
 
+	ret = submit_perfmon_validate(submit, cmdbuf, pmrs, args->nr_pmrs);
+	if (ret)
+		goto out;
+
 	memcpy(cmdbuf->vaddr, stream, args->stream_size);
 	cmdbuf->user_size = ALIGN(args->stream_size, 8);
 
@@ -494,6 +557,8 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
 		kvfree(bos);
 	if (relocs)
 		kvfree(relocs);
+	if (pmrs)
+		kvfree(pmrs);
 
 	return ret;
 }
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 09/23] drm/etnaviv: add performance monitor request processing

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 16 ++++++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.h |  3 +++
 2 files changed, 19 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 3ecedeed87bd..41e12f2a1dbd 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -16,6 +16,7 @@
  */
 
 #include "etnaviv_gpu.h"
+#include "etnaviv_perfmon.h"
 
 struct etnaviv_pm_domain;
 
@@ -98,3 +99,18 @@ int etnaviv_pm_req_validate(const struct drm_etnaviv_gem_submit_pmr *r)
 
 	return 0;
 }
+
+void etnaviv_perfmon_process(struct etnaviv_gpu *gpu,
+	const struct etnaviv_perfmon_request *pmr)
+{
+	const struct etnaviv_pm_domain *dom;
+	const struct etnaviv_pm_signal *sig;
+	u32 *bo = pmr->bo_vma;
+	u32 val;
+
+	dom = &doms[pmr->domain];
+	sig = &dom->signal[pmr->signal];
+	val = sig->sample(gpu, dom, sig);
+
+	*(bo + pmr->offset) = val;
+}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
index f20b69c942fc..f9c8d7ec8a9b 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.h
@@ -42,4 +42,7 @@ int etnaviv_pm_query_sig(struct etnaviv_gpu *gpu,
 
 int etnaviv_pm_req_validate(const struct drm_etnaviv_gem_submit_pmr *r);
 
+void etnaviv_perfmon_process(struct etnaviv_gpu *gpu,
+	const struct etnaviv_perfmon_request *pmr);
+
 #endif /* __ETNAVIV_PERFMON_H__ */
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 10/23] drm/etnaviv: add 'sync point' support

[Christian Gmeiner]

In order to support performance counters in a sane way we need to provide
a method to sync the GPU with the CPU. The GPU can process multpile command
buffers/events per irq. With the help of a 'sync point' we can trigger an event
and stop the GPU/FE immediately. When the CPU is done with is processing it
simply needs to restart the FE and the GPU will process the command stream.

Changes from v1 -> v2:
- process sync point with a work item to keep irq as fast as possible

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_buffer.c | 36 ++++++++++++++++++++++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_drv.h    |  1 +
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c    | 25 ++++++++++++++++++++++
 drivers/gpu/drm/etnaviv/etnaviv_gpu.h    |  6 ++++++
 4 files changed, 68 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
index ed9588f36bc9..9e7098e3207f 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
@@ -250,6 +250,42 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
 	}
 }
 
+/* Append a 'sync point' to the ring buffer. */
+void etnaviv_sync_point_queue(struct etnaviv_gpu *gpu, unsigned int event)
+{
+	struct etnaviv_cmdbuf *buffer = gpu->buffer;
+	unsigned int waitlink_offset = buffer->user_size - 16;
+	u32 dwords, target;
+
+	/*
+	 * We need at most 3 dwords in the return target:
+	 * 1 event + 1 end + 1 wait + 1 link.
+	 */
+	dwords = 4;
+	target = etnaviv_buffer_reserve(gpu, buffer, dwords);
+
+	/* Signal sync point event */
+	CMD_LOAD_STATE(buffer, VIVS_GL_EVENT, VIVS_GL_EVENT_EVENT_ID(event) |
+		       VIVS_GL_EVENT_FROM_PE);
+
+	/* Stop the FE to 'pause' the GPU */
+	CMD_END(buffer);
+
+	/* Append waitlink */
+	CMD_WAIT(buffer);
+	CMD_LINK(buffer, 2, etnaviv_cmdbuf_get_va(buffer) +
+			    buffer->user_size - 4);
+
+	/*
+	 * Kick off the 'sync point' command by replacing the previous
+	 * WAIT with a link to the address in the ring buffer.
+	 */
+	etnaviv_buffer_replace_wait(buffer, waitlink_offset,
+				    VIV_FE_LINK_HEADER_OP_LINK |
+				    VIV_FE_LINK_HEADER_PREFETCH(dwords),
+				    target);
+}
+
 /* Append a command buffer to the ring buffer. */
 void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, unsigned int event,
 	struct etnaviv_cmdbuf *cmdbuf)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.h b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
index 058389f93b69..f6cdd694ca51 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
@@ -101,6 +101,7 @@ int etnaviv_gem_new_userptr(struct drm_device *dev, struct drm_file *file,
 u16 etnaviv_buffer_init(struct etnaviv_gpu *gpu);
 u16 etnaviv_buffer_config_mmuv2(struct etnaviv_gpu *gpu, u32 mtlb_addr, u32 safe_addr);
 void etnaviv_buffer_end(struct etnaviv_gpu *gpu);
+void etnaviv_sync_point_queue(struct etnaviv_gpu *gpu, unsigned int event);
 void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, unsigned int event,
 	struct etnaviv_cmdbuf *cmdbuf);
 void etnaviv_validate_init(void);
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index ce6c869e214f..bc1b96b4c7e9 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -25,6 +25,7 @@
 #include "etnaviv_gpu.h"
 #include "etnaviv_gem.h"
 #include "etnaviv_mmu.h"
+#include "etnaviv_perfmon.h"
 #include "common.xml.h"
 #include "state.xml.h"
 #include "state_hi.xml.h"
@@ -1353,6 +1354,7 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 	}
 
 	gpu->event[event].fence = fence;
+	gpu->event[event].sync_point = NULL;
 	submit->fence = dma_fence_get(fence);
 	gpu->active_fence = submit->fence->seqno;
 
@@ -1398,6 +1400,23 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 	return ret;
 }
 
+static void etnaviv_process_sync_point(struct etnaviv_gpu *gpu,
+	struct etnaviv_event *event)
+{
+	u32 addr = gpu_read(gpu, VIVS_FE_DMA_ADDRESS);
+
+	event->sync_point(gpu, event);
+	etnaviv_gpu_start_fe(gpu, addr + 2, 2);
+}
+
+static void pmrs_worker(struct work_struct *work)
+{
+	struct etnaviv_gpu *gpu = container_of(work, struct etnaviv_gpu,
+					       pmrs_work);
+
+	etnaviv_process_sync_point(gpu, &gpu->event[gpu->pmrs_event]);
+}
+
 /*
  * Init/Cleanup:
  */
@@ -1444,6 +1463,11 @@ static irqreturn_t irq_handler(int irq, void *data)
 
 			dev_dbg(gpu->dev, "event %u\n", event);
 
+			if (gpu->event[event].sync_point) {
+				gpu->pmrs_event = event;
+				etnaviv_queue_work(gpu->drm, &gpu->pmrs_work);
+			}
+
 			fence = gpu->event[event].fence;
 			gpu->event[event].fence = NULL;
 			dma_fence_signal(fence);
@@ -1647,6 +1671,7 @@ static int etnaviv_gpu_bind(struct device *dev, struct device *master,
 
 	INIT_LIST_HEAD(&gpu->active_cmd_list);
 	INIT_WORK(&gpu->retire_work, retire_worker);
+	INIT_WORK(&gpu->pmrs_work, pmrs_worker);
 	INIT_WORK(&gpu->recover_work, recover_worker);
 	init_waitqueue_head(&gpu->fence_event);
 
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
index 70e6590aacdf..7d5f785b0f08 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
@@ -89,6 +89,8 @@ struct etnaviv_chip_identity {
 
 struct etnaviv_event {
 	struct dma_fence *fence;
+
+	void (*sync_point)(struct etnaviv_gpu *gpu, struct etnaviv_event *event);
 };
 
 struct etnaviv_cmdbuf_suballoc;
@@ -135,6 +137,10 @@ struct etnaviv_gpu {
 	/* worker for handling active-list retiring: */
 	struct work_struct retire_work;
 
+	/* worker for handling performance monitor requests: */
+	struct work_struct pmrs_work;
+	int pmrs_event;
+
 	void __iomem *mmio;
 	int irq;
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 11/23] drm/etnaviv: clear alloced event

[Christian Gmeiner]

Results in less code as the users do not set every struct member to 0/NULL.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index bc1b96b4c7e9..93e3f14c0599 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -1163,6 +1163,7 @@ static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
 		int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
 
 		events[i] = event;
+		memset(&gpu->event[event], 0, sizeof(struct etnaviv_event));
 		set_bit(event, gpu->event_bitmap);
 	}
 
@@ -1354,7 +1355,6 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 	}
 
 	gpu->event[event].fence = fence;
-	gpu->event[event].sync_point = NULL;
 	submit->fence = dma_fence_get(fence);
 	gpu->active_fence = submit->fence->seqno;
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 12/23] drm/etnaviv: use 'sync points' for performance monitor requests

[Christian Gmeiner]

With 'sync points' we can sample the reqeustes perform signals
before and/or after the submited command buffer.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 106 +++++++++++++++++++++++++++-------
 drivers/gpu/drm/etnaviv/etnaviv_gpu.h |   1 +
 2 files changed, 86 insertions(+), 21 deletions(-)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index 93e3f14c0599..c176781788ac 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -1318,12 +1318,48 @@ void etnaviv_gpu_pm_put(struct etnaviv_gpu *gpu)
 	pm_runtime_put_autosuspend(gpu->dev);
 }
 
+static void sync_point_perfmon_sample(struct etnaviv_gpu *gpu,
+	struct etnaviv_event *event, unsigned int flags)
+{
+	const struct etnaviv_cmdbuf *cmdbuf = event->cmdbuf;
+	unsigned int i;
+
+	for (i = 0; i < cmdbuf->nr_pmrs; i++) {
+		const struct etnaviv_perfmon_request *pmr = cmdbuf->pmrs + i;
+
+		if (pmr->flags == flags)
+			etnaviv_perfmon_process(gpu, pmr);
+	}
+}
+
+static void sync_point_perfmon_sample_pre(struct etnaviv_gpu *gpu,
+	struct etnaviv_event *event)
+{
+	sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_PRE);
+}
+
+static void sync_point_perfmon_sample_post(struct etnaviv_gpu *gpu,
+	struct etnaviv_event *event)
+{
+	const struct etnaviv_cmdbuf *cmdbuf = event->cmdbuf;
+	unsigned int i;
+
+	sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_POST);
+
+	for (i = 0; i < cmdbuf->nr_pmrs; i++) {
+		const struct etnaviv_perfmon_request *pmr = cmdbuf->pmrs + i;
+
+		*pmr->bo_vma = pmr->sequence;
+	}
+}
+
+
 /* add bo's to gpu's ring, and kick gpu: */
 int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 	struct etnaviv_gem_submit *submit, struct etnaviv_cmdbuf *cmdbuf)
 {
 	struct dma_fence *fence;
-	unsigned int event, i;
+	unsigned int i, nr_events, event[3];
 	int ret;
 
 	ret = etnaviv_gpu_pm_get_sync(gpu);
@@ -1339,9 +1375,21 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 	 *
 	 */
 
-	ret = event_alloc(gpu, 1, &event);
-	if (!ret) {
-		DRM_ERROR("no free event\n");
+	 /*
+ 	 * if there are performance monitor requests we need to have
+	 * - a sync point to re-configure gpu and process ETNA_PM_PROCESS_PRE
+	 *   requests.
+	 * - a sync point to re-configure gpu, process ETNA_PM_PROCESS_POST requests
+	 *   and update the sequence number for userspace.
+ 	 */
+	 if (cmdbuf->nr_pmrs)
+	 	nr_events = 3;
+	else
+		nr_events = 1;
+
+	ret = event_alloc(gpu, nr_events, event);
+	if (ret < 0) {
+		DRM_ERROR("no free events\n");
 		goto out_pm_put;
 	}
 
@@ -1349,12 +1397,14 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 
 	fence = etnaviv_gpu_fence_alloc(gpu);
 	if (!fence) {
-		event_free(gpu, event);
+		for (i = 0; i < nr_events; i++)
+			event_free(gpu, event[i]);
+
 		ret = -ENOMEM;
 		goto out_unlock;
 	}
 
-	gpu->event[event].fence = fence;
+	gpu->event[event[0]].fence = fence;
 	submit->fence = dma_fence_get(fence);
 	gpu->active_fence = submit->fence->seqno;
 
@@ -1364,7 +1414,19 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
 		gpu->lastctx = cmdbuf->ctx;
 	}
 
-	etnaviv_buffer_queue(gpu, event, cmdbuf);
+	if (cmdbuf->nr_pmrs) {
+		gpu->event[event[1]].sync_point = &sync_point_perfmon_sample_pre;
+		gpu->event[event[1]].cmdbuf = cmdbuf;
+		etnaviv_sync_point_queue(gpu, event[1]);
+	}
+
+	etnaviv_buffer_queue(gpu, event[0], cmdbuf);
+
+	if (cmdbuf->nr_pmrs) {
+		gpu->event[event[2]].sync_point = &sync_point_perfmon_sample_post;
+		gpu->event[event[2]].cmdbuf = cmdbuf;
+		etnaviv_sync_point_queue(gpu, event[2]);
+	}
 
 	cmdbuf->fence = fence;
 	list_add_tail(&cmdbuf->node, &gpu->active_cmd_list);
@@ -1469,20 +1531,22 @@ static irqreturn_t irq_handler(int irq, void *data)
 			}
 
 			fence = gpu->event[event].fence;
-			gpu->event[event].fence = NULL;
-			dma_fence_signal(fence);
-
-			/*
-			 * Events can be processed out of order.  Eg,
-			 * - allocate and queue event 0
-			 * - allocate event 1
-			 * - event 0 completes, we process it
-			 * - allocate and queue event 0
-			 * - event 1 and event 0 complete
-			 * we can end up processing event 0 first, then 1.
-			 */
-			if (fence_after(fence->seqno, gpu->completed_fence))
-				gpu->completed_fence = fence->seqno;
+			if (fence) {
+				gpu->event[event].fence = NULL;
+				dma_fence_signal(fence);
+
+				/*
+				 * Events can be processed out of order.  Eg,
+				 * - allocate and queue event 0
+				 * - allocate event 1
+				 * - event 0 completes, we process it
+				 * - allocate and queue event 0
+				 * - event 1 and event 0 complete
+				 * we can end up processing event 0 first, then 1.
+				 */
+				if (fence_after(fence->seqno, gpu->completed_fence))
+					gpu->completed_fence = fence->seqno;
+			}
 
 			event_free(gpu, event);
 		}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
index 7d5f785b0f08..c75a7b5c397c 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
@@ -89,6 +89,7 @@ struct etnaviv_chip_identity {
 
 struct etnaviv_event {
 	struct dma_fence *fence;
+	struct etnaviv_cmdbuf *cmdbuf;
 
 	void (*sync_point)(struct etnaviv_gpu *gpu, struct etnaviv_event *event);
 };
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 13/23] drm/etnaviv: add HI perf domain

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 55 +++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 41e12f2a1dbd..212e1cee61fa 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -17,6 +17,7 @@
 
 #include "etnaviv_gpu.h"
 #include "etnaviv_perfmon.h"
+#include "state_hi.xml.h"
 
 struct etnaviv_pm_domain;
 
@@ -31,11 +32,65 @@ struct etnaviv_pm_signal {
 
 struct etnaviv_pm_domain {
 	char name[64];
+
+	/* profile register */
+	u32 profile_read;
+	u32 profile_config;
+
 	u8 nr_signals;
 	const struct etnaviv_pm_signal *signal;
 };
 
+static u32 simple_reg_read(struct etnaviv_gpu *gpu,
+	const struct etnaviv_pm_domain *domain,
+	const struct etnaviv_pm_signal *signal)
+{
+	return gpu_read(gpu, signal->data);
+}
+
+static u32 perf_reg_read(struct etnaviv_gpu *gpu,
+	const struct etnaviv_pm_domain *domain,
+	const struct etnaviv_pm_signal *signal)
+{
+	gpu_write(gpu, domain->profile_config, signal->data);
+
+	return gpu_read(gpu, domain->profile_read);
+}
+
 static const struct etnaviv_pm_domain doms[] = {
+	{
+		.name = "HI",
+		.profile_read = VIVS_MC_PROFILE_HI_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG2,
+		.nr_signals = 5,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"TOTAL_CYCLES",
+				VIVS_HI_PROFILE_TOTAL_CYCLES,
+				&simple_reg_read
+			},
+			{
+				"IDLE_CYCLES",
+				VIVS_HI_PROFILE_IDLE_CYCLES,
+				&simple_reg_read
+			},
+			{
+				"AXI_CYCLES_READ_REQUEST_STALLED",
+				VIVS_MC_PROFILE_CONFIG2_HI_AXI_CYCLES_READ_REQUEST_STALLED,
+				&perf_reg_read
+			},
+			{
+				"AXI_CYCLES_WRITE_REQUEST_STALLED",
+				VIVS_MC_PROFILE_CONFIG2_HI_AXI_CYCLES_WRITE_REQUEST_STALLED,
+				&perf_reg_read
+			},
+			{
+				"AXI_CYCLES_WRITE_DATA_STALLED",
+				VIVS_MC_PROFILE_CONFIG2_HI_AXI_CYCLES_WRITE_DATA_STALLED,
+				&perf_reg_read
+			}
+		}
+	}
 };
 
 int etnaviv_pm_query_dom(struct etnaviv_gpu *gpu,
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 14/23] drm/etnaviv: add PE perf domain

[Christian Gmeiner]

We need to iterate over all pixel pipelines to get overall value.

Changes from v1 -> v2:
 - switch back to pixel pipe 0 to prevent GPU hang

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 57 +++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 212e1cee61fa..5d7ff5c25268 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -57,6 +57,30 @@ static u32 perf_reg_read(struct etnaviv_gpu *gpu,
 	return gpu_read(gpu, domain->profile_read);
 }
 
+static u32 pipe_reg_read(struct etnaviv_gpu *gpu,
+	const struct etnaviv_pm_domain *domain,
+	const struct etnaviv_pm_signal *signal)
+{
+	u32 clock = gpu_read(gpu, VIVS_HI_CLOCK_CONTROL);
+	u32 value = 0;
+	unsigned i;
+
+	for (i = 0; i < gpu->identity.pixel_pipes; i++) {
+		clock &= ~(VIVS_HI_CLOCK_CONTROL_DEBUG_PIXEL_PIPE__MASK);
+		clock |= VIVS_HI_CLOCK_CONTROL_DEBUG_PIXEL_PIPE(i);
+		gpu_write(gpu, VIVS_HI_CLOCK_CONTROL, clock);
+		gpu_write(gpu, domain->profile_config, signal->data);
+		value += gpu_read(gpu, domain->profile_read);
+	}
+
+	/* switch back to pixel pipe 0 to prevent GPU hang */
+	clock &= ~(VIVS_HI_CLOCK_CONTROL_DEBUG_PIXEL_PIPE__MASK);
+	clock |= VIVS_HI_CLOCK_CONTROL_DEBUG_PIXEL_PIPE(0);
+	gpu_write(gpu, VIVS_HI_CLOCK_CONTROL, clock);
+
+	return value;
+}
+
 static const struct etnaviv_pm_domain doms[] = {
 	{
 		.name = "HI",
@@ -90,6 +114,39 @@ static const struct etnaviv_pm_domain doms[] = {
 				&perf_reg_read
 			}
 		}
+	},
+	{
+		.name = "PE",
+		.profile_read = VIVS_MC_PROFILE_PE_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG0,
+		.nr_signals = 5,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"PIXEL_COUNT_KILLED_BY_COLOR_PIPE",
+				VIVS_MC_PROFILE_CONFIG0_PE_PIXEL_COUNT_KILLED_BY_COLOR_PIPE,
+				&pipe_reg_read
+			},
+			{
+				"PIXEL_COUNT_KILLED_BY_DEPTH_PIPE",
+				VIVS_MC_PROFILE_CONFIG0_PE_PIXEL_COUNT_KILLED_BY_DEPTH_PIPE,
+				&pipe_reg_read
+			},
+			{
+				"PIXEL_COUNT_DRAWN_BY_COLOR_PIPE",
+				VIVS_MC_PROFILE_CONFIG0_PE_PIXEL_COUNT_DRAWN_BY_COLOR_PIPE,
+				&pipe_reg_read
+			},
+			{
+				"PIXEL_COUNT_DRAWN_BY_DEPTH_PIPE",
+				VIVS_MC_PROFILE_CONFIG0_PE_PIXEL_COUNT_DRAWN_BY_DEPTH_PIPE,
+				&pipe_reg_read
+			},
+			{
+				"PIXELS_RENDERED_2D",
+				VIVS_MC_PROFILE_CONFIG0_PE_PIXELS_RENDERED_2D,
+				&pipe_reg_read
+			}
+		}
 	}
 };
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 15/23] drm/etnaviv: add SH perf domain

[Christian Gmeiner]

Changes from v1 -> v2:
 - all SH perf counters are accessed via perf_reg_read(..)

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 53 +++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 5d7ff5c25268..2b2470761920 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -147,6 +147,59 @@ static const struct etnaviv_pm_domain doms[] = {
 				&pipe_reg_read
 			}
 		}
+	},
+	{
+		.name = "SH",
+		.profile_read = VIVS_MC_PROFILE_SH_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG0,
+		.nr_signals = 9,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"SHADER_CYCLES",
+				VIVS_MC_PROFILE_CONFIG0_SH_SHADER_CYCLES,
+				&perf_reg_read
+			},
+			{
+				"PS_INST_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_PS_INST_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"RENDERED_PIXEL_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_RENDERED_PIXEL_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"VS_INST_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_VS_INST_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"RENDERED_VERTICE_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_RENDERED_VERTICE_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"VTX_BRANCH_INST_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_VTX_BRANCH_INST_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"VTX_TEXLD_INST_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_VTX_TEXLD_INST_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"PXL_BRANCH_INST_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_PXL_BRANCH_INST_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"PXL_TEXLD_INST_COUNTER",
+				VIVS_MC_PROFILE_CONFIG0_SH_PXL_TEXLD_INST_COUNTER,
+				&perf_reg_read
+			}
+		}
 	}
 };
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 16/23] drm/etnaviv: add PA perf domain

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 38 +++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 2b2470761920..e712fa7aa758 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -200,6 +200,44 @@ static const struct etnaviv_pm_domain doms[] = {
 				&perf_reg_read
 			}
 		}
+	},
+	{
+		.name = "PA",
+		.profile_read = VIVS_MC_PROFILE_PA_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG1,
+		.nr_signals = 6,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"INPUT_VTX_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_PA_INPUT_VTX_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"INPUT_PRIM_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_PA_INPUT_PRIM_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"OUTPUT_PRIM_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_PA_OUTPUT_PRIM_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"DEPTH_CLIPPED_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_PA_DEPTH_CLIPPED_COUNTER,
+				&pipe_reg_read
+			},
+			{
+				"TRIVIAL_REJECTED_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_PA_TRIVIAL_REJECTED_COUNTER,
+				&pipe_reg_read
+			},
+			{
+				"CULLED_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_PA_CULLED_COUNTER,
+				&pipe_reg_read
+			}
+		}
 	}
 };
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 17/23] drm/etnaviv: add SE perf domain

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index e712fa7aa758..881762cad07e 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -238,6 +238,24 @@ static const struct etnaviv_pm_domain doms[] = {
 				&pipe_reg_read
 			}
 		}
+	},
+	{
+		.name = "SE",
+		.profile_read = VIVS_MC_PROFILE_SE_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG1,
+		.nr_signals = 2,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"CULLED_TRIANGLE_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_SE_CULLED_TRIANGLE_COUNT,
+				&perf_reg_read
+			},
+			{
+				"CULLED_LINES_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_SE_CULLED_LINES_COUNT,
+				&perf_reg_read
+			}
+		}
 	}
 };
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 18/23] drm/etnaviv: add RA perf domain

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 43 +++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 881762cad07e..726d7627de28 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -256,6 +256,49 @@ static const struct etnaviv_pm_domain doms[] = {
 				&perf_reg_read
 			}
 		}
+	},
+	{
+		.name = "RA",
+		.profile_read = VIVS_MC_PROFILE_RA_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG1,
+		.nr_signals = 7,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"VALID_PIXEL_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_RA_VALID_PIXEL_COUNT,
+				&perf_reg_read
+			},
+			{
+				"TOTAL_QUAD_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_RA_TOTAL_QUAD_COUNT,
+				&perf_reg_read
+			},
+			{
+				"VALID_QUAD_COUNT_AFTER_EARLY_Z",
+				VIVS_MC_PROFILE_CONFIG1_RA_VALID_QUAD_COUNT_AFTER_EARLY_Z,
+				&perf_reg_read
+			},
+			{
+				"TOTAL_PRIMITIVE_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_RA_TOTAL_PRIMITIVE_COUNT,
+				&perf_reg_read
+			},
+			{
+				"PIPE_CACHE_MISS_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_RA_PIPE_CACHE_MISS_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"PREFETCH_CACHE_MISS_COUNTER",
+				VIVS_MC_PROFILE_CONFIG1_RA_PREFETCH_CACHE_MISS_COUNTER,
+				&perf_reg_read
+			},
+			{
+				"CULLED_QUAD_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_RA_CULLED_QUAD_COUNT,
+				&perf_reg_read
+			}
+		}
 	}
 };
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 19/23] drm/etnaviv: add TX perf domain

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 53 +++++++++++++++++++++++++++++++
 1 file changed, 53 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 726d7627de28..6df78a1c4830 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -299,6 +299,59 @@ static const struct etnaviv_pm_domain doms[] = {
 				&perf_reg_read
 			}
 		}
+	},
+	{
+		.name = "TX",
+		.profile_read = VIVS_MC_PROFILE_TX_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG1,
+		.nr_signals = 9,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"TOTAL_BILINEAR_REQUESTS",
+				VIVS_MC_PROFILE_CONFIG1_TX_TOTAL_BILINEAR_REQUESTS,
+				&perf_reg_read
+			},
+			{
+				"TOTAL_TRILINEAR_REQUESTS",
+				VIVS_MC_PROFILE_CONFIG1_TX_TOTAL_TRILINEAR_REQUESTS,
+				&perf_reg_read
+			},
+			{
+				"TOTAL_DISCARDED_TEXTURE_REQUESTS",
+				VIVS_MC_PROFILE_CONFIG1_TX_TOTAL_DISCARDED_TEXTURE_REQUESTS,
+				&perf_reg_read
+			},
+			{
+				"TOTAL_TEXTURE_REQUESTS",
+				VIVS_MC_PROFILE_CONFIG1_TX_TOTAL_TEXTURE_REQUESTS,
+				&perf_reg_read
+			},
+			{
+				"MEM_READ_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_TX_MEM_READ_COUNT,
+				&perf_reg_read
+			},
+			{
+				"MEM_READ_IN_8B_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_TX_MEM_READ_IN_8B_COUNT,
+				&perf_reg_read
+			},
+			{
+				"CACHE_MISS_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_TX_CACHE_MISS_COUNT,
+				&perf_reg_read
+			},
+			{
+				"CACHE_HIT_TEXEL_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_TX_CACHE_HIT_TEXEL_COUNT,
+				&perf_reg_read
+			},
+			{
+				"CACHE_MISS_TEXEL_COUNT",
+				VIVS_MC_PROFILE_CONFIG1_TX_CACHE_MISS_TEXEL_COUNT,
+				&perf_reg_read
+			}
+		}
 	}
 };
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 20/23] drm/etnaviv: add MC perf domain

[Christian Gmeiner]

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
---
 drivers/gpu/drm/etnaviv/etnaviv_perfmon.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
index 6df78a1c4830..8dd99b6787c9 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_perfmon.c
@@ -352,6 +352,29 @@ static const struct etnaviv_pm_domain doms[] = {
 				&perf_reg_read
 			}
 		}
+	},
+	{
+		.name = "MC",
+		.profile_read = VIVS_MC_PROFILE_MC_READ,
+		.profile_config = VIVS_MC_PROFILE_CONFIG2,
+		.nr_signals = 3,
+		.signal = (const struct etnaviv_pm_signal[]) {
+			{
+				"TOTAL_READ_REQ_8B_FROM_PIPELINE",
+				VIVS_MC_PROFILE_CONFIG2_MC_TOTAL_READ_REQ_8B_FROM_PIPELINE,
+				&perf_reg_read
+			},
+			{
+				"TOTAL_READ_REQ_8B_FROM_IP",
+				VIVS_MC_PROFILE_CONFIG2_MC_TOTAL_READ_REQ_8B_FROM_IP,
+				&perf_reg_read
+			},
+			{
+				"TOTAL_WRITE_REQ_8B_FROM_PIPELINE",
+				VIVS_MC_PROFILE_CONFIG2_MC_TOTAL_WRITE_REQ_8B_FROM_PIPELINE,
+				&perf_reg_read
+			}
+		}
 	}
 };
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 21/23] drm/etnaviv: need to disable clock gating when doing profiling

[Christian Gmeiner]

As done by Vivante kernel driver.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index c176781788ac..5964ee36d93b 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -1335,6 +1335,13 @@ static void sync_point_perfmon_sample(struct etnaviv_gpu *gpu,
 static void sync_point_perfmon_sample_pre(struct etnaviv_gpu *gpu,
 	struct etnaviv_event *event)
 {
+	u32 val;
+
+	/* disable clock gating */
+	val = gpu_read(gpu, VIVS_PM_POWER_CONTROLS);
+	val &= ~VIVS_PM_POWER_CONTROLS_ENABLE_MODULE_CLOCK_GATING;
+	gpu_write(gpu, VIVS_PM_POWER_CONTROLS, val);
+
 	sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_PRE);
 }
 
@@ -1343,6 +1350,7 @@ static void sync_point_perfmon_sample_post(struct etnaviv_gpu *gpu,
 {
 	const struct etnaviv_cmdbuf *cmdbuf = event->cmdbuf;
 	unsigned int i;
+	u32 val;
 
 	sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_POST);
 
@@ -1351,6 +1359,11 @@ static void sync_point_perfmon_sample_post(struct etnaviv_gpu *gpu,
 
 		*pmr->bo_vma = pmr->sequence;
 	}
+
+	/* enable clock gating */
+	val = gpu_read(gpu, VIVS_PM_POWER_CONTROLS);
+	val |= VIVS_PM_POWER_CONTROLS_ENABLE_MODULE_CLOCK_GATING;
+	gpu_write(gpu, VIVS_PM_POWER_CONTROLS, val);
 }
 
 
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 22/23] drm/etnaviv: enable debug registers on demand

[Christian Gmeiner]

Some performance register are debug register and they need to
be enabled in order to be functional.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
---
 drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
index 5964ee36d93b..2301a3af8bad 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
@@ -1342,6 +1342,11 @@ static void sync_point_perfmon_sample_pre(struct etnaviv_gpu *gpu,
 	val &= ~VIVS_PM_POWER_CONTROLS_ENABLE_MODULE_CLOCK_GATING;
 	gpu_write(gpu, VIVS_PM_POWER_CONTROLS, val);
 
+	/* enable debug register */
+	val = gpu_read(gpu, VIVS_HI_CLOCK_CONTROL);
+	val &= ~VIVS_HI_CLOCK_CONTROL_DISABLE_DEBUG_REGISTERS;
+	gpu_write(gpu, VIVS_HI_CLOCK_CONTROL, val);
+
 	sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_PRE);
 }
 
@@ -1360,6 +1365,11 @@ static void sync_point_perfmon_sample_post(struct etnaviv_gpu *gpu,
 		*pmr->bo_vma = pmr->sequence;
 	}
 
+	/* disable debug register */
+	val = gpu_read(gpu, VIVS_HI_CLOCK_CONTROL);
+	val |= VIVS_HI_CLOCK_CONTROL_DISABLE_DEBUG_REGISTERS;
+	gpu_write(gpu, VIVS_HI_CLOCK_CONTROL, val);
+
 	/* enable clock gating */
 	val = gpu_read(gpu, VIVS_PM_POWER_CONTROLS);
 	val |= VIVS_PM_POWER_CONTROLS_ENABLE_MODULE_CLOCK_GATING;
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH V2 23/23] drm/etnaviv: submit supports performance monitor requests

[Christian Gmeiner]

We increment the minor driver version so userspace can detect perfmon support.

Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
Reviewed-by: Lucas Stach <l.stach@pengutronix.de>
---
 drivers/gpu/drm/etnaviv/etnaviv_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
index 536760a9fef3..eec6c00f7f80 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c
@@ -550,7 +550,7 @@ static struct drm_driver etnaviv_drm_driver = {
 	.desc               = "etnaviv DRM",
 	.date               = "20151214",
 	.major              = 1,
-	.minor              = 1,
+	.minor              = 2,
 };
 
 /*
-- 
2.13.3

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 02/23] drm/etnaviv: make it possible to allocate multiple events

[Lucas Stach]

Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
> This makes it possible to allocate multiple events under the event
> spinlock. This change is needed to support 'sync'-points.
> 
> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
> ---
>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 31 ++++++++++++++++++++-----------
>  1 file changed, 20 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> index fa9c7bd98e9c..ab108b0ed573 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> @@ -1137,10 +1137,12 @@ int etnaviv_gpu_fence_sync_obj(struct etnaviv_gem_object *etnaviv_obj,
>   * event management:
>   */
>  
> -static unsigned int event_alloc(struct etnaviv_gpu *gpu)
> +static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
> +	unsigned int *events)
>  {
>  	unsigned long ret, flags;
> -	unsigned int event;
> +	unsigned used, i;
> +	int err = 0;
>  
>  	ret = wait_for_completion_timeout(&gpu->event_free,
>  					  msecs_to_jiffies(10 * 10000));

This isn't obvious from the current code, but there are exactly as much
completions in the queue, as there are events. See initialization of the
completions in etnaviv_gpu_init(). This means the event allocation under
spinlock always succeeds if the wait hasn't timed out. To keep this
working you need to wait for the completion nr_event times, probably
changing this to an absolute timeout, so we don't lengthen the timeout
with the number of events.

> @@ -1149,16 +1151,24 @@ static unsigned int event_alloc(struct etnaviv_gpu *gpu)
>  
>  	spin_lock_irqsave(&gpu->event_spinlock, flags);
>  
> -	/* find first free event */
> -	event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
> -	if (event < ETNA_NR_EVENTS)
> +	/* are there enough free events? */
> +	used = bitmap_weight(gpu->event_bitmap, ETNA_NR_EVENTS);
> +	if (used + nr_events > ETNA_NR_EVENTS) {
> +		err = -EBUSY;
> +		goto out;
> +	}

This isn't necessary if you waited successfully for the completion to
signal nr_event times. The allocation is guaranteed to succeed in that
case. We just want an early return before even locking the spinlock,
giving back the already collected completions if one of the waits timed
out.

> +
> +	for (i = 0; i < nr_events; i++) {
> +		int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
> +
> +		events[i] = event;
>  		set_bit(event, gpu->event_bitmap);
> -	else
> -		event = ~0U;
> +	}
>  
> +out:
>  	spin_unlock_irqrestore(&gpu->event_spinlock, flags);
>  
> -	return event;
> +	return err;
>  }
>  
>  static void event_free(struct etnaviv_gpu *gpu, unsigned int event)
> @@ -1327,10 +1337,9 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>  	 *
>  	 */
>  
> -	event = event_alloc(gpu);
> -	if (unlikely(event == ~0U)) {
> +	ret = event_alloc(gpu, 1, &event);
> +	if (!ret) {
>  		DRM_ERROR("no free event\n");
> -		ret = -EBUSY;
>  		goto out_pm_put;
>  	}
>  


_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 08/23] drm/etnaviv: copy pmrs from userspace

[Lucas Stach]

Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
> Changes from v1 -> v2:
> - renamed submit_perfmon_request() to submit_perfmon_validate()
> - extended flags validation
> - added comment about offset 0
> - moved assigment of cmdbuf->nr_pmrs below the copy_from_user of the pmrs.
> 
> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
> ---
>  drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 69 +++++++++++++++++++++++++++-
>  1 file changed, 67 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> index 9c57b14dfcbf..91e35114d25c 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
> @@ -21,6 +21,7 @@
>  #include "etnaviv_drv.h"
>  #include "etnaviv_gpu.h"
>  #include "etnaviv_gem.h"
> +#include "etnaviv_perfmon.h"
>  
>  /*
>   * Cmdstream submission:
> @@ -283,6 +284,54 @@ static int submit_reloc(struct etnaviv_gem_submit *submit, void *stream,
>  	return 0;
>  }
>  
> +static int submit_perfmon_validate(struct etnaviv_gem_submit *submit,
> +		struct etnaviv_cmdbuf *cmdbuf,
> +		const struct drm_etnaviv_gem_submit_pmr *pmrs,
> +		u32 nr_pms)
> +{
> +	u32 i;
> +
> +	for (i = 0; i < nr_pms; i++) {
> +		const struct drm_etnaviv_gem_submit_pmr *r = pmrs + i;
> +		struct etnaviv_gem_submit_bo *bo;
> +		int ret;
> +
> +		ret = submit_bo(submit, r->read_idx, &bo);
> +		if (ret)
> +			return ret;
> +
> +		/* at offset 0 a sequence number gets stored used for userspace sync */
> +		if (r->read_offset == 0) {
> +			DRM_ERROR("perfmon request: offset is 0");
> +			return -EINVAL;
> +		}
> +
> +		if (r->read_offset >= bo->obj->base.size - sizeof(u32)) {
> +			DRM_ERROR("perfmon request: offset %u outside object", i);
> +			return -EINVAL;
> +		}
> +
> +		if (!(r->flags & (ETNA_PM_PROCESS_PRE | ETNA_PM_PROCESS_POST))) {

This isn't a proper validation, as it allows other bits to be set. This
should be:

if (r->flags & ~(ETNA_PM_PROCESS_PRE | ETNA_PM_PROCESS_POST))

> +			DRM_ERROR("perfmon request: flags are not valid");
> +			return -EINVAL;
> +		}
> +
> +		if (etnaviv_pm_req_validate(r)) {
> +			DRM_ERROR("perfmon request: domain or signal not valid");
> +			return -EINVAL;
> +		}
> +
> +		cmdbuf->pmrs[i].flags = r->flags;
> +		cmdbuf->pmrs[i].domain = r->domain;
> +		cmdbuf->pmrs[i].signal = r->signal;
> +		cmdbuf->pmrs[i].sequence = r->sequence;
> +		cmdbuf->pmrs[i].offset = r->read_offset;
> +		cmdbuf->pmrs[i].bo_vma = etnaviv_gem_vmap(&bo->obj->base);
> +	}
> +
> +	return 0;
> +}
> +
>  static void submit_cleanup(struct etnaviv_gem_submit *submit)
>  {
>  	unsigned i;
> @@ -306,6 +355,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>  	struct etnaviv_drm_private *priv = dev->dev_private;
>  	struct drm_etnaviv_gem_submit *args = data;
>  	struct drm_etnaviv_gem_submit_reloc *relocs;
> +	struct drm_etnaviv_gem_submit_pmr *pmrs;
>  	struct drm_etnaviv_gem_submit_bo *bos;
>  	struct etnaviv_gem_submit *submit;
>  	struct etnaviv_cmdbuf *cmdbuf;
> @@ -347,11 +397,12 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>  	 */
>  	bos = kvmalloc_array(args->nr_bos, sizeof(*bos), GFP_KERNEL);
>  	relocs = kvmalloc_array(args->nr_relocs, sizeof(*relocs), GFP_KERNEL);
> +	pmrs = kvmalloc_array(args->nr_pmrs, sizeof(*pmrs), GFP_KERNEL);
>  	stream = kvmalloc_array(1, args->stream_size, GFP_KERNEL);
>  	cmdbuf = etnaviv_cmdbuf_new(gpu->cmdbuf_suballoc,
>  				    ALIGN(args->stream_size, 8) + 8,
> -				    args->nr_bos, 0);
> -	if (!bos || !relocs || !stream || !cmdbuf) {
> +				    args->nr_bos, args->nr_pmrs);
> +	if (!bos || !relocs || !pmrs || !stream || !cmdbuf) {
>  		ret = -ENOMEM;
>  		goto err_submit_cmds;
>  	}
> @@ -373,6 +424,14 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>  		goto err_submit_cmds;
>  	}
>  
> +	ret = copy_from_user(pmrs, u64_to_user_ptr(args->pmrs),
> +			     args->nr_pmrs * sizeof(*pmrs));
> +	if (ret) {
> +		ret = -EFAULT;
> +		goto err_submit_cmds;
> +	}
> +	cmdbuf->nr_pmrs = args->nr_pmrs;
> +
>  	ret = copy_from_user(stream, u64_to_user_ptr(args->stream),
>  			     args->stream_size);
>  	if (ret) {
> @@ -441,6 +500,10 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>  	if (ret)
>  		goto out;
>  
> +	ret = submit_perfmon_validate(submit, cmdbuf, pmrs, args->nr_pmrs);
> +	if (ret)
> +		goto out;
> +
>  	memcpy(cmdbuf->vaddr, stream, args->stream_size);
>  	cmdbuf->user_size = ALIGN(args->stream_size, 8);
>  
> @@ -494,6 +557,8 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>  		kvfree(bos);
>  	if (relocs)
>  		kvfree(relocs);
> +	if (pmrs)
> +		kvfree(pmrs);
>  
>  	return ret;
>  }


_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 10/23] drm/etnaviv: add 'sync point' support

[Lucas Stach]

Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
> In order to support performance counters in a sane way we need to provide
> a method to sync the GPU with the CPU. The GPU can process multpile command
> buffers/events per irq. With the help of a 'sync point' we can trigger an event
> and stop the GPU/FE immediately. When the CPU is done with is processing it
> simply needs to restart the FE and the GPU will process the command stream.
> 
> Changes from v1 -> v2:
> - process sync point with a work item to keep irq as fast as possible
> 
> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
> ---
>  drivers/gpu/drm/etnaviv/etnaviv_buffer.c | 36 ++++++++++++++++++++++++++++++++
>  drivers/gpu/drm/etnaviv/etnaviv_drv.h    |  1 +
>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c    | 25 ++++++++++++++++++++++
>  drivers/gpu/drm/etnaviv/etnaviv_gpu.h    |  6 ++++++
>  4 files changed, 68 insertions(+)
> 
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
> index ed9588f36bc9..9e7098e3207f 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
> @@ -250,6 +250,42 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
>  	}
>  }
>  
> +/* Append a 'sync point' to the ring buffer. */
> +void etnaviv_sync_point_queue(struct etnaviv_gpu *gpu, unsigned int event)
> +{
> +	struct etnaviv_cmdbuf *buffer = gpu->buffer;
> +	unsigned int waitlink_offset = buffer->user_size - 16;
> +	u32 dwords, target;
> +
> +	/*
> +	 * We need at most 3 dwords in the return target:
> +	 * 1 event + 1 end + 1 wait + 1 link.
> +	 */
> +	dwords = 4;
> +	target = etnaviv_buffer_reserve(gpu, buffer, dwords);
> +
> +	/* Signal sync point event */
> +	CMD_LOAD_STATE(buffer, VIVS_GL_EVENT, VIVS_GL_EVENT_EVENT_ID(event) |
> +		       VIVS_GL_EVENT_FROM_PE);
> +
> +	/* Stop the FE to 'pause' the GPU */
> +	CMD_END(buffer);
> +
> +	/* Append waitlink */
> +	CMD_WAIT(buffer);
> +	CMD_LINK(buffer, 2, etnaviv_cmdbuf_get_va(buffer) +
> +			    buffer->user_size - 4);
> +
> +	/*
> +	 * Kick off the 'sync point' command by replacing the previous
> +	 * WAIT with a link to the address in the ring buffer.
> +	 */
> +	etnaviv_buffer_replace_wait(buffer, waitlink_offset,
> +				    VIV_FE_LINK_HEADER_OP_LINK |
> +				    VIV_FE_LINK_HEADER_PREFETCH(dwords),
> +				    target);
> +}
> +
>  /* Append a command buffer to the ring buffer. */
>  void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, unsigned int event,
>  	struct etnaviv_cmdbuf *cmdbuf)
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.h b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
> index 058389f93b69..f6cdd694ca51 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_drv.h
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
> @@ -101,6 +101,7 @@ int etnaviv_gem_new_userptr(struct drm_device *dev, struct drm_file *file,
>  u16 etnaviv_buffer_init(struct etnaviv_gpu *gpu);
>  u16 etnaviv_buffer_config_mmuv2(struct etnaviv_gpu *gpu, u32 mtlb_addr, u32 safe_addr);
>  void etnaviv_buffer_end(struct etnaviv_gpu *gpu);
> +void etnaviv_sync_point_queue(struct etnaviv_gpu *gpu, unsigned int event);
>  void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, unsigned int event,
>  	struct etnaviv_cmdbuf *cmdbuf);
>  void etnaviv_validate_init(void);
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> index ce6c869e214f..bc1b96b4c7e9 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> @@ -25,6 +25,7 @@
>  #include "etnaviv_gpu.h"
>  #include "etnaviv_gem.h"
>  #include "etnaviv_mmu.h"
> +#include "etnaviv_perfmon.h"
>  #include "common.xml.h"
>  #include "state.xml.h"
>  #include "state_hi.xml.h"
> @@ -1353,6 +1354,7 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>  	}
>  
>  	gpu->event[event].fence = fence;
> +	gpu->event[event].sync_point = NULL;
>  	submit->fence = dma_fence_get(fence);
>  	gpu->active_fence = submit->fence->seqno;
>  
> @@ -1398,6 +1400,23 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>  	return ret;
>  }
>  
> +static void etnaviv_process_sync_point(struct etnaviv_gpu *gpu,
> +	struct etnaviv_event *event)
> +{
> +	u32 addr = gpu_read(gpu, VIVS_FE_DMA_ADDRESS);
> +
> +	event->sync_point(gpu, event);
> +	etnaviv_gpu_start_fe(gpu, addr + 2, 2);
> +}
> +
> +static void pmrs_worker(struct work_struct *work)
> +{
> +	struct etnaviv_gpu *gpu = container_of(work, struct etnaviv_gpu,
> +					       pmrs_work);
> +
> +	etnaviv_process_sync_point(gpu, &gpu->event[gpu->pmrs_event]);
> +}
> +
>  /*
>   * Init/Cleanup:
>   */
> @@ -1444,6 +1463,11 @@ static irqreturn_t irq_handler(int irq, void *data)
>  
>  			dev_dbg(gpu->dev, "event %u\n", event);
>  
> +			if (gpu->event[event].sync_point) {
> +				gpu->pmrs_event = event;
> +				etnaviv_queue_work(gpu->drm, &gpu->pmrs_work);

If the handler is delayed we might handle multiple events per
invocation, in which case the events might not be in order. E.g. the FE
stop event might be event 30, while the FE start event might be event 0.
In that case you would execute the FE start before the FE stop has been
queued -> not good. You need to make sure that your PMRS events are
processed in the correct order.

> +			}
> +
>  			fence = gpu->event[event].fence;
>  			gpu->event[event].fence = NULL;
>  			dma_fence_signal(fence);
> @@ -1647,6 +1671,7 @@ static int etnaviv_gpu_bind(struct device *dev, struct device *master,
>  
>  	INIT_LIST_HEAD(&gpu->active_cmd_list);
>  	INIT_WORK(&gpu->retire_work, retire_worker);
> +	INIT_WORK(&gpu->pmrs_work, pmrs_worker);
>  	INIT_WORK(&gpu->recover_work, recover_worker);
>  	init_waitqueue_head(&gpu->fence_event);
>  
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
> index 70e6590aacdf..7d5f785b0f08 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
> @@ -89,6 +89,8 @@ struct etnaviv_chip_identity {
>  
>  struct etnaviv_event {
>  	struct dma_fence *fence;
> +
> +	void (*sync_point)(struct etnaviv_gpu *gpu, struct etnaviv_event *event);
>  };
>  
>  struct etnaviv_cmdbuf_suballoc;
> @@ -135,6 +137,10 @@ struct etnaviv_gpu {
>  	/* worker for handling active-list retiring: */
>  	struct work_struct retire_work;
>  
> +	/* worker for handling performance monitor requests: */
> +	struct work_struct pmrs_work;
> +	int pmrs_event;
> +
>  	void __iomem *mmio;
>  	int irq;
>  


_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 12/23] drm/etnaviv: use 'sync points' for performance monitor requests

[Lucas Stach]

Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
> With 'sync points' we can sample the reqeustes perform signals
> before and/or after the submited command buffer.
> 
> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
> ---
>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 106 +++++++++++++++++++++++++++-------
>  drivers/gpu/drm/etnaviv/etnaviv_gpu.h |   1 +
>  2 files changed, 86 insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> index 93e3f14c0599..c176781788ac 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> @@ -1318,12 +1318,48 @@ void etnaviv_gpu_pm_put(struct etnaviv_gpu *gpu)
>  	pm_runtime_put_autosuspend(gpu->dev);
>  }
>  
> +static void sync_point_perfmon_sample(struct etnaviv_gpu *gpu,
> +	struct etnaviv_event *event, unsigned int flags)
> +{
> +	const struct etnaviv_cmdbuf *cmdbuf = event->cmdbuf;
> +	unsigned int i;
> +
> +	for (i = 0; i < cmdbuf->nr_pmrs; i++) {
> +		const struct etnaviv_perfmon_request *pmr = cmdbuf->pmrs + i;
> +
> +		if (pmr->flags == flags)
> +			etnaviv_perfmon_process(gpu, pmr);
> +	}
> +}
> +
> +static void sync_point_perfmon_sample_pre(struct etnaviv_gpu *gpu,
> +	struct etnaviv_event *event)
> +{
> +	sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_PRE);
> +}
> +
> +static void sync_point_perfmon_sample_post(struct etnaviv_gpu *gpu,
> +	struct etnaviv_event *event)
> +{
> +	const struct etnaviv_cmdbuf *cmdbuf = event->cmdbuf;
> +	unsigned int i;
> +
> +	sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_POST);
> +
> +	for (i = 0; i < cmdbuf->nr_pmrs; i++) {
> +		const struct etnaviv_perfmon_request *pmr = cmdbuf->pmrs + i;
> +
> +		*pmr->bo_vma = pmr->sequence;
> +	}
> +}
> +
> +
>  /* add bo's to gpu's ring, and kick gpu: */
>  int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>  	struct etnaviv_gem_submit *submit, struct etnaviv_cmdbuf *cmdbuf)
>  {
>  	struct dma_fence *fence;
> -	unsigned int event, i;
> +	unsigned int i, nr_events, event[3];
>  	int ret;
>  
>  	ret = etnaviv_gpu_pm_get_sync(gpu);
> @@ -1339,9 +1375,21 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>  	 *
>  	 */
>  
> -	ret = event_alloc(gpu, 1, &event);
> -	if (!ret) {
> -		DRM_ERROR("no free event\n");
> +	 /*
> + 	 * if there are performance monitor requests we need to have
> +	 * - a sync point to re-configure gpu and process ETNA_PM_PROCESS_PRE
> +	 *   requests.
> +	 * - a sync point to re-configure gpu, process ETNA_PM_PROCESS_POST requests
> +	 *   and update the sequence number for userspace.
> + 	 */
> +	 if (cmdbuf->nr_pmrs)
> +	 	nr_events = 3;
> +	else
> +		nr_events = 1;

Indentation of comment and code is off here. Also I would prefer if
nr_events is just initialized to 1, so we can spare the else path.

> +
> +	ret = event_alloc(gpu, nr_events, event);
> +	if (ret < 0) {
> +		DRM_ERROR("no free events\n");
>  		goto out_pm_put;
>  	}
>  
> @@ -1349,12 +1397,14 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>  
>  	fence = etnaviv_gpu_fence_alloc(gpu);
>  	if (!fence) {
> -		event_free(gpu, event);
> +		for (i = 0; i < nr_events; i++)
> +			event_free(gpu, event[i]);
> +
>  		ret = -ENOMEM;
>  		goto out_unlock;
>  	}
>  
> -	gpu->event[event].fence = fence;
> +	gpu->event[event[0]].fence = fence;
>  	submit->fence = dma_fence_get(fence);
>  	gpu->active_fence = submit->fence->seqno;
>  
> @@ -1364,7 +1414,19 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>  		gpu->lastctx = cmdbuf->ctx;
>  	}
>  
> -	etnaviv_buffer_queue(gpu, event, cmdbuf);
> +	if (cmdbuf->nr_pmrs) {
> +		gpu->event[event[1]].sync_point = &sync_point_perfmon_sample_pre;
> +		gpu->event[event[1]].cmdbuf = cmdbuf;
> +		etnaviv_sync_point_queue(gpu, event[1]);
> +	}
> +
> +	etnaviv_buffer_queue(gpu, event[0], cmdbuf);
> +
> +	if (cmdbuf->nr_pmrs) {
> +		gpu->event[event[2]].sync_point = &sync_point_perfmon_sample_post;
> +		gpu->event[event[2]].cmdbuf = cmdbuf;
> +		etnaviv_sync_point_queue(gpu, event[2]);
> +	}
>  
>  	cmdbuf->fence = fence;
>  	list_add_tail(&cmdbuf->node, &gpu->active_cmd_list);
> @@ -1469,20 +1531,22 @@ static irqreturn_t irq_handler(int irq, void *data)
>  			}
>  
>  			fence = gpu->event[event].fence;
> -			gpu->event[event].fence = NULL;
> -			dma_fence_signal(fence);
> -
> -			/*
> -			 * Events can be processed out of order.  Eg,
> -			 * - allocate and queue event 0
> -			 * - allocate event 1
> -			 * - event 0 completes, we process it
> -			 * - allocate and queue event 0
> -			 * - event 1 and event 0 complete
> -			 * we can end up processing event 0 first, then 1.
> -			 */
> -			if (fence_after(fence->seqno, gpu->completed_fence))
> -				gpu->completed_fence = fence->seqno;
> +			if (fence) {
> +				gpu->event[event].fence = NULL;
> +				dma_fence_signal(fence);
> +
> +				/*
> +				 * Events can be processed out of order.  Eg,
> +				 * - allocate and queue event 0
> +				 * - allocate event 1
> +				 * - event 0 completes, we process it
> +				 * - allocate and queue event 0
> +				 * - event 1 and event 0 complete
> +				 * we can end up processing event 0 first, then 1.
> +				 */
> +				if (fence_after(fence->seqno, gpu->completed_fence))
> +					gpu->completed_fence = fence->seqno;
> +			}
>  
>  			event_free(gpu, event);
>  		}
> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
> index 7d5f785b0f08..c75a7b5c397c 100644
> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
> @@ -89,6 +89,7 @@ struct etnaviv_chip_identity {
>  
>  struct etnaviv_event {
>  	struct dma_fence *fence;
> +	struct etnaviv_cmdbuf *cmdbuf;
>  
>  	void (*sync_point)(struct etnaviv_gpu *gpu, struct etnaviv_event *event);
>  };


_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 02/23] drm/etnaviv: make it possible to allocate multiple events

[Christian Gmeiner]

Hi Lucas.

Thanks for your review - hopefully there will be only one last v3
series of that patches.

2017-08-08 12:00 GMT+02:00 Lucas Stach <l.stach@pengutronix.de>:
> Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
>> This makes it possible to allocate multiple events under the event
>> spinlock. This change is needed to support 'sync'-points.
>>
>> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
>> ---
>>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 31 ++++++++++++++++++++-----------
>>  1 file changed, 20 insertions(+), 11 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> index fa9c7bd98e9c..ab108b0ed573 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> @@ -1137,10 +1137,12 @@ int etnaviv_gpu_fence_sync_obj(struct etnaviv_gem_object *etnaviv_obj,
>>   * event management:
>>   */
>>
>> -static unsigned int event_alloc(struct etnaviv_gpu *gpu)
>> +static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
>> +     unsigned int *events)
>>  {
>>       unsigned long ret, flags;
>> -     unsigned int event;
>> +     unsigned used, i;
>> +     int err = 0;
>>
>>       ret = wait_for_completion_timeout(&gpu->event_free,
>>                                         msecs_to_jiffies(10 * 10000));
>
> This isn't obvious from the current code, but there are exactly as much
> completions in the queue, as there are events. See initialization of the
> completions in etnaviv_gpu_init(). This means the event allocation under
> spinlock always succeeds if the wait hasn't timed out. To keep this
> working you need to wait for the completion nr_event times, probably
> changing this to an absolute timeout, so we don't lengthen the timeout
> with the number of events.
>

Makes sense but I not really sure how to best implement an absolute
timeout. We could wait
absolute timeout / nr_events in each wait_for_completion_timeout(..) call.

>> @@ -1149,16 +1151,24 @@ static unsigned int event_alloc(struct etnaviv_gpu *gpu)
>>
>>       spin_lock_irqsave(&gpu->event_spinlock, flags);
>>
>> -     /* find first free event */
>> -     event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
>> -     if (event < ETNA_NR_EVENTS)
>> +     /* are there enough free events? */
>> +     used = bitmap_weight(gpu->event_bitmap, ETNA_NR_EVENTS);
>> +     if (used + nr_events > ETNA_NR_EVENTS) {
>> +             err = -EBUSY;
>> +             goto out;
>> +     }
>
> This isn't necessary if you waited successfully for the completion to
> signal nr_event times. The allocation is guaranteed to succeed in that
> case. We just want an early return before even locking the spinlock,
> giving back the already collected completions if one of the waits timed
> out.
>

Yes - feels more elegant that way.

>> +
>> +     for (i = 0; i < nr_events; i++) {
>> +             int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
>> +
>> +             events[i] = event;
>>               set_bit(event, gpu->event_bitmap);
>> -     else
>> -             event = ~0U;
>> +     }
>>
>> +out:
>>       spin_unlock_irqrestore(&gpu->event_spinlock, flags);
>>
>> -     return event;
>> +     return err;
>>  }
>>
>>  static void event_free(struct etnaviv_gpu *gpu, unsigned int event)
>> @@ -1327,10 +1337,9 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>>        *
>>        */
>>
>> -     event = event_alloc(gpu);
>> -     if (unlikely(event == ~0U)) {
>> +     ret = event_alloc(gpu, 1, &event);
>> +     if (!ret) {
>>               DRM_ERROR("no free event\n");
>> -             ret = -EBUSY;
>>               goto out_pm_put;
>>       }
>>
>
>

What about something like this:

static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
    unsigned int *events)
{
    unsigned long flags;
    unsigned i;
    int err = 0;

    for (i = 0; i < nr_events; i++) {
        unsigned long ret;

        ret = wait_for_completion_timeout(&gpu->event_free,
                        msecs_to_jiffies(10 * 10000));
        if (!ret) {
            dev_err(gpu->dev, "wait_for_completion_timeout failed");
            err = -EBUSY;
            goto out;
        }
    }

    spin_lock_irqsave(&gpu->event_spinlock, flags);

    for (i = 0; i < nr_events; i++) {
        int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);

        events[i] = event;
        set_bit(event, gpu->event_bitmap);
    }

    spin_unlock_irqrestore(&gpu->event_spinlock, flags);

out:
    return err;
}

greets
--
Christian Gmeiner, MSc

https://christian-gmeiner.info
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 08/23] drm/etnaviv: copy pmrs from userspace

[Christian Gmeiner]

2017-08-08 12:13 GMT+02:00 Lucas Stach <l.stach@pengutronix.de>:
> Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
>> Changes from v1 -> v2:
>> - renamed submit_perfmon_request() to submit_perfmon_validate()
>> - extended flags validation
>> - added comment about offset 0
>> - moved assigment of cmdbuf->nr_pmrs below the copy_from_user of the pmrs.
>>
>> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
>> ---
>>  drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c | 69 +++++++++++++++++++++++++++-
>>  1 file changed, 67 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
>> index 9c57b14dfcbf..91e35114d25c 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c
>> @@ -21,6 +21,7 @@
>>  #include "etnaviv_drv.h"
>>  #include "etnaviv_gpu.h"
>>  #include "etnaviv_gem.h"
>> +#include "etnaviv_perfmon.h"
>>
>>  /*
>>   * Cmdstream submission:
>> @@ -283,6 +284,54 @@ static int submit_reloc(struct etnaviv_gem_submit *submit, void *stream,
>>       return 0;
>>  }
>>
>> +static int submit_perfmon_validate(struct etnaviv_gem_submit *submit,
>> +             struct etnaviv_cmdbuf *cmdbuf,
>> +             const struct drm_etnaviv_gem_submit_pmr *pmrs,
>> +             u32 nr_pms)
>> +{
>> +     u32 i;
>> +
>> +     for (i = 0; i < nr_pms; i++) {
>> +             const struct drm_etnaviv_gem_submit_pmr *r = pmrs + i;
>> +             struct etnaviv_gem_submit_bo *bo;
>> +             int ret;
>> +
>> +             ret = submit_bo(submit, r->read_idx, &bo);
>> +             if (ret)
>> +                     return ret;
>> +
>> +             /* at offset 0 a sequence number gets stored used for userspace sync */
>> +             if (r->read_offset == 0) {
>> +                     DRM_ERROR("perfmon request: offset is 0");
>> +                     return -EINVAL;
>> +             }
>> +
>> +             if (r->read_offset >= bo->obj->base.size - sizeof(u32)) {
>> +                     DRM_ERROR("perfmon request: offset %u outside object", i);
>> +                     return -EINVAL;
>> +             }
>> +
>> +             if (!(r->flags & (ETNA_PM_PROCESS_PRE | ETNA_PM_PROCESS_POST))) {
>
> This isn't a proper validation, as it allows other bits to be set. This
> should be:
>
> if (r->flags & ~(ETNA_PM_PROCESS_PRE | ETNA_PM_PROCESS_POST))

Fixed in v3

>
>> +                     DRM_ERROR("perfmon request: flags are not valid");
>> +                     return -EINVAL;
>> +             }
>> +
>> +             if (etnaviv_pm_req_validate(r)) {
>> +                     DRM_ERROR("perfmon request: domain or signal not valid");
>> +                     return -EINVAL;
>> +             }
>> +
>> +             cmdbuf->pmrs[i].flags = r->flags;
>> +             cmdbuf->pmrs[i].domain = r->domain;
>> +             cmdbuf->pmrs[i].signal = r->signal;
>> +             cmdbuf->pmrs[i].sequence = r->sequence;
>> +             cmdbuf->pmrs[i].offset = r->read_offset;
>> +             cmdbuf->pmrs[i].bo_vma = etnaviv_gem_vmap(&bo->obj->base);
>> +     }
>> +
>> +     return 0;
>> +}
>> +
>>  static void submit_cleanup(struct etnaviv_gem_submit *submit)
>>  {
>>       unsigned i;
>> @@ -306,6 +355,7 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>>       struct etnaviv_drm_private *priv = dev->dev_private;
>>       struct drm_etnaviv_gem_submit *args = data;
>>       struct drm_etnaviv_gem_submit_reloc *relocs;
>> +     struct drm_etnaviv_gem_submit_pmr *pmrs;
>>       struct drm_etnaviv_gem_submit_bo *bos;
>>       struct etnaviv_gem_submit *submit;
>>       struct etnaviv_cmdbuf *cmdbuf;
>> @@ -347,11 +397,12 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>>        */
>>       bos = kvmalloc_array(args->nr_bos, sizeof(*bos), GFP_KERNEL);
>>       relocs = kvmalloc_array(args->nr_relocs, sizeof(*relocs), GFP_KERNEL);
>> +     pmrs = kvmalloc_array(args->nr_pmrs, sizeof(*pmrs), GFP_KERNEL);
>>       stream = kvmalloc_array(1, args->stream_size, GFP_KERNEL);
>>       cmdbuf = etnaviv_cmdbuf_new(gpu->cmdbuf_suballoc,
>>                                   ALIGN(args->stream_size, 8) + 8,
>> -                                 args->nr_bos, 0);
>> -     if (!bos || !relocs || !stream || !cmdbuf) {
>> +                                 args->nr_bos, args->nr_pmrs);
>> +     if (!bos || !relocs || !pmrs || !stream || !cmdbuf) {
>>               ret = -ENOMEM;
>>               goto err_submit_cmds;
>>       }
>> @@ -373,6 +424,14 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>>               goto err_submit_cmds;
>>       }
>>
>> +     ret = copy_from_user(pmrs, u64_to_user_ptr(args->pmrs),
>> +                          args->nr_pmrs * sizeof(*pmrs));
>> +     if (ret) {
>> +             ret = -EFAULT;
>> +             goto err_submit_cmds;
>> +     }
>> +     cmdbuf->nr_pmrs = args->nr_pmrs;
>> +
>>       ret = copy_from_user(stream, u64_to_user_ptr(args->stream),
>>                            args->stream_size);
>>       if (ret) {
>> @@ -441,6 +500,10 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>>       if (ret)
>>               goto out;
>>
>> +     ret = submit_perfmon_validate(submit, cmdbuf, pmrs, args->nr_pmrs);
>> +     if (ret)
>> +             goto out;
>> +
>>       memcpy(cmdbuf->vaddr, stream, args->stream_size);
>>       cmdbuf->user_size = ALIGN(args->stream_size, 8);
>>
>> @@ -494,6 +557,8 @@ int etnaviv_ioctl_gem_submit(struct drm_device *dev, void *data,
>>               kvfree(bos);
>>       if (relocs)
>>               kvfree(relocs);
>> +     if (pmrs)
>> +             kvfree(pmrs);
>>
>>       return ret;
>>  }
>
>

greets
--
Christian Gmeiner, MSc

https://christian-gmeiner.info
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 12/23] drm/etnaviv: use 'sync points' for performance monitor requests

[Christian Gmeiner]

2017-08-08 12:49 GMT+02:00 Lucas Stach <l.stach@pengutronix.de>:
> Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
>> With 'sync points' we can sample the reqeustes perform signals
>> before and/or after the submited command buffer.
>>
>> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
>> ---
>>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 106 +++++++++++++++++++++++++++-------
>>  drivers/gpu/drm/etnaviv/etnaviv_gpu.h |   1 +
>>  2 files changed, 86 insertions(+), 21 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> index 93e3f14c0599..c176781788ac 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> @@ -1318,12 +1318,48 @@ void etnaviv_gpu_pm_put(struct etnaviv_gpu *gpu)
>>       pm_runtime_put_autosuspend(gpu->dev);
>>  }
>>
>> +static void sync_point_perfmon_sample(struct etnaviv_gpu *gpu,
>> +     struct etnaviv_event *event, unsigned int flags)
>> +{
>> +     const struct etnaviv_cmdbuf *cmdbuf = event->cmdbuf;
>> +     unsigned int i;
>> +
>> +     for (i = 0; i < cmdbuf->nr_pmrs; i++) {
>> +             const struct etnaviv_perfmon_request *pmr = cmdbuf->pmrs + i;
>> +
>> +             if (pmr->flags == flags)
>> +                     etnaviv_perfmon_process(gpu, pmr);
>> +     }
>> +}
>> +
>> +static void sync_point_perfmon_sample_pre(struct etnaviv_gpu *gpu,
>> +     struct etnaviv_event *event)
>> +{
>> +     sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_PRE);
>> +}
>> +
>> +static void sync_point_perfmon_sample_post(struct etnaviv_gpu *gpu,
>> +     struct etnaviv_event *event)
>> +{
>> +     const struct etnaviv_cmdbuf *cmdbuf = event->cmdbuf;
>> +     unsigned int i;
>> +
>> +     sync_point_perfmon_sample(gpu, event, ETNA_PM_PROCESS_POST);
>> +
>> +     for (i = 0; i < cmdbuf->nr_pmrs; i++) {
>> +             const struct etnaviv_perfmon_request *pmr = cmdbuf->pmrs + i;
>> +
>> +             *pmr->bo_vma = pmr->sequence;
>> +     }
>> +}
>> +
>> +
>>  /* add bo's to gpu's ring, and kick gpu: */
>>  int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>>       struct etnaviv_gem_submit *submit, struct etnaviv_cmdbuf *cmdbuf)
>>  {
>>       struct dma_fence *fence;
>> -     unsigned int event, i;
>> +     unsigned int i, nr_events, event[3];
>>       int ret;
>>
>>       ret = etnaviv_gpu_pm_get_sync(gpu);
>> @@ -1339,9 +1375,21 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>>        *
>>        */
>>
>> -     ret = event_alloc(gpu, 1, &event);
>> -     if (!ret) {
>> -             DRM_ERROR("no free event\n");
>> +      /*
>> +      * if there are performance monitor requests we need to have
>> +      * - a sync point to re-configure gpu and process ETNA_PM_PROCESS_PRE
>> +      *   requests.
>> +      * - a sync point to re-configure gpu, process ETNA_PM_PROCESS_POST requests
>> +      *   and update the sequence number for userspace.
>> +      */
>> +      if (cmdbuf->nr_pmrs)
>> +             nr_events = 3;
>> +     else
>> +             nr_events = 1;
>
> Indentation of comment and code is off here. Also I would prefer if
> nr_events is just initialized to 1, so we can spare the else path.
>

Both will be fixed in V3.

>> +
>> +     ret = event_alloc(gpu, nr_events, event);
>> +     if (ret < 0) {
>> +             DRM_ERROR("no free events\n");
>>               goto out_pm_put;
>>       }
>>
>> @@ -1349,12 +1397,14 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>>
>>       fence = etnaviv_gpu_fence_alloc(gpu);
>>       if (!fence) {
>> -             event_free(gpu, event);
>> +             for (i = 0; i < nr_events; i++)
>> +                     event_free(gpu, event[i]);
>> +
>>               ret = -ENOMEM;
>>               goto out_unlock;
>>       }
>>
>> -     gpu->event[event].fence = fence;
>> +     gpu->event[event[0]].fence = fence;
>>       submit->fence = dma_fence_get(fence);
>>       gpu->active_fence = submit->fence->seqno;
>>
>> @@ -1364,7 +1414,19 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>>               gpu->lastctx = cmdbuf->ctx;
>>       }
>>
>> -     etnaviv_buffer_queue(gpu, event, cmdbuf);
>> +     if (cmdbuf->nr_pmrs) {
>> +             gpu->event[event[1]].sync_point = &sync_point_perfmon_sample_pre;
>> +             gpu->event[event[1]].cmdbuf = cmdbuf;
>> +             etnaviv_sync_point_queue(gpu, event[1]);
>> +     }
>> +
>> +     etnaviv_buffer_queue(gpu, event[0], cmdbuf);
>> +
>> +     if (cmdbuf->nr_pmrs) {
>> +             gpu->event[event[2]].sync_point = &sync_point_perfmon_sample_post;
>> +             gpu->event[event[2]].cmdbuf = cmdbuf;
>> +             etnaviv_sync_point_queue(gpu, event[2]);
>> +     }
>>
>>       cmdbuf->fence = fence;
>>       list_add_tail(&cmdbuf->node, &gpu->active_cmd_list);
>> @@ -1469,20 +1531,22 @@ static irqreturn_t irq_handler(int irq, void *data)
>>                       }
>>
>>                       fence = gpu->event[event].fence;
>> -                     gpu->event[event].fence = NULL;
>> -                     dma_fence_signal(fence);
>> -
>> -                     /*
>> -                      * Events can be processed out of order.  Eg,
>> -                      * - allocate and queue event 0
>> -                      * - allocate event 1
>> -                      * - event 0 completes, we process it
>> -                      * - allocate and queue event 0
>> -                      * - event 1 and event 0 complete
>> -                      * we can end up processing event 0 first, then 1.
>> -                      */
>> -                     if (fence_after(fence->seqno, gpu->completed_fence))
>> -                             gpu->completed_fence = fence->seqno;
>> +                     if (fence) {
>> +                             gpu->event[event].fence = NULL;
>> +                             dma_fence_signal(fence);
>> +
>> +                             /*
>> +                              * Events can be processed out of order.  Eg,
>> +                              * - allocate and queue event 0
>> +                              * - allocate event 1
>> +                              * - event 0 completes, we process it
>> +                              * - allocate and queue event 0
>> +                              * - event 1 and event 0 complete
>> +                              * we can end up processing event 0 first, then 1.
>> +                              */
>> +                             if (fence_after(fence->seqno, gpu->completed_fence))
>> +                                     gpu->completed_fence = fence->seqno;
>> +                     }
>>
>>                       event_free(gpu, event);
>>               }
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
>> index 7d5f785b0f08..c75a7b5c397c 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
>> @@ -89,6 +89,7 @@ struct etnaviv_chip_identity {
>>
>>  struct etnaviv_event {
>>       struct dma_fence *fence;
>> +     struct etnaviv_cmdbuf *cmdbuf;
>>
>>       void (*sync_point)(struct etnaviv_gpu *gpu, struct etnaviv_event *event);
>>  };
>
>

greets
--
Christian Gmeiner, MSc

https://christian-gmeiner.info
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 02/23] drm/etnaviv: make it possible to allocate multiple events

[Lucas Stach]

Hi Christian,

Am Dienstag, den 22.08.2017, 10:27 +0200 schrieb Christian Gmeiner:
> Hi Lucas.
> 
> Thanks for your review - hopefully there will be only one last v3
> series of that patches.

Yep, I would really like to merge this series. It's been in the making
for long enough. :)

> 2017-08-08 12:00 GMT+02:00 Lucas Stach <l.stach@pengutronix.de>:
> > Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
> >> This makes it possible to allocate multiple events under the event
> >> spinlock. This change is needed to support 'sync'-points.
> >>
> >> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
> >> ---
> >>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 31 ++++++++++++++++++++-----------
> >>  1 file changed, 20 insertions(+), 11 deletions(-)
> >>
> >> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> >> index fa9c7bd98e9c..ab108b0ed573 100644
> >> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> >> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
> >> @@ -1137,10 +1137,12 @@ int etnaviv_gpu_fence_sync_obj(struct etnaviv_gem_object *etnaviv_obj,
> >>   * event management:
> >>   */
> >>
> >> -static unsigned int event_alloc(struct etnaviv_gpu *gpu)
> >> +static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
> >> +     unsigned int *events)
> >>  {
> >>       unsigned long ret, flags;
> >> -     unsigned int event;
> >> +     unsigned used, i;
> >> +     int err = 0;
> >>
> >>       ret = wait_for_completion_timeout(&gpu->event_free,
> >>                                         msecs_to_jiffies(10 * 10000));
> >
> > This isn't obvious from the current code, but there are exactly as much
> > completions in the queue, as there are events. See initialization of the
> > completions in etnaviv_gpu_init(). This means the event allocation under
> > spinlock always succeeds if the wait hasn't timed out. To keep this
> > working you need to wait for the completion nr_event times, probably
> > changing this to an absolute timeout, so we don't lengthen the timeout
> > with the number of events.
> >
> 
> Makes sense but I not really sure how to best implement an absolute
> timeout. We could wait
> absolute timeout / nr_events in each wait_for_completion_timeout(..) call.

I think we should just keep the 10sec timeout regardless of the number
of events. If we don't acquire the needed events after 10secs, it's
probably fine to give up.

> >> @@ -1149,16 +1151,24 @@ static unsigned int event_alloc(struct etnaviv_gpu *gpu)
> >>
> >>       spin_lock_irqsave(&gpu->event_spinlock, flags);
> >>
> >> -     /* find first free event */
> >> -     event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
> >> -     if (event < ETNA_NR_EVENTS)
> >> +     /* are there enough free events? */
> >> +     used = bitmap_weight(gpu->event_bitmap, ETNA_NR_EVENTS);
> >> +     if (used + nr_events > ETNA_NR_EVENTS) {
> >> +             err = -EBUSY;
> >> +             goto out;
> >> +     }
> >
> > This isn't necessary if you waited successfully for the completion to
> > signal nr_event times. The allocation is guaranteed to succeed in that
> > case. We just want an early return before even locking the spinlock,
> > giving back the already collected completions if one of the waits timed
> > out.
> >
> 
> Yes - feels more elegant that way.
> 
> >> +
> >> +     for (i = 0; i < nr_events; i++) {
> >> +             int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
> >> +
> >> +             events[i] = event;
> >>               set_bit(event, gpu->event_bitmap);
> >> -     else
> >> -             event = ~0U;
> >> +     }
> >>
> >> +out:
> >>       spin_unlock_irqrestore(&gpu->event_spinlock, flags);
> >>
> >> -     return event;
> >> +     return err;
> >>  }
> >>
> >>  static void event_free(struct etnaviv_gpu *gpu, unsigned int event)
> >> @@ -1327,10 +1337,9 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
> >>        *
> >>        */
> >>
> >> -     event = event_alloc(gpu);
> >> -     if (unlikely(event == ~0U)) {
> >> +     ret = event_alloc(gpu, 1, &event);
> >> +     if (!ret) {
> >>               DRM_ERROR("no free event\n");
> >> -             ret = -EBUSY;
> >>               goto out_pm_put;
> >>       }
> >>
> >
> >
> 
> What about something like this:

A few notes inline.

> 
> static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
>     unsigned int *events)
> {
>     unsigned long flags;
>     unsigned i;
>     int err = 0;
> 
>     for (i = 0; i < nr_events; i++) {
>         unsigned long ret;
	  timeout = msecs_to_jiffies(10 * 10000);
> 
>         ret = wait_for_completion_timeout(&gpu->event_free,
>                         msecs_to_jiffies(10 * 10000));
                          ^^ use timeout here
>         if (!ret) {
>             dev_err(gpu->dev, "wait_for_completion_timeout failed");
>             err = -EBUSY;
>             goto out;
>         }

If we waited successfully, wait_for_completion_timeout will return the
remaining jiffies, so we can re-initialize the timeout with the return
value to shorten the next wait.

          timeout = ret;
>     }
> 
>     spin_lock_irqsave(&gpu->event_spinlock, flags);
> 
>     for (i = 0; i < nr_events; i++) {
>         int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
> 
>         events[i] = event;
>         set_bit(event, gpu->event_bitmap);
>     }
> 
>     spin_unlock_irqrestore(&gpu->event_spinlock, flags);
> 
> out:

If you end up here you need to return the already acquired completions,
by rolling back your for loop above and do a complete(&gpu->event_free)
for each acquired completion to avoid depleting the completion queue.

>     return err;
> }

Regards,
Lucas

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 02/23] drm/etnaviv: make it possible to allocate multiple events

[Christian Gmeiner]

Hi Lucas,

2017-08-22 10:39 GMT+02:00 Lucas Stach <l.stach@pengutronix.de>:
> Hi Christian,
>
> Am Dienstag, den 22.08.2017, 10:27 +0200 schrieb Christian Gmeiner:
>> Hi Lucas.
>>
>> Thanks for your review - hopefully there will be only one last v3
>> series of that patches.
>
> Yep, I would really like to merge this series. It's been in the making
> for long enough. :)
>
>> 2017-08-08 12:00 GMT+02:00 Lucas Stach <l.stach@pengutronix.de>:
>> > Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
>> >> This makes it possible to allocate multiple events under the event
>> >> spinlock. This change is needed to support 'sync'-points.
>> >>
>> >> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
>> >> ---
>> >>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c | 31 ++++++++++++++++++++-----------
>> >>  1 file changed, 20 insertions(+), 11 deletions(-)
>> >>
>> >> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> >> index fa9c7bd98e9c..ab108b0ed573 100644
>> >> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> >> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> >> @@ -1137,10 +1137,12 @@ int etnaviv_gpu_fence_sync_obj(struct etnaviv_gem_object *etnaviv_obj,
>> >>   * event management:
>> >>   */
>> >>
>> >> -static unsigned int event_alloc(struct etnaviv_gpu *gpu)
>> >> +static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
>> >> +     unsigned int *events)
>> >>  {
>> >>       unsigned long ret, flags;
>> >> -     unsigned int event;
>> >> +     unsigned used, i;
>> >> +     int err = 0;
>> >>
>> >>       ret = wait_for_completion_timeout(&gpu->event_free,
>> >>                                         msecs_to_jiffies(10 * 10000));
>> >
>> > This isn't obvious from the current code, but there are exactly as much
>> > completions in the queue, as there are events. See initialization of the
>> > completions in etnaviv_gpu_init(). This means the event allocation under
>> > spinlock always succeeds if the wait hasn't timed out. To keep this
>> > working you need to wait for the completion nr_event times, probably
>> > changing this to an absolute timeout, so we don't lengthen the timeout
>> > with the number of events.
>> >
>>
>> Makes sense but I not really sure how to best implement an absolute
>> timeout. We could wait
>> absolute timeout / nr_events in each wait_for_completion_timeout(..) call.
>
> I think we should just keep the 10sec timeout regardless of the number
> of events. If we don't acquire the needed events after 10secs, it's
> probably fine to give up.
>

Yeah :)

>> >> @@ -1149,16 +1151,24 @@ static unsigned int event_alloc(struct etnaviv_gpu *gpu)
>> >>
>> >>       spin_lock_irqsave(&gpu->event_spinlock, flags);
>> >>
>> >> -     /* find first free event */
>> >> -     event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
>> >> -     if (event < ETNA_NR_EVENTS)
>> >> +     /* are there enough free events? */
>> >> +     used = bitmap_weight(gpu->event_bitmap, ETNA_NR_EVENTS);
>> >> +     if (used + nr_events > ETNA_NR_EVENTS) {
>> >> +             err = -EBUSY;
>> >> +             goto out;
>> >> +     }
>> >
>> > This isn't necessary if you waited successfully for the completion to
>> > signal nr_event times. The allocation is guaranteed to succeed in that
>> > case. We just want an early return before even locking the spinlock,
>> > giving back the already collected completions if one of the waits timed
>> > out.
>> >
>>
>> Yes - feels more elegant that way.
>>
>> >> +
>> >> +     for (i = 0; i < nr_events; i++) {
>> >> +             int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
>> >> +
>> >> +             events[i] = event;
>> >>               set_bit(event, gpu->event_bitmap);
>> >> -     else
>> >> -             event = ~0U;
>> >> +     }
>> >>
>> >> +out:
>> >>       spin_unlock_irqrestore(&gpu->event_spinlock, flags);
>> >>
>> >> -     return event;
>> >> +     return err;
>> >>  }
>> >>
>> >>  static void event_free(struct etnaviv_gpu *gpu, unsigned int event)
>> >> @@ -1327,10 +1337,9 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>> >>        *
>> >>        */
>> >>
>> >> -     event = event_alloc(gpu);
>> >> -     if (unlikely(event == ~0U)) {
>> >> +     ret = event_alloc(gpu, 1, &event);
>> >> +     if (!ret) {
>> >>               DRM_ERROR("no free event\n");
>> >> -             ret = -EBUSY;
>> >>               goto out_pm_put;
>> >>       }
>> >>
>> >
>> >
>>
>> What about something like this:
>
> A few notes inline.
>
>>
>> static int event_alloc(struct etnaviv_gpu *gpu, unsigned nr_events,
>>     unsigned int *events)
>> {
>>     unsigned long flags;
>>     unsigned i;
>>     int err = 0;
>>
>>     for (i = 0; i < nr_events; i++) {
>>         unsigned long ret;
>           timeout = msecs_to_jiffies(10 * 10000);
>>
>>         ret = wait_for_completion_timeout(&gpu->event_free,
>>                         msecs_to_jiffies(10 * 10000));
>                           ^^ use timeout here
>>         if (!ret) {
>>             dev_err(gpu->dev, "wait_for_completion_timeout failed");
>>             err = -EBUSY;
>>             goto out;
>>         }
>
> If we waited successfully, wait_for_completion_timeout will return the
> remaining jiffies, so we can re-initialize the timeout with the return
> value to shorten the next wait.
>
>           timeout = ret;

ok

>>     }
>>
>>     spin_lock_irqsave(&gpu->event_spinlock, flags);
>>
>>     for (i = 0; i < nr_events; i++) {
>>         int event = find_first_zero_bit(gpu->event_bitmap, ETNA_NR_EVENTS);
>>
>>         events[i] = event;
>>         set_bit(event, gpu->event_bitmap);
>>     }
>>
>>     spin_unlock_irqrestore(&gpu->event_spinlock, flags);
>>
>> out:
>
> If you end up here you need to return the already acquired completions,
> by rolling back your for loop above and do a complete(&gpu->event_free)
> for each acquired completion to avoid depleting the completion queue.
>

Oops.. totally missed that part - time for a coffee.

greets
--
Christian Gmeiner, MSc

https://christian-gmeiner.info
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 10/23] drm/etnaviv: add 'sync point' support

[Christian Gmeiner]

Hi Lucas

2017-08-08 12:34 GMT+02:00 Lucas Stach <l.stach@pengutronix.de>:
> Am Samstag, den 22.07.2017, 11:53 +0200 schrieb Christian Gmeiner:
>> In order to support performance counters in a sane way we need to provide
>> a method to sync the GPU with the CPU. The GPU can process multpile command
>> buffers/events per irq. With the help of a 'sync point' we can trigger an event
>> and stop the GPU/FE immediately. When the CPU is done with is processing it
>> simply needs to restart the FE and the GPU will process the command stream.
>>
>> Changes from v1 -> v2:
>> - process sync point with a work item to keep irq as fast as possible
>>
>> Signed-off-by: Christian Gmeiner <christian.gmeiner@gmail.com>
>> ---
>>  drivers/gpu/drm/etnaviv/etnaviv_buffer.c | 36 ++++++++++++++++++++++++++++++++
>>  drivers/gpu/drm/etnaviv/etnaviv_drv.h    |  1 +
>>  drivers/gpu/drm/etnaviv/etnaviv_gpu.c    | 25 ++++++++++++++++++++++
>>  drivers/gpu/drm/etnaviv/etnaviv_gpu.h    |  6 ++++++
>>  4 files changed, 68 insertions(+)
>>
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
>> index ed9588f36bc9..9e7098e3207f 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_buffer.c
>> @@ -250,6 +250,42 @@ void etnaviv_buffer_end(struct etnaviv_gpu *gpu)
>>       }
>>  }
>>
>> +/* Append a 'sync point' to the ring buffer. */
>> +void etnaviv_sync_point_queue(struct etnaviv_gpu *gpu, unsigned int event)
>> +{
>> +     struct etnaviv_cmdbuf *buffer = gpu->buffer;
>> +     unsigned int waitlink_offset = buffer->user_size - 16;
>> +     u32 dwords, target;
>> +
>> +     /*
>> +      * We need at most 3 dwords in the return target:
>> +      * 1 event + 1 end + 1 wait + 1 link.
>> +      */
>> +     dwords = 4;
>> +     target = etnaviv_buffer_reserve(gpu, buffer, dwords);
>> +
>> +     /* Signal sync point event */
>> +     CMD_LOAD_STATE(buffer, VIVS_GL_EVENT, VIVS_GL_EVENT_EVENT_ID(event) |
>> +                    VIVS_GL_EVENT_FROM_PE);
>> +
>> +     /* Stop the FE to 'pause' the GPU */
>> +     CMD_END(buffer);
>> +
>> +     /* Append waitlink */
>> +     CMD_WAIT(buffer);
>> +     CMD_LINK(buffer, 2, etnaviv_cmdbuf_get_va(buffer) +
>> +                         buffer->user_size - 4);
>> +
>> +     /*
>> +      * Kick off the 'sync point' command by replacing the previous
>> +      * WAIT with a link to the address in the ring buffer.
>> +      */
>> +     etnaviv_buffer_replace_wait(buffer, waitlink_offset,
>> +                                 VIV_FE_LINK_HEADER_OP_LINK |
>> +                                 VIV_FE_LINK_HEADER_PREFETCH(dwords),
>> +                                 target);
>> +}
>> +
>>  /* Append a command buffer to the ring buffer. */
>>  void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, unsigned int event,
>>       struct etnaviv_cmdbuf *cmdbuf)
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.h b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
>> index 058389f93b69..f6cdd694ca51 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_drv.h
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.h
>> @@ -101,6 +101,7 @@ int etnaviv_gem_new_userptr(struct drm_device *dev, struct drm_file *file,
>>  u16 etnaviv_buffer_init(struct etnaviv_gpu *gpu);
>>  u16 etnaviv_buffer_config_mmuv2(struct etnaviv_gpu *gpu, u32 mtlb_addr, u32 safe_addr);
>>  void etnaviv_buffer_end(struct etnaviv_gpu *gpu);
>> +void etnaviv_sync_point_queue(struct etnaviv_gpu *gpu, unsigned int event);
>>  void etnaviv_buffer_queue(struct etnaviv_gpu *gpu, unsigned int event,
>>       struct etnaviv_cmdbuf *cmdbuf);
>>  void etnaviv_validate_init(void);
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> index ce6c869e214f..bc1b96b4c7e9 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.c
>> @@ -25,6 +25,7 @@
>>  #include "etnaviv_gpu.h"
>>  #include "etnaviv_gem.h"
>>  #include "etnaviv_mmu.h"
>> +#include "etnaviv_perfmon.h"
>>  #include "common.xml.h"
>>  #include "state.xml.h"
>>  #include "state_hi.xml.h"
>> @@ -1353,6 +1354,7 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>>       }
>>
>>       gpu->event[event].fence = fence;
>> +     gpu->event[event].sync_point = NULL;
>>       submit->fence = dma_fence_get(fence);
>>       gpu->active_fence = submit->fence->seqno;
>>
>> @@ -1398,6 +1400,23 @@ int etnaviv_gpu_submit(struct etnaviv_gpu *gpu,
>>       return ret;
>>  }
>>
>> +static void etnaviv_process_sync_point(struct etnaviv_gpu *gpu,
>> +     struct etnaviv_event *event)
>> +{
>> +     u32 addr = gpu_read(gpu, VIVS_FE_DMA_ADDRESS);
>> +
>> +     event->sync_point(gpu, event);
>> +     etnaviv_gpu_start_fe(gpu, addr + 2, 2);
>> +}
>> +
>> +static void pmrs_worker(struct work_struct *work)
>> +{
>> +     struct etnaviv_gpu *gpu = container_of(work, struct etnaviv_gpu,
>> +                                            pmrs_work);
>> +
>> +     etnaviv_process_sync_point(gpu, &gpu->event[gpu->pmrs_event]);
>> +}
>> +
>>  /*
>>   * Init/Cleanup:
>>   */
>> @@ -1444,6 +1463,11 @@ static irqreturn_t irq_handler(int irq, void *data)
>>
>>                       dev_dbg(gpu->dev, "event %u\n", event);
>>
>> +                     if (gpu->event[event].sync_point) {
>> +                             gpu->pmrs_event = event;
>> +                             etnaviv_queue_work(gpu->drm, &gpu->pmrs_work);
>
> If the handler is delayed we might handle multiple events per
> invocation, in which case the events might not be in order. E.g. the FE
> stop event might be event 30, while the FE start event might be event 0.
> In that case you would execute the FE start before the FE stop has been
> queued -> not good. You need to make sure that your PMRS events are
> processed in the correct order.
>

I thought about this problem for some time and I do not fully get your
point - sorry.

First there is no FE start event. I am using 'sync' points for pre and
post pmrs points.
A 'sync' point is represented like this in the cmd stream:

| EVENT | FE STOP | WAIT | LINK back to WAIT | - see
etnaviv_sync_point_queue(..) in etnaviv_buffer.c

The restarting of the FE happens in pmrs_worker(..) triggered by EVENT
and the FE starts processing the
cmd stream at "| WAIT | LINK back to WAIT |".

Maybe you can point me into the right direction.

>> +                     }
>> +
>>                       fence = gpu->event[event].fence;
>>                       gpu->event[event].fence = NULL;
>>                       dma_fence_signal(fence);
>> @@ -1647,6 +1671,7 @@ static int etnaviv_gpu_bind(struct device *dev, struct device *master,
>>
>>       INIT_LIST_HEAD(&gpu->active_cmd_list);
>>       INIT_WORK(&gpu->retire_work, retire_worker);
>> +     INIT_WORK(&gpu->pmrs_work, pmrs_worker);
>>       INIT_WORK(&gpu->recover_work, recover_worker);
>>       init_waitqueue_head(&gpu->fence_event);
>>
>> diff --git a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
>> index 70e6590aacdf..7d5f785b0f08 100644
>> --- a/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
>> +++ b/drivers/gpu/drm/etnaviv/etnaviv_gpu.h
>> @@ -89,6 +89,8 @@ struct etnaviv_chip_identity {
>>
>>  struct etnaviv_event {
>>       struct dma_fence *fence;
>> +
>> +     void (*sync_point)(struct etnaviv_gpu *gpu, struct etnaviv_event *event);
>>  };
>>
>>  struct etnaviv_cmdbuf_suballoc;
>> @@ -135,6 +137,10 @@ struct etnaviv_gpu {
>>       /* worker for handling active-list retiring: */
>>       struct work_struct retire_work;
>>
>> +     /* worker for handling performance monitor requests: */
>> +     struct work_struct pmrs_work;
>> +     int pmrs_event;
>> +
>>       void __iomem *mmio;
>>       int irq;
>>
>
>

greets
--
Christian Gmeiner, MSc

https://christian-gmeiner.info
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH V2 10/23] drm/etnaviv: add 'sync point' support

[Lucas Stach]

Am Dienstag, den 22.08.2017, 11:58 +0200 schrieb Christian Gmeiner:
[...]
> >> @@ -1444,6 +1463,11 @@ static irqreturn_t irq_handler(int irq, void *data)
> >>
> >>                       dev_dbg(gpu->dev, "event %u\n", event);
> >>
> >> +                     if (gpu->event[event].sync_point) {
> >> +                             gpu->pmrs_event = event;
> >> +                             etnaviv_queue_work(gpu->drm, &gpu->pmrs_work);
> >
> > If the handler is delayed we might handle multiple events per
> > invocation, in which case the events might not be in order. E.g. the FE
> > stop event might be event 30, while the FE start event might be event 0.
> > In that case you would execute the FE start before the FE stop has been
> > queued -> not good. You need to make sure that your PMRS events are
> > processed in the correct order.
> >
> 
> I thought about this problem for some time and I do not fully get your
> point - sorry.
> 
> First there is no FE start event. I am using 'sync' points for pre and
> post pmrs points.

You are right. I was just about to type up a lengthy explanation of what
I meant, but while thinking it through I realized that my assumptions
where invalid. As both the PRE and POST events stop the FE, the GPU can
never get ahead of the event workers.

Please scratch my earlier comments.

Regards,
Lucas

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel