[PATCH v3 0/4] drm/panfrost: Expose perf counters to userspace

[PATCH v3 0/4] drm/panfrost: Expose perf counters to userspace

[Boris Brezillon]

Hello,

This a new version of the panfrost perfcnt series, this time exposing
this functionality through 2 ioctls instead of the debugfs approach
used in v2.

I also went for Emil's suggestion to expose those ioctls only when
the unstable_iocts unsafe param is set to true. This way, I hope we'll
be able to deprecate those ioctls when the generic solution to expose
global perf counters is out.

Also addressed the problems reported by Rob and Steven.

Regards,

Boris

Changes in v3:
* Expose things through ioctls instead of debugfs (needed for
  per-FD address space that is being worked on by Rob)

Changes in v2:
* Complete rewrite to expose things through debugfs

Boris Brezillon (4):
  drm/panfrost: Move gpu_{write,read}() macros to panfrost_regs.h
  drm/panfrost: Add a module parameter to expose unstable ioctls
  drm/panfrost: Add an helper to check the GPU generation
  drm/panfrost: Expose performance counters through unstable ioctls

 drivers/gpu/drm/panfrost/Makefile           |   3 +-
 drivers/gpu/drm/panfrost/panfrost_device.c  |   8 +
 drivers/gpu/drm/panfrost/panfrost_device.h  |  10 +
 drivers/gpu/drm/panfrost/panfrost_drv.c     |  15 +
 drivers/gpu/drm/panfrost/panfrost_gpu.c     |  10 +-
 drivers/gpu/drm/panfrost/panfrost_perfcnt.c | 329 ++++++++++++++++++++
 drivers/gpu/drm/panfrost/panfrost_perfcnt.h |  18 ++
 drivers/gpu/drm/panfrost/panfrost_regs.h    |  22 ++
 include/uapi/drm/panfrost_drm.h             |  24 ++
 9 files changed, 435 insertions(+), 4 deletions(-)
 create mode 100644 drivers/gpu/drm/panfrost/panfrost_perfcnt.c
 create mode 100644 drivers/gpu/drm/panfrost/panfrost_perfcnt.h

-- 
2.20.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v3 1/4] drm/panfrost: Move gpu_{write, read}() macros to panfrost_regs.h

[Boris Brezillon]

So they can be used from other files.

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes in v3:
- None

Changes in v2:
- None
---
 drivers/gpu/drm/panfrost/panfrost_gpu.c  | 3 ---
 drivers/gpu/drm/panfrost/panfrost_regs.h | 3 +++
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/panfrost/panfrost_gpu.c b/drivers/gpu/drm/panfrost/panfrost_gpu.c
index 58ef25573cda..6e68a100291c 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gpu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gpu.c
@@ -17,9 +17,6 @@
 #include "panfrost_gpu.h"
 #include "panfrost_regs.h"
 
-#define gpu_write(dev, reg, data) writel(data, dev->iomem + reg)
-#define gpu_read(dev, reg) readl(dev->iomem + reg)
-
 static irqreturn_t panfrost_gpu_irq_handler(int irq, void *data)
 {
 	struct panfrost_device *pfdev = data;
diff --git a/drivers/gpu/drm/panfrost/panfrost_regs.h b/drivers/gpu/drm/panfrost/panfrost_regs.h
index 578c5fc2188b..42d08860fd76 100644
--- a/drivers/gpu/drm/panfrost/panfrost_regs.h
+++ b/drivers/gpu/drm/panfrost/panfrost_regs.h
@@ -295,4 +295,7 @@
 #define AS_FAULTSTATUS_ACCESS_TYPE_READ		(0x2 << 8)
 #define AS_FAULTSTATUS_ACCESS_TYPE_WRITE	(0x3 << 8)
 
+#define gpu_write(dev, reg, data) writel(data, dev->iomem + reg)
+#define gpu_read(dev, reg) readl(dev->iomem + reg)
+
 #endif
-- 
2.20.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v3 2/4] drm/panfrost: Add a module parameter to expose unstable ioctls

[Boris Brezillon]

We plan to expose performance counters through 2 driver specific
ioctls until there's a solution to expose them in a generic way.
In order to be able to deprecate those ioctls when this new
infrastructure is in place we add an unsafe module parameter that
will keep those ioctls hidden unless it's set to true (which also
has the effect of tainting the kernel).

All unstable ioctl handlers should use panfrost_unstable_ioctl_check()
to check whether they're supposed to handle the request or reject it
with ENOSYS.

Suggested-by: Emil Velikov <emil.velikov@collabora.com>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Chnages in v3:
* New patch
---
 drivers/gpu/drm/panfrost/panfrost_device.h |  2 ++
 drivers/gpu/drm/panfrost/panfrost_drv.c    | 11 +++++++++++
 2 files changed, 13 insertions(+)

diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
index 8074f221034b..031168f83bd2 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.h
+++ b/drivers/gpu/drm/panfrost/panfrost_device.h
@@ -115,6 +115,8 @@ static inline bool panfrost_model_eq(struct panfrost_device *pfdev, s32 id)
 	return !panfrost_model_cmp(pfdev, id);
 }
 
+int panfrost_unstable_ioctl_check(void);
+
 int panfrost_device_init(struct panfrost_device *pfdev);
 void panfrost_device_fini(struct panfrost_device *pfdev);
 
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index d11e2281dde6..754881ece8d7 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -20,6 +20,9 @@
 #include "panfrost_job.h"
 #include "panfrost_gpu.h"
 
+static bool unstable_ioctls;
+module_param_unsafe(unstable_ioctls, bool, 0600);
+
 static int panfrost_ioctl_get_param(struct drm_device *ddev, void *data, struct drm_file *file)
 {
 	struct drm_panfrost_get_param *param = data;
@@ -297,6 +300,14 @@ static int panfrost_ioctl_get_bo_offset(struct drm_device *dev, void *data,
 	return 0;
 }
 
+int panfrost_unstable_ioctl_check(void)
+{
+	if (!unstable_ioctls)
+		return -ENOSYS;
+
+	return 0;
+}
+
 static int
 panfrost_open(struct drm_device *dev, struct drm_file *file)
 {
-- 
2.20.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v3 3/4] drm/panfrost: Add an helper to check the GPU generation

[Boris Brezillon]

All models with an ID >= 0x1000 are Bifrost GPUs for now (might change
with new gens).

Suggested-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes in v3:
* New patch
---
 drivers/gpu/drm/panfrost/panfrost_device.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
index 031168f83bd2..e86581c4af7b 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.h
+++ b/drivers/gpu/drm/panfrost/panfrost_device.h
@@ -110,6 +110,11 @@ static inline int panfrost_model_cmp(struct panfrost_device *pfdev, s32 id)
 	return match_id - id;
 }
 
+static inline bool panfrost_model_is_bifrost(struct panfrost_device *pfdev)
+{
+	return panfrost_model_cmp(pfdev, 0x1000) >= 0;
+}
+
 static inline bool panfrost_model_eq(struct panfrost_device *pfdev, s32 id)
 {
 	return !panfrost_model_cmp(pfdev, id);
-- 
2.20.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v3 4/4] drm/panfrost: Expose performance counters through unstable ioctls

[Boris Brezillon]

Expose performance counters through 2 driver specific ioctls: one to
enable/disable the perfcnt block, and one to dump the counter values.

There are discussions to expose global performance monitors (those
counters that can't be retrieved on a per-job basis) in a consistent
way, but this is likely to take time to settle on something that works
for various HW/users.
The ioctls are marked unstable so we can get rid of them when the time
comes. We initally went for a debugfs-based interface, but this was
making the transition to per-FD address space more complicated (we need
to specify the namespace the GPU has to use when dumping the perf
counters), hence the decision to switch back to driver specific ioctls
which are passed the FD they operate on and thus will have a dedicated
address space attached to them.

Other than that, the implementation is pretty simple: it basically dumps
all counters and copy the values to a userspace buffer. The parsing is
left to userspace which has to know the specific layout that's used
by the GPU (layout differs on a per-revision basis).

Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>
---
Changes in v3:
* Expose things through ioctls instead of debugs (suggested by Rob)
* Fix the BO size calculation (reported by Steven)
* Drop perfcnt suspend/resume logic (runtime suspend is prevented when
  perfcnt is enabled)
* Disable the perfcnt block in the _fini() path (suggested by Rob)
* Do not mess with the INT reg in _init() (suggested by Rob)
* Provide a way to select both set of counters on Bifrost GPUs
  (suggested by Steven)

Changes in v2:
* Complete rewrite to expose things through debugfs in a simple way
  (no counter layout abstraction and no per-job counter tracking to
   avoid the perf penalty incurred by the extra serialization fence)
---
 drivers/gpu/drm/panfrost/Makefile           |   3 +-
 drivers/gpu/drm/panfrost/panfrost_device.c  |   8 +
 drivers/gpu/drm/panfrost/panfrost_device.h  |   3 +
 drivers/gpu/drm/panfrost/panfrost_drv.c     |   4 +
 drivers/gpu/drm/panfrost/panfrost_gpu.c     |   7 +
 drivers/gpu/drm/panfrost/panfrost_perfcnt.c | 329 ++++++++++++++++++++
 drivers/gpu/drm/panfrost/panfrost_perfcnt.h |  18 ++
 drivers/gpu/drm/panfrost/panfrost_regs.h    |  19 ++
 include/uapi/drm/panfrost_drm.h             |  24 ++
 9 files changed, 414 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/panfrost/panfrost_perfcnt.c
 create mode 100644 drivers/gpu/drm/panfrost/panfrost_perfcnt.h

diff --git a/drivers/gpu/drm/panfrost/Makefile b/drivers/gpu/drm/panfrost/Makefile
index 6de72d13c58f..ecf0864cb515 100644
--- a/drivers/gpu/drm/panfrost/Makefile
+++ b/drivers/gpu/drm/panfrost/Makefile
@@ -7,6 +7,7 @@ panfrost-y := \
 	panfrost_gem.o \
 	panfrost_gpu.o \
 	panfrost_job.o \
-	panfrost_mmu.o
+	panfrost_mmu.o \
+	panfrost_perfcnt.o
 
 obj-$(CONFIG_DRM_PANFROST) += panfrost.o
diff --git a/drivers/gpu/drm/panfrost/panfrost_device.c b/drivers/gpu/drm/panfrost/panfrost_device.c
index ccb8eb2a518c..8a111d7c0200 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.c
+++ b/drivers/gpu/drm/panfrost/panfrost_device.c
@@ -14,6 +14,7 @@
 #include "panfrost_gpu.h"
 #include "panfrost_job.h"
 #include "panfrost_mmu.h"
+#include "panfrost_perfcnt.h"
 
 static int panfrost_reset_init(struct panfrost_device *pfdev)
 {
@@ -171,7 +172,13 @@ int panfrost_device_init(struct panfrost_device *pfdev)
 	pm_runtime_mark_last_busy(pfdev->dev);
 	pm_runtime_put_autosuspend(pfdev->dev);
 
+	err = panfrost_perfcnt_init(pfdev);
+	if (err)
+		goto err_out5;
+
 	return 0;
+err_out5:
+	panfrost_job_fini(pfdev);
 err_out4:
 	panfrost_mmu_fini(pfdev);
 err_out3:
@@ -187,6 +194,7 @@ int panfrost_device_init(struct panfrost_device *pfdev)
 
 void panfrost_device_fini(struct panfrost_device *pfdev)
 {
+	panfrost_perfcnt_fini(pfdev);
 	panfrost_job_fini(pfdev);
 	panfrost_mmu_fini(pfdev);
 	panfrost_gpu_fini(pfdev);
diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
index e86581c4af7b..83cc01cafde1 100644
--- a/drivers/gpu/drm/panfrost/panfrost_device.h
+++ b/drivers/gpu/drm/panfrost/panfrost_device.h
@@ -14,6 +14,7 @@ struct panfrost_device;
 struct panfrost_mmu;
 struct panfrost_job_slot;
 struct panfrost_job;
+struct panfrost_perfcnt;
 
 #define NUM_JOB_SLOTS 3
 
@@ -78,6 +79,8 @@ struct panfrost_device {
 	struct panfrost_job *jobs[NUM_JOB_SLOTS];
 	struct list_head scheduled_jobs;
 
+	struct panfrost_perfcnt *perfcnt;
+
 	struct mutex sched_lock;
 	struct mutex reset_lock;
 
diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
index 754881ece8d7..e34e86a7378a 100644
--- a/drivers/gpu/drm/panfrost/panfrost_drv.c
+++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
@@ -19,6 +19,7 @@
 #include "panfrost_mmu.h"
 #include "panfrost_job.h"
 #include "panfrost_gpu.h"
+#include "panfrost_perfcnt.h"
 
 static bool unstable_ioctls;
 module_param_unsafe(unstable_ioctls, bool, 0600);
@@ -329,6 +330,7 @@ panfrost_postclose(struct drm_device *dev, struct drm_file *file)
 {
 	struct panfrost_file_priv *panfrost_priv = file->driver_priv;
 
+	panfrost_perfcnt_close(panfrost_priv);
 	panfrost_job_close(panfrost_priv);
 
 	kfree(panfrost_priv);
@@ -348,6 +350,8 @@ static const struct drm_ioctl_desc panfrost_drm_driver_ioctls[] = {
 	PANFROST_IOCTL(MMAP_BO,		mmap_bo,	DRM_RENDER_ALLOW),
 	PANFROST_IOCTL(GET_PARAM,	get_param,	DRM_RENDER_ALLOW),
 	PANFROST_IOCTL(GET_BO_OFFSET,	get_bo_offset,	DRM_RENDER_ALLOW),
+	PANFROST_IOCTL(PERFCNT_ENABLE,	perfcnt_enable,	DRM_RENDER_ALLOW),
+	PANFROST_IOCTL(PERFCNT_DUMP,	perfcnt_dump,	DRM_RENDER_ALLOW),
 };
 
 DEFINE_DRM_GEM_SHMEM_FOPS(panfrost_drm_driver_fops);
diff --git a/drivers/gpu/drm/panfrost/panfrost_gpu.c b/drivers/gpu/drm/panfrost/panfrost_gpu.c
index 6e68a100291c..20ab333fc925 100644
--- a/drivers/gpu/drm/panfrost/panfrost_gpu.c
+++ b/drivers/gpu/drm/panfrost/panfrost_gpu.c
@@ -15,6 +15,7 @@
 #include "panfrost_features.h"
 #include "panfrost_issues.h"
 #include "panfrost_gpu.h"
+#include "panfrost_perfcnt.h"
 #include "panfrost_regs.h"
 
 static irqreturn_t panfrost_gpu_irq_handler(int irq, void *data)
@@ -40,6 +41,12 @@ static irqreturn_t panfrost_gpu_irq_handler(int irq, void *data)
 		gpu_write(pfdev, GPU_INT_MASK, 0);
 	}
 
+	if (state & GPU_IRQ_PERFCNT_SAMPLE_COMPLETED)
+		panfrost_perfcnt_sample_done(pfdev);
+
+	if (state & GPU_IRQ_CLEAN_CACHES_COMPLETED)
+		panfrost_perfcnt_clean_cache_done(pfdev);
+
 	gpu_write(pfdev, GPU_INT_CLEAR, state);
 
 	return IRQ_HANDLED;
diff --git a/drivers/gpu/drm/panfrost/panfrost_perfcnt.c b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
new file mode 100644
index 000000000000..f01508a20146
--- /dev/null
+++ b/drivers/gpu/drm/panfrost/panfrost_perfcnt.c
@@ -0,0 +1,329 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright 2019 Collabora Ltd */
+
+#include <drm/drm_file.h>
+#include <drm/drm_gem_shmem_helper.h>
+#include <drm/panfrost_drm.h>
+#include <linux/completion.h>
+#include <linux/iopoll.h>
+#include <linux/pm_runtime.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+
+#include "panfrost_device.h"
+#include "panfrost_features.h"
+#include "panfrost_gem.h"
+#include "panfrost_issues.h"
+#include "panfrost_job.h"
+#include "panfrost_mmu.h"
+#include "panfrost_regs.h"
+
+#define COUNTERS_PER_BLOCK		64
+#define BYTES_PER_COUNTER		4
+#define BLOCKS_PER_COREGROUP		8
+#define V4_SHADERS_PER_COREGROUP	4
+
+struct panfrost_perfcnt {
+	struct panfrost_gem_object *bo;
+	size_t bosize;
+	void *buf;
+	struct panfrost_file_priv *user;
+	struct mutex lock;
+	struct completion dump_comp;
+};
+
+void panfrost_perfcnt_clean_cache_done(struct panfrost_device *pfdev)
+{
+	complete(&pfdev->perfcnt->dump_comp);
+}
+
+void panfrost_perfcnt_sample_done(struct panfrost_device *pfdev)
+{
+	gpu_write(pfdev, GPU_CMD, GPU_CMD_CLEAN_CACHES);
+}
+
+static int panfrost_perfcnt_dump_locked(struct panfrost_device *pfdev)
+{
+	u64 gpuva;
+	int ret;
+
+	reinit_completion(&pfdev->perfcnt->dump_comp);
+	gpuva = pfdev->perfcnt->bo->node.start << PAGE_SHIFT;
+	gpu_write(pfdev, GPU_PERFCNT_BASE_LO, gpuva);
+	gpu_write(pfdev, GPU_PERFCNT_BASE_HI, gpuva >> 32);
+	gpu_write(pfdev, GPU_INT_CLEAR,
+		  GPU_IRQ_CLEAN_CACHES_COMPLETED |
+		  GPU_IRQ_PERFCNT_SAMPLE_COMPLETED);
+	gpu_write(pfdev, GPU_CMD, GPU_CMD_PERFCNT_SAMPLE);
+	ret = wait_for_completion_interruptible_timeout(&pfdev->perfcnt->dump_comp,
+							msecs_to_jiffies(1000));
+	if (!ret)
+		ret = -ETIMEDOUT;
+	else if (ret > 0)
+		ret = 0;
+
+	return ret;
+}
+
+static int panfrost_perfcnt_enable_locked(struct panfrost_device *pfdev,
+					  struct panfrost_file_priv *user,
+					  unsigned int counterset)
+{
+	struct panfrost_perfcnt *perfcnt = pfdev->perfcnt;
+	struct drm_gem_shmem_object *bo;
+	u32 cfg;
+	int ret;
+
+	if (user == perfcnt->user)
+		return 0;
+	else if (perfcnt->user)
+		return -EBUSY;
+
+	ret = pm_runtime_get_sync(pfdev->dev);
+	if (ret < 0)
+		return ret;
+
+	bo = drm_gem_shmem_create(pfdev->ddev, perfcnt->bosize);
+	if (IS_ERR(bo))
+		return PTR_ERR(bo);
+
+	perfcnt->bo = to_panfrost_bo(&bo->base);
+
+	/* Map the perfcnt buf in the address space attached to file_priv. */
+	ret = panfrost_mmu_map(perfcnt->bo);
+	if (ret)
+		goto err_put_bo;
+
+	perfcnt->buf = drm_gem_vmap(&bo->base);
+	if (IS_ERR(perfcnt->buf)) {
+		ret = PTR_ERR(perfcnt->buf);
+		goto err_put_bo;
+	}
+
+	/*
+	 * Invalidate the cache and clear the counters to start from a fresh
+	 * state.
+	 */
+	reinit_completion(&pfdev->perfcnt->dump_comp);
+	gpu_write(pfdev, GPU_INT_CLEAR,
+		  GPU_IRQ_CLEAN_CACHES_COMPLETED |
+		  GPU_IRQ_PERFCNT_SAMPLE_COMPLETED);
+	gpu_write(pfdev, GPU_CMD, GPU_CMD_PERFCNT_CLEAR);
+	gpu_write(pfdev, GPU_CMD, GPU_CMD_CLEAN_INV_CACHES);
+	ret = wait_for_completion_timeout(&pfdev->perfcnt->dump_comp,
+					  msecs_to_jiffies(1000));
+	if (!ret) {
+		ret = -ETIMEDOUT;
+		goto err_vunmap;
+	}
+
+	perfcnt->user = user;
+
+	/*
+	 * Always use address space 0 for now.
+	 * FIXME: this needs to be updated when we start using different
+	 * address space.
+	 */
+	cfg = GPU_PERFCNT_CFG_AS(0) |
+	      GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_MANUAL);
+
+	/*
+	 * Bifrost GPUs have 2 set of counters, but we're only interested by
+	 * the first one for now.
+	 */
+	if (panfrost_model_is_bifrost(pfdev))
+		cfg |= GPU_PERFCNT_CFG_SETSEL(counterset);
+
+	gpu_write(pfdev, GPU_PRFCNT_JM_EN, 0xffffffff);
+	gpu_write(pfdev, GPU_PRFCNT_SHADER_EN, 0xffffffff);
+	gpu_write(pfdev, GPU_PRFCNT_MMU_L2_EN, 0xffffffff);
+
+	/*
+	 * Due to PRLAM-8186 we need to disable the Tiler before we enable HW
+	 * counters.
+	 */
+	if (panfrost_has_hw_issue(pfdev, HW_ISSUE_8186))
+		gpu_write(pfdev, GPU_PRFCNT_TILER_EN, 0);
+	else
+		gpu_write(pfdev, GPU_PRFCNT_TILER_EN, 0xffffffff);
+
+	gpu_write(pfdev, GPU_PERFCNT_CFG, cfg);
+
+	if (panfrost_has_hw_issue(pfdev, HW_ISSUE_8186))
+		gpu_write(pfdev, GPU_PRFCNT_TILER_EN, 0xffffffff);
+
+	return 0;
+
+err_vunmap:
+	drm_gem_vunmap(&perfcnt->bo->base.base, perfcnt->buf);
+err_put_bo:
+	drm_gem_object_put_unlocked(&bo->base);
+	return ret;
+}
+
+static int panfrost_perfcnt_disable_locked(struct panfrost_device *pfdev,
+					   struct panfrost_file_priv *user)
+{
+	struct panfrost_perfcnt *perfcnt = pfdev->perfcnt;
+
+	if (user != perfcnt->user)
+		return -EINVAL;
+
+	gpu_write(pfdev, GPU_PRFCNT_JM_EN, 0x0);
+	gpu_write(pfdev, GPU_PRFCNT_SHADER_EN, 0x0);
+	gpu_write(pfdev, GPU_PRFCNT_MMU_L2_EN, 0x0);
+	gpu_write(pfdev, GPU_PRFCNT_TILER_EN, 0);
+	gpu_write(pfdev, GPU_PERFCNT_CFG,
+		  GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_OFF));
+
+	perfcnt->user = NULL;
+	drm_gem_vunmap(&perfcnt->bo->base.base, perfcnt->buf);
+	perfcnt->buf = NULL;
+	drm_gem_object_put_unlocked(&perfcnt->bo->base.base);
+	perfcnt->bo = NULL;
+	pm_runtime_mark_last_busy(pfdev->dev);
+	pm_runtime_put_autosuspend(pfdev->dev);
+
+	return 0;
+}
+
+int panfrost_ioctl_perfcnt_enable(struct drm_device *dev, void *data,
+				  struct drm_file *file_priv)
+{
+	struct panfrost_file_priv *pfile = file_priv->driver_priv;
+	struct panfrost_device *pfdev = dev->dev_private;
+	struct panfrost_perfcnt *perfcnt = pfdev->perfcnt;
+	struct drm_panfrost_perfcnt_enable *req = data;
+	int ret;
+
+	ret = panfrost_unstable_ioctl_check();
+	if (ret)
+		return ret;
+
+	/* Only Bifrost GPUs have 2 set of counters. */
+	if (req->counterset > (panfrost_model_is_bifrost(pfdev) ? 1 : 0))
+		return -EINVAL;
+
+	mutex_lock(&perfcnt->lock);
+	if (req->enable)
+		ret = panfrost_perfcnt_enable_locked(pfdev, pfile,
+						     req->counterset);
+	else
+		ret = panfrost_perfcnt_disable_locked(pfdev, pfile);
+	mutex_unlock(&perfcnt->lock);
+
+	return ret;
+}
+
+int panfrost_ioctl_perfcnt_dump(struct drm_device *dev, void *data,
+				struct drm_file *file_priv)
+{
+	struct panfrost_device *pfdev = dev->dev_private;
+	struct panfrost_perfcnt *perfcnt = pfdev->perfcnt;
+	struct drm_panfrost_perfcnt_dump *req = data;
+	void __user *user_ptr = (void __user *)req->buf_ptr;
+	int ret;
+
+	ret = panfrost_unstable_ioctl_check();
+	if (ret)
+		return ret;
+
+	mutex_lock(&perfcnt->lock);
+	if (perfcnt->user != file_priv->driver_priv) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	ret = panfrost_perfcnt_dump_locked(pfdev);
+	if (ret)
+		goto out;
+
+	if (copy_to_user(user_ptr, perfcnt->buf, perfcnt->bosize))
+		ret = -EFAULT;
+
+out:
+	mutex_unlock(&perfcnt->lock);
+
+	return ret;
+}
+
+void panfrost_perfcnt_close(struct panfrost_file_priv *pfile)
+{
+	struct panfrost_device *pfdev = pfile->pfdev;
+	struct panfrost_perfcnt *perfcnt = pfdev->perfcnt;
+
+	pm_runtime_get_sync(pfdev->dev);
+	mutex_lock(&perfcnt->lock);
+	if (perfcnt->user == pfile)
+		panfrost_perfcnt_disable_locked(pfdev, pfile);
+	mutex_unlock(&perfcnt->lock);
+	pm_runtime_mark_last_busy(pfdev->dev);
+	pm_runtime_put_autosuspend(pfdev->dev);
+}
+
+int panfrost_perfcnt_init(struct panfrost_device *pfdev)
+{
+	struct panfrost_perfcnt *perfcnt;
+	size_t size;
+
+	if (panfrost_has_hw_feature(pfdev, HW_FEATURE_V4)) {
+		unsigned int ncoregroups;
+
+		ncoregroups = hweight64(pfdev->features.l2_present);
+		size = ncoregroups * BLOCKS_PER_COREGROUP *
+		       COUNTERS_PER_BLOCK * BYTES_PER_COUNTER;
+	} else {
+		unsigned int nl2c, ncores;
+
+		/*
+		 * TODO: define a macro to extract the number of l2 caches from
+		 * mem_features.
+		 */
+		nl2c = ((pfdev->features.mem_features >> 8) & GENMASK(3, 0)) + 1;
+
+		/*
+		 * shader_present might be sparse, but the counters layout
+		 * forces to dump unused regions too, hence the fls64() call
+		 * instead of hweight64().
+		 */
+		ncores = fls64(pfdev->features.shader_present);
+
+		/*
+		 * There's always one JM and one Tiler block, hence the '+ 2'
+		 * here.
+		 */
+		size = (nl2c + ncores + 2) *
+		       COUNTERS_PER_BLOCK * BYTES_PER_COUNTER;
+	}
+
+	perfcnt = devm_kzalloc(pfdev->dev, sizeof(*perfcnt), GFP_KERNEL);
+	if (!perfcnt)
+		return -ENOMEM;
+
+	perfcnt->bosize = size;
+
+	/* Start with everything disabled. */
+	gpu_write(pfdev, GPU_PERFCNT_CFG,
+		  GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_OFF));
+	gpu_write(pfdev, GPU_PRFCNT_JM_EN, 0);
+	gpu_write(pfdev, GPU_PRFCNT_SHADER_EN, 0);
+	gpu_write(pfdev, GPU_PRFCNT_MMU_L2_EN, 0);
+	gpu_write(pfdev, GPU_PRFCNT_TILER_EN, 0);
+
+	init_completion(&perfcnt->dump_comp);
+	mutex_init(&perfcnt->lock);
+	pfdev->perfcnt = perfcnt;
+
+	return 0;
+}
+
+void panfrost_perfcnt_fini(struct panfrost_device *pfdev)
+{
+	/* Disable everything before leaving. */
+	gpu_write(pfdev, GPU_PERFCNT_CFG,
+		  GPU_PERFCNT_CFG_MODE(GPU_PERFCNT_CFG_MODE_OFF));
+	gpu_write(pfdev, GPU_PRFCNT_JM_EN, 0);
+	gpu_write(pfdev, GPU_PRFCNT_SHADER_EN, 0);
+	gpu_write(pfdev, GPU_PRFCNT_MMU_L2_EN, 0);
+	gpu_write(pfdev, GPU_PRFCNT_TILER_EN, 0);
+}
diff --git a/drivers/gpu/drm/panfrost/panfrost_perfcnt.h b/drivers/gpu/drm/panfrost/panfrost_perfcnt.h
new file mode 100644
index 000000000000..13b8fdaa1b43
--- /dev/null
+++ b/drivers/gpu/drm/panfrost/panfrost_perfcnt.h
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright 2019 Collabora Ltd */
+#ifndef __PANFROST_PERFCNT_H__
+#define __PANFROST_PERFCNT_H__
+
+#include "panfrost_device.h"
+
+void panfrost_perfcnt_sample_done(struct panfrost_device *pfdev);
+void panfrost_perfcnt_clean_cache_done(struct panfrost_device *pfdev);
+int panfrost_perfcnt_init(struct panfrost_device *pfdev);
+void panfrost_perfcnt_fini(struct panfrost_device *pfdev);
+void panfrost_perfcnt_close(struct panfrost_file_priv *pfile);
+int panfrost_ioctl_perfcnt_enable(struct drm_device *dev, void *data,
+				  struct drm_file *file_priv);
+int panfrost_ioctl_perfcnt_dump(struct drm_device *dev, void *data,
+				struct drm_file *file_priv);
+
+#endif
diff --git a/drivers/gpu/drm/panfrost/panfrost_regs.h b/drivers/gpu/drm/panfrost/panfrost_regs.h
index 42d08860fd76..ea38ac60581c 100644
--- a/drivers/gpu/drm/panfrost/panfrost_regs.h
+++ b/drivers/gpu/drm/panfrost/panfrost_regs.h
@@ -44,12 +44,31 @@
 	 GPU_IRQ_MULTIPLE_FAULT)
 #define GPU_CMD				0x30
 #define   GPU_CMD_SOFT_RESET		0x01
+#define   GPU_CMD_PERFCNT_CLEAR		0x03
+#define   GPU_CMD_PERFCNT_SAMPLE	0x04
+#define   GPU_CMD_CLEAN_CACHES		0x07
+#define   GPU_CMD_CLEAN_INV_CACHES	0x08
 #define GPU_STATUS			0x34
+#define   GPU_STATUS_PRFCNT_ACTIVE	BIT(2)
 #define GPU_LATEST_FLUSH_ID		0x38
 #define GPU_FAULT_STATUS		0x3C
 #define GPU_FAULT_ADDRESS_LO		0x40
 #define GPU_FAULT_ADDRESS_HI		0x44
 
+#define GPU_PERFCNT_BASE_LO		0x60
+#define GPU_PERFCNT_BASE_HI		0x64
+#define GPU_PERFCNT_CFG			0x68
+#define   GPU_PERFCNT_CFG_MODE(x)	(x)
+#define   GPU_PERFCNT_CFG_MODE_OFF	0
+#define   GPU_PERFCNT_CFG_MODE_MANUAL	1
+#define   GPU_PERFCNT_CFG_MODE_TILE	2
+#define   GPU_PERFCNT_CFG_AS(x)		((x) << 4)
+#define   GPU_PERFCNT_CFG_SETSEL(x)	((x) << 8)
+#define GPU_PRFCNT_JM_EN		0x6c
+#define GPU_PRFCNT_SHADER_EN		0x70
+#define GPU_PRFCNT_TILER_EN		0x74
+#define GPU_PRFCNT_MMU_L2_EN		0x7c
+
 #define GPU_THREAD_MAX_THREADS		0x0A0	/* (RO) Maximum number of threads per core */
 #define GPU_THREAD_MAX_WORKGROUP_SIZE	0x0A4	/* (RO) Maximum workgroup size */
 #define GPU_THREAD_MAX_BARRIER_SIZE	0x0A8	/* (RO) Maximum threads waiting at a barrier */
diff --git a/include/uapi/drm/panfrost_drm.h b/include/uapi/drm/panfrost_drm.h
index a52e0283b90d..b5d370638846 100644
--- a/include/uapi/drm/panfrost_drm.h
+++ b/include/uapi/drm/panfrost_drm.h
@@ -18,6 +18,8 @@ extern "C" {
 #define DRM_PANFROST_MMAP_BO			0x03
 #define DRM_PANFROST_GET_PARAM			0x04
 #define DRM_PANFROST_GET_BO_OFFSET		0x05
+#define DRM_PANFROST_PERFCNT_ENABLE		0x06
+#define DRM_PANFROST_PERFCNT_DUMP		0x07
 
 #define DRM_IOCTL_PANFROST_SUBMIT		DRM_IOW(DRM_COMMAND_BASE + DRM_PANFROST_SUBMIT, struct drm_panfrost_submit)
 #define DRM_IOCTL_PANFROST_WAIT_BO		DRM_IOW(DRM_COMMAND_BASE + DRM_PANFROST_WAIT_BO, struct drm_panfrost_wait_bo)
@@ -26,6 +28,15 @@ extern "C" {
 #define DRM_IOCTL_PANFROST_GET_PARAM		DRM_IOWR(DRM_COMMAND_BASE + DRM_PANFROST_GET_PARAM, struct drm_panfrost_get_param)
 #define DRM_IOCTL_PANFROST_GET_BO_OFFSET	DRM_IOWR(DRM_COMMAND_BASE + DRM_PANFROST_GET_BO_OFFSET, struct drm_panfrost_get_bo_offset)
 
+/*
+ * Unstable ioctl(s): only exposed when the unsafe unstable_ioctls module
+ * param is set to true.
+ * All these ioctl(s) are subject to deprecation, so please don't rely on
+ * them for anything but debugging purpose.
+ */
+#define DRM_IOCTL_PANFROST_PERFCNT_ENABLE	DRM_IOW(DRM_COMMAND_BASE + DRM_PANFROST_PERFCNT_ENABLE, struct drm_panfrost_perfcnt_enable)
+#define DRM_IOCTL_PANFROST_PERFCNT_DUMP		DRM_IOW(DRM_COMMAND_BASE + DRM_PANFROST_PERFCNT_DUMP, struct drm_panfrost_perfcnt_dump)
+
 #define PANFROST_JD_REQ_FS (1 << 0)
 /**
  * struct drm_panfrost_submit - ioctl argument for submitting commands to the 3D
@@ -135,6 +146,19 @@ struct drm_panfrost_get_bo_offset {
 	__u64 offset;
 };
 
+struct drm_panfrost_perfcnt_enable {
+	__u32 enable;
+	/*
+	 * On bifrost we have 2 sets of counters, this parameter defines the
+	 * one to track.
+	 */
+	__u32 counterset;
+};
+
+struct drm_panfrost_perfcnt_dump {
+	__u64 buf_ptr;
+};
+
 #if defined(__cplusplus)
 }
 #endif
-- 
2.20.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v3 0/4] drm/panfrost: Expose perf counters to userspace

[Alyssa Rosenzweig]

Woohoo! Patches 1-3 are R-b; patch 4 is A-b. Exciting progress! Hoping
to hear what Rob and Steven think :)
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v3 4/4] drm/panfrost: Expose performance counters through unstable ioctls

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 2321 bytes --]

Hi Boris,

I love your patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on v5.2-rc2 next-20190530]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Boris-Brezillon/drm-panfrost-Expose-perf-counters-to-userspace/20190531-020136
config: arm-allmodconfig (attached as .config)
compiler: arm-linux-gnueabi-gcc (GCC) 7.4.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.4.0 make.cross ARCH=arm 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All warnings (new ones prefixed by >>):

   drivers/gpu//drm/panfrost/panfrost_perfcnt.c: In function 'panfrost_ioctl_perfcnt_dump':
>> drivers/gpu//drm/panfrost/panfrost_perfcnt.c:224:26: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast]
     void __user *user_ptr = (void __user *)req->buf_ptr;
                             ^

vim +224 drivers/gpu//drm/panfrost/panfrost_perfcnt.c

   217	
   218	int panfrost_ioctl_perfcnt_dump(struct drm_device *dev, void *data,
   219					struct drm_file *file_priv)
   220	{
   221		struct panfrost_device *pfdev = dev->dev_private;
   222		struct panfrost_perfcnt *perfcnt = pfdev->perfcnt;
   223		struct drm_panfrost_perfcnt_dump *req = data;
 > 224		void __user *user_ptr = (void __user *)req->buf_ptr;
   225		int ret;
   226	
   227		ret = panfrost_unstable_ioctl_check();
   228		if (ret)
   229			return ret;
   230	
   231		mutex_lock(&perfcnt->lock);
   232		if (perfcnt->user != file_priv->driver_priv) {
   233			ret = -EINVAL;
   234			goto out;
   235		}
   236	
   237		ret = panfrost_perfcnt_dump_locked(pfdev);
   238		if (ret)
   239			goto out;
   240	
   241		if (copy_to_user(user_ptr, perfcnt->buf, perfcnt->bosize))
   242			ret = -EFAULT;
   243	
   244	out:
   245		mutex_unlock(&perfcnt->lock);
   246	
   247		return ret;
   248	}
   249	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 71023 bytes --]

[-- Attachment #3: Type: text/plain, Size: 159 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v3 2/4] drm/panfrost: Add a module parameter to expose unstable ioctls

[Rob Herring]

+Daniel and David

On Wed, May 29, 2019 at 3:52 AM Boris Brezillon
<boris.brezillon@collabora.com> wrote:
>
> We plan to expose performance counters through 2 driver specific
> ioctls until there's a solution to expose them in a generic way.
> In order to be able to deprecate those ioctls when this new
> infrastructure is in place we add an unsafe module parameter that
> will keep those ioctls hidden unless it's set to true (which also
> has the effect of tainting the kernel).
>
> All unstable ioctl handlers should use panfrost_unstable_ioctl_check()
> to check whether they're supposed to handle the request or reject it
> with ENOSYS.
>
> Suggested-by: Emil Velikov <emil.velikov@collabora.com>
> Signed-off-by: Boris Brezillon <boris.brezillon@collabora.com>

Daniel, David, Any issues with this approach for an unstable interface?

> ---
> Chnages in v3:
> * New patch
> ---
>  drivers/gpu/drm/panfrost/panfrost_device.h |  2 ++
>  drivers/gpu/drm/panfrost/panfrost_drv.c    | 11 +++++++++++
>  2 files changed, 13 insertions(+)
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_device.h b/drivers/gpu/drm/panfrost/panfrost_device.h
> index 8074f221034b..031168f83bd2 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_device.h
> +++ b/drivers/gpu/drm/panfrost/panfrost_device.h
> @@ -115,6 +115,8 @@ static inline bool panfrost_model_eq(struct panfrost_device *pfdev, s32 id)
>         return !panfrost_model_cmp(pfdev, id);
>  }
>
> +int panfrost_unstable_ioctl_check(void);
> +
>  int panfrost_device_init(struct panfrost_device *pfdev);
>  void panfrost_device_fini(struct panfrost_device *pfdev);
>
> diff --git a/drivers/gpu/drm/panfrost/panfrost_drv.c b/drivers/gpu/drm/panfrost/panfrost_drv.c
> index d11e2281dde6..754881ece8d7 100644
> --- a/drivers/gpu/drm/panfrost/panfrost_drv.c
> +++ b/drivers/gpu/drm/panfrost/panfrost_drv.c
> @@ -20,6 +20,9 @@
>  #include "panfrost_job.h"
>  #include "panfrost_gpu.h"
>
> +static bool unstable_ioctls;
> +module_param_unsafe(unstable_ioctls, bool, 0600);
> +
>  static int panfrost_ioctl_get_param(struct drm_device *ddev, void *data, struct drm_file *file)
>  {
>         struct drm_panfrost_get_param *param = data;
> @@ -297,6 +300,14 @@ static int panfrost_ioctl_get_bo_offset(struct drm_device *dev, void *data,
>         return 0;
>  }
>
> +int panfrost_unstable_ioctl_check(void)
> +{
> +       if (!unstable_ioctls)
> +               return -ENOSYS;
> +
> +       return 0;
> +}
> +
>  static int
>  panfrost_open(struct drm_device *dev, struct drm_file *file)
>  {
> --
> 2.20.1
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v3 0/4] drm/panfrost: Expose perf counters to userspace

[Rob Herring]

On Wed, May 29, 2019 at 9:16 AM Alyssa Rosenzweig <alyssa@rosenzweig.io> wrote:
>
> Woohoo! Patches 1-3 are R-b; patch 4 is A-b. Exciting progress! Hoping
> to hear what Rob and Steven think :)

All looks fine to me, but there's a kbuild error on patch 4 that needs
to be fixed.

Rob
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v3 0/4] drm/panfrost: Expose perf counters to userspace

[Boris Brezillon]

On Fri, 14 Jun 2019 09:12:39 -0600
Rob Herring <robh+dt@kernel.org> wrote:

> On Wed, May 29, 2019 at 9:16 AM Alyssa Rosenzweig <alyssa@rosenzweig.io> wrote:
> >
> > Woohoo! Patches 1-3 are R-b; patch 4 is A-b. Exciting progress! Hoping
> > to hear what Rob and Steven think :)  
> 
> All looks fine to me, but there's a kbuild error on patch 4 that needs
> to be fixed.

Yes, missing uintptr_t cast. I'll send a new version with that fixed.

Thanks,

Boris
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel