[PATCH 0/5] drm/lima: add error debug functionality

[PATCH 0/5] drm/lima: add error debug functionality

[Qiang Yu]

Save task error state when it fail and export to user by
sysfs as a binary file which can be dumped and replayed
by lima_dump tool at:
https://gitlab.freedesktop.org/lima/lima_dump

Qiang Yu (5):
  drm/lima: save process info for debug usage
  drm/lima: add max_error_tasks module parameter
  drm/lima: save task info dump when task fail
  drm/lima: add error sysfs to export error task dump
  drm/lima: add LIMA_BO_FLAG_FORCE_VA

 drivers/gpu/drm/lima/lima_ctx.c    |   3 +
 drivers/gpu/drm/lima/lima_ctx.h    |   5 ++
 drivers/gpu/drm/lima/lima_device.c |  13 +++
 drivers/gpu/drm/lima/lima_device.h |   8 ++
 drivers/gpu/drm/lima/lima_drv.c    | 123 +++++++++++++++++++++++++--
 drivers/gpu/drm/lima/lima_drv.h    |   1 +
 drivers/gpu/drm/lima/lima_dump.h   |  77 +++++++++++++++++
 drivers/gpu/drm/lima/lima_gem.c    |   7 +-
 drivers/gpu/drm/lima/lima_gem.h    |   4 +-
 drivers/gpu/drm/lima/lima_sched.c  | 128 +++++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h  |   7 ++
 drivers/gpu/drm/lima/lima_vm.c     |  13 ++-
 include/uapi/drm/lima_drm.h        |   9 +-
 13 files changed, 385 insertions(+), 13 deletions(-)
 create mode 100644 drivers/gpu/drm/lima/lima_dump.h

-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 1/5] drm/lima: save process info for debug usage

[Qiang Yu]

When task fail, we can find its process with this information.

Signed-off-by: Qiang Yu <yuq825@gmail.com>
---
 drivers/gpu/drm/lima/lima_ctx.c | 3 +++
 drivers/gpu/drm/lima/lima_ctx.h | 5 +++++
 2 files changed, 8 insertions(+)

diff --git a/drivers/gpu/drm/lima/lima_ctx.c b/drivers/gpu/drm/lima/lima_ctx.c
index 22fff6caa961..891d5cd5019a 100644
--- a/drivers/gpu/drm/lima/lima_ctx.c
+++ b/drivers/gpu/drm/lima/lima_ctx.c
@@ -27,6 +27,9 @@ int lima_ctx_create(struct lima_device *dev, struct lima_ctx_mgr *mgr, u32 *id)
 	if (err < 0)
 		goto err_out0;
 
+	ctx->pid = task_pid_nr(current);
+	get_task_comm(ctx->pname, current);
+
 	return 0;
 
 err_out0:
diff --git a/drivers/gpu/drm/lima/lima_ctx.h b/drivers/gpu/drm/lima/lima_ctx.h
index 6154e5c9bfe4..74e2be09090f 100644
--- a/drivers/gpu/drm/lima/lima_ctx.h
+++ b/drivers/gpu/drm/lima/lima_ctx.h
@@ -5,6 +5,7 @@
 #define __LIMA_CTX_H__
 
 #include <linux/xarray.h>
+#include <linux/sched.h>
 
 #include "lima_device.h"
 
@@ -13,6 +14,10 @@ struct lima_ctx {
 	struct lima_device *dev;
 	struct lima_sched_context context[lima_pipe_num];
 	atomic_t guilty;
+
+	/* debug info */
+	char pname[TASK_COMM_LEN];
+	pid_t pid;
 };
 
 struct lima_ctx_mgr {
-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 2/5] drm/lima: add max_error_tasks module parameter

[Qiang Yu]

Limit error tasks to save.

Signed-off-by: Qiang Yu <yuq825@gmail.com>
---
 drivers/gpu/drm/lima/lima_drv.c | 4 ++++
 drivers/gpu/drm/lima/lima_drv.h | 1 +
 2 files changed, 5 insertions(+)

diff --git a/drivers/gpu/drm/lima/lima_drv.c b/drivers/gpu/drm/lima/lima_drv.c
index 2daac64d8955..e235d4545b6c 100644
--- a/drivers/gpu/drm/lima/lima_drv.c
+++ b/drivers/gpu/drm/lima/lima_drv.c
@@ -16,6 +16,7 @@
 
 int lima_sched_timeout_ms;
 uint lima_heap_init_nr_pages = 8;
+uint lima_max_error_tasks;
 
 MODULE_PARM_DESC(sched_timeout_ms, "task run timeout in ms");
 module_param_named(sched_timeout_ms, lima_sched_timeout_ms, int, 0444);
@@ -23,6 +24,9 @@ module_param_named(sched_timeout_ms, lima_sched_timeout_ms, int, 0444);
 MODULE_PARM_DESC(heap_init_nr_pages, "heap buffer init number of pages");
 module_param_named(heap_init_nr_pages, lima_heap_init_nr_pages, uint, 0444);
 
+MODULE_PARM_DESC(max_error_tasks, "max number of error tasks to save");
+module_param_named(max_error_tasks, lima_max_error_tasks, uint, 0644);
+
 static int lima_ioctl_get_param(struct drm_device *dev, void *data, struct drm_file *file)
 {
 	struct drm_lima_get_param *args = data;
diff --git a/drivers/gpu/drm/lima/lima_drv.h b/drivers/gpu/drm/lima/lima_drv.h
index f492ecc6a5d9..fdbd4077c768 100644
--- a/drivers/gpu/drm/lima/lima_drv.h
+++ b/drivers/gpu/drm/lima/lima_drv.h
@@ -10,6 +10,7 @@
 
 extern int lima_sched_timeout_ms;
 extern uint lima_heap_init_nr_pages;
+extern uint lima_max_error_tasks;
 
 struct lima_vm;
 struct lima_bo;
-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 3/5] drm/lima: save task info dump when task fail

[Qiang Yu]

Save all information to start a task which can be exported to user
for debug usage. Dump file data format is specified in lima_dump.h

Signed-off-by: Qiang Yu <yuq825@gmail.com>
---
 drivers/gpu/drm/lima/lima_device.c |  13 +++
 drivers/gpu/drm/lima/lima_device.h |   8 ++
 drivers/gpu/drm/lima/lima_dump.h   |  77 +++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c  | 128 +++++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h  |   7 ++
 5 files changed, 233 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/lima_dump.h

diff --git a/drivers/gpu/drm/lima/lima_device.c b/drivers/gpu/drm/lima/lima_device.c
index 19829b543024..42a00171fea5 100644
--- a/drivers/gpu/drm/lima/lima_device.c
+++ b/drivers/gpu/drm/lima/lima_device.c
@@ -344,6 +344,12 @@ int lima_device_init(struct lima_device *ldev)
 	if (err)
 		goto err_out5;
 
+	ldev->dump.magic = LIMA_DUMP_MAGIC;
+	ldev->dump.version_major = LIMA_DUMP_MAJOR;
+	ldev->dump.version_minor = LIMA_DUMP_MINOR;
+	INIT_LIST_HEAD(&ldev->error_task_list);
+	mutex_init(&ldev->error_task_list_lock);
+
 	dev_info(ldev->dev, "bus rate = %lu\n", clk_get_rate(ldev->clk_bus));
 	dev_info(ldev->dev, "mod rate = %lu", clk_get_rate(ldev->clk_gpu));
 
@@ -370,6 +376,13 @@ int lima_device_init(struct lima_device *ldev)
 void lima_device_fini(struct lima_device *ldev)
 {
 	int i;
+	struct lima_sched_error_task *et, *tmp;
+
+	list_for_each_entry_safe(et, tmp, &ldev->error_task_list, list) {
+		list_del(&et->list);
+		kvfree(et);
+	}
+	mutex_destroy(&ldev->error_task_list_lock);
 
 	lima_fini_pp_pipe(ldev);
 	lima_fini_gp_pipe(ldev);
diff --git a/drivers/gpu/drm/lima/lima_device.h b/drivers/gpu/drm/lima/lima_device.h
index 31158d86271c..f17173f47f26 100644
--- a/drivers/gpu/drm/lima/lima_device.h
+++ b/drivers/gpu/drm/lima/lima_device.h
@@ -6,8 +6,11 @@
 
 #include <drm/drm_device.h>
 #include <linux/delay.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
 
 #include "lima_sched.h"
+#include "lima_dump.h"
 
 enum lima_gpu_id {
 	lima_gpu_mali400 = 0,
@@ -94,6 +97,11 @@ struct lima_device {
 
 	u32 *dlbu_cpu;
 	dma_addr_t dlbu_dma;
+
+	/* debug info */
+	struct lima_dump_head dump;
+	struct list_head error_task_list;
+	struct mutex error_task_list_lock;
 };
 
 static inline struct lima_device *
diff --git a/drivers/gpu/drm/lima/lima_dump.h b/drivers/gpu/drm/lima/lima_dump.h
new file mode 100644
index 000000000000..ca243d99c51b
--- /dev/null
+++ b/drivers/gpu/drm/lima/lima_dump.h
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2020 Qiang Yu <yuq825@gmail.com> */
+
+#ifndef __LIMA_DUMP_H__
+#define __LIMA_DUMP_H__
+
+#include <linux/types.h>
+
+/**
+ * dump file format for all the information to start a lima task
+ *
+ * top level format
+ * | magic code "LIMA" | format version | num tasks | data size |
+ * | reserved | reserved | reserved | reserved |
+ * | task 1 ID | task 1 size | num chunks | reserved | task 1 data |
+ * | task 2 ID | task 2 size | num chunks | reserved | task 2 data |
+ * ...
+ *
+ * task data format
+ * | chunk 1 ID | chunk 1 size | reserved | reserved | chunk 1 data |
+ * | chunk 2 ID | chunk 2 size | reserved | reserved | chunk 2 data |
+ * ...
+ *
+ */
+
+#define LIMA_DUMP_MAJOR 1
+#define LIMA_DUMP_MINOR 0
+
+#define LIMA_DUMP_MAGIC 0x414d494c
+
+struct lima_dump_head {
+	__u32 magic;
+	__u16 version_major;
+	__u16 version_minor;
+	__u32 num_tasks;
+	__u32 size;
+	__u32 reserved[4];
+};
+
+#define LIMA_DUMP_TASK_GP   0
+#define LIMA_DUMP_TASK_PP   1
+#define LIMA_DUMP_TASK_NUM  2
+
+struct lima_dump_task {
+	__u32 id;
+	__u32 size;
+	__u32 num_chunks;
+	__u32 reserved;
+};
+
+#define LIMA_DUMP_CHUNK_FRAME         0
+#define LIMA_DUMP_CHUNK_BUFFER        1
+#define LIMA_DUMP_CHUNK_PROCESS_NAME  2
+#define LIMA_DUMP_CHUNK_PROCESS_ID    3
+#define LIMA_DUMP_CHUNK_NUM           4
+
+struct lima_dump_chunk {
+	__u32 id;
+	__u32 size;
+	__u32 reserved[2];
+};
+
+struct lima_dump_chunk_buffer {
+	__u32 id;
+	__u32 size;
+	__u32 va;
+	__u32 reserved;
+};
+
+struct lima_dump_chunk_pid {
+	__u32 id;
+	__u32 size;
+	__u32 pid;
+	__u32 reserved;
+};
+
+#endif
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index 3886999b4533..cd1bf3ad9bb5 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -256,6 +256,132 @@ static struct dma_fence *lima_sched_run_job(struct drm_sched_job *job)
 	return task->fence;
 }
 
+static void lima_sched_build_error_task_list(struct lima_sched_task *task)
+{
+	struct lima_sched_error_task *et;
+	struct lima_sched_pipe *pipe = to_lima_pipe(task->base.sched);
+	struct lima_ip *ip = pipe->processor[0];
+	int pipe_id = ip->id == lima_ip_gp ? lima_pipe_gp : lima_pipe_pp;
+	struct lima_device *dev = ip->dev;
+	struct lima_sched_context *sched_ctx =
+		container_of(task->base.entity, struct lima_sched_context, base);
+	struct lima_ctx *ctx =
+		container_of(sched_ctx, struct lima_ctx, context[pipe_id]);
+	struct lima_dump_task *dt;
+	struct lima_dump_chunk *chunk;
+	struct lima_dump_chunk_pid *pid_chunk;
+	struct lima_dump_chunk_buffer *buffer_chunk;
+	uint32_t size, task_size, mem_size;
+	int i;
+
+	mutex_lock(&dev->error_task_list_lock);
+
+	if (dev->dump.num_tasks >= lima_max_error_tasks) {
+		dev_info(dev->dev, "fail to save task state: error task list is full\n");
+		goto out;
+	}
+
+	/* frame chunk */
+	size = sizeof(struct lima_dump_chunk) + pipe->frame_size;
+	/* process name chunk */
+	size += sizeof(struct lima_dump_chunk) + sizeof(ctx->pname);
+	/* pid chunk */
+	size += sizeof(struct lima_dump_chunk);
+	/* buffer chunks */
+	for (i = 0; i < task->num_bos; i++) {
+		struct lima_bo *bo = task->bos[i];
+
+		size += sizeof(struct lima_dump_chunk);
+		size += bo->heap_size ? bo->heap_size : lima_bo_size(bo);
+	}
+
+	task_size = size + sizeof(struct lima_dump_task);
+	mem_size = task_size + sizeof(*et);
+	et = kvmalloc(mem_size, GFP_KERNEL);
+	if (!et) {
+		dev_err(dev->dev, "fail to alloc task dump buffer of size %x\n",
+			mem_size);
+		goto out;
+	}
+
+	et->data = et + 1;
+	et->size = task_size;
+
+	dt = et->data;
+	memset(dt, 0, sizeof(*dt));
+	dt->id = pipe_id;
+	dt->size = size;
+
+	chunk = (struct lima_dump_chunk *)(dt + 1);
+	memset(chunk, 0, sizeof(*chunk));
+	chunk->id = LIMA_DUMP_CHUNK_FRAME;
+	chunk->size = pipe->frame_size;
+	memcpy(chunk + 1, task->frame, pipe->frame_size);
+	dt->num_chunks++;
+
+	chunk = (void *)(chunk + 1) + chunk->size;
+	memset(chunk, 0, sizeof(*chunk));
+	chunk->id = LIMA_DUMP_CHUNK_PROCESS_NAME;
+	chunk->size = sizeof(ctx->pname);
+	memcpy(chunk + 1, ctx->pname, sizeof(ctx->pname));
+	dt->num_chunks++;
+
+	pid_chunk = (void *)(chunk + 1) + chunk->size;
+	memset(pid_chunk, 0, sizeof(*pid_chunk));
+	pid_chunk->id = LIMA_DUMP_CHUNK_PROCESS_ID;
+	pid_chunk->pid = ctx->pid;
+	dt->num_chunks++;
+
+	buffer_chunk = (void *)(pid_chunk + 1) + pid_chunk->size;
+	for (i = 0; i < task->num_bos; i++) {
+		struct lima_bo *bo = task->bos[i];
+		void *data;
+
+		memset(buffer_chunk, 0, sizeof(*buffer_chunk));
+	        buffer_chunk->id = LIMA_DUMP_CHUNK_BUFFER;
+		buffer_chunk->va = lima_vm_get_va(task->vm, bo);
+
+		if (bo->heap_size) {
+			buffer_chunk->size = bo->heap_size;
+
+			data = vmap(bo->base.pages, bo->heap_size >> PAGE_SHIFT,
+				    VM_MAP, pgprot_writecombine(PAGE_KERNEL));
+			if (!data) {
+				kvfree(et);
+				goto out;
+			}
+
+			memcpy(buffer_chunk + 1, data, buffer_chunk->size);
+
+			vunmap(data);
+		} else {
+			buffer_chunk->size = lima_bo_size(bo);
+
+			data = drm_gem_shmem_vmap(&bo->base.base);
+			if (IS_ERR_OR_NULL(data)) {
+				kvfree(et);
+				goto out;
+			}
+
+			memcpy(buffer_chunk + 1, data, buffer_chunk->size);
+
+			drm_gem_shmem_vunmap(&bo->base.base, data);
+		}
+
+		buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
+		dt->num_chunks++;
+	}
+
+	list_add(&et->list, &dev->error_task_list);
+	dev->dump.size += et->size;
+	dev->dump.num_tasks++;
+
+	dev_info(dev->dev, "save error task state success\n");
+
+out:
+	mutex_unlock(&dev->error_task_list_lock);
+}
+
 static void lima_sched_timedout_job(struct drm_sched_job *job)
 {
 	struct lima_sched_pipe *pipe = to_lima_pipe(job->sched);
@@ -268,6 +394,8 @@ static void lima_sched_timedout_job(struct drm_sched_job *job)
 
 	drm_sched_increase_karma(&task->base);
 
+	lima_sched_build_error_task_list(task);
+
 	pipe->task_error(pipe);
 
 	if (pipe->bcast_mmu)
diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
index d64393fb50a9..e29f5e3b675b 100644
--- a/drivers/gpu/drm/lima/lima_sched.h
+++ b/drivers/gpu/drm/lima/lima_sched.h
@@ -5,9 +5,16 @@
 #define __LIMA_SCHED_H__
 
 #include <drm/gpu_scheduler.h>
+#include <linux/list.h>
 
 struct lima_vm;
 
+struct lima_sched_error_task {
+	struct list_head list;
+	void *data;
+	uint32_t size;
+};
+
 struct lima_sched_task {
 	struct drm_sched_job base;
 
-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 4/5] drm/lima: add error sysfs to export error task dump

[Qiang Yu]

Export /sys/class/drm/cardX/device/error sysfs for user read out
error task dump file.

Signed-off-by: Qiang Yu <yuq825@gmail.com>
---
 drivers/gpu/drm/lima/lima_drv.c | 93 +++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)

diff --git a/drivers/gpu/drm/lima/lima_drv.c b/drivers/gpu/drm/lima/lima_drv.c
index e235d4545b6c..8c5adc025902 100644
--- a/drivers/gpu/drm/lima/lima_drv.c
+++ b/drivers/gpu/drm/lima/lima_drv.c
@@ -276,6 +276,92 @@ static struct drm_driver lima_drm_driver = {
 	.gem_prime_mmap = drm_gem_prime_mmap,
 };
 
+struct lima_block_reader {
+	void *dst;
+	size_t base;
+	size_t count;
+	size_t off;
+	ssize_t read;
+};
+
+static bool lima_read_block(struct lima_block_reader *reader, void *src, size_t src_size)
+{
+	size_t max_off = reader->base + src_size;
+
+	if (reader->off < max_off) {
+		size_t size = min_t(size_t, max_off - reader->off, reader->count);
+
+		memcpy(reader->dst, src + (reader->off - reader->base), size);
+
+		reader->dst += size;
+		reader->off += size;
+		reader->read += size;
+		reader->count -= size;
+	}
+
+	reader->base = max_off;
+
+	return !!reader->count;
+}
+
+static ssize_t lima_error_state_read(struct file *filp, struct kobject *kobj,
+				     struct bin_attribute *attr, char *buf,
+				     loff_t off, size_t count)
+{
+
+	struct device *dev = kobj_to_dev(kobj);
+	struct lima_device *ldev = dev_get_drvdata(dev);
+	struct lima_sched_error_task *et;
+	struct lima_block_reader reader = {
+		.dst = buf,
+		.count = count,
+		.off = off,
+	};
+
+	mutex_lock(&ldev->error_task_list_lock);
+
+	if (lima_read_block(&reader, &ldev->dump, sizeof(ldev->dump))) {
+		list_for_each_entry(et, &ldev->error_task_list, list) {
+			if (!lima_read_block(&reader, et->data, et->size))
+				break;
+		}
+	}
+
+	mutex_unlock(&ldev->error_task_list_lock);
+	return reader.read;
+}
+
+static ssize_t lima_error_state_write(struct file *file, struct kobject *kobj,
+				      struct bin_attribute *attr, char *buf,
+				      loff_t off, size_t count)
+{
+	struct device *dev = kobj_to_dev(kobj);
+	struct lima_device *ldev = dev_get_drvdata(dev);
+	struct lima_sched_error_task *et, *tmp;
+
+	mutex_lock(&ldev->error_task_list_lock);
+
+	list_for_each_entry_safe(et, tmp, &ldev->error_task_list, list) {
+		list_del(&et->list);
+		kvfree(et);
+	}
+
+	ldev->dump.size = 0;
+	ldev->dump.num_tasks = 0;
+
+	mutex_unlock(&ldev->error_task_list_lock);
+
+	return count;
+}
+
+static const struct bin_attribute lima_error_state_attr = {
+	.attr.name = "error",
+	.attr.mode = S_IRUSR | S_IWUSR,
+	.size = 0,
+	.read = lima_error_state_read,
+	.write = lima_error_state_write,
+};
+
 static int lima_pdev_probe(struct platform_device *pdev)
 {
 	struct lima_device *ldev;
@@ -318,6 +404,11 @@ static int lima_pdev_probe(struct platform_device *pdev)
 	if (err < 0)
 		goto err_out2;
 
+	platform_set_drvdata(pdev, ldev);
+
+	if (sysfs_create_bin_file(&ldev->dev->kobj, &lima_error_state_attr))
+		dev_warn(ldev->dev, "fail to create error state sysfs\n");
+
 	return 0;
 
 err_out2:
@@ -334,6 +425,8 @@ static int lima_pdev_remove(struct platform_device *pdev)
 	struct lima_device *ldev = platform_get_drvdata(pdev);
 	struct drm_device *ddev = ldev->ddev;
 
+	sysfs_remove_bin_file(&ldev->dev->kobj, &lima_error_state_attr);
+	platform_set_drvdata(pdev, NULL);
 	drm_dev_unregister(ddev);
 	lima_device_fini(ldev);
 	drm_dev_put(ddev);
-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH 5/5] drm/lima: add LIMA_BO_FLAG_FORCE_VA

[Qiang Yu]

User can force created buffer to be mapped to GPU VM at a user
specified address. This is used for debug tools in user space to
replay some task.

Signed-off-by: Qiang Yu <yuq825@gmail.com>
---
 drivers/gpu/drm/lima/lima_drv.c | 26 +++++++++++++++++++-------
 drivers/gpu/drm/lima/lima_gem.c |  7 +++++--
 drivers/gpu/drm/lima/lima_gem.h |  4 +++-
 drivers/gpu/drm/lima/lima_vm.c  | 13 ++++++++++++-
 include/uapi/drm/lima_drm.h     |  9 +++++++--
 5 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/drivers/gpu/drm/lima/lima_drv.c b/drivers/gpu/drm/lima/lima_drv.c
index 8c5adc025902..f65021ea9598 100644
--- a/drivers/gpu/drm/lima/lima_drv.c
+++ b/drivers/gpu/drm/lima/lima_drv.c
@@ -73,16 +73,27 @@ static int lima_ioctl_gem_create(struct drm_device *dev, void *data, struct drm_
 {
 	struct drm_lima_gem_create *args = data;
 
-	if (args->pad)
-		return -EINVAL;
-
-	if (args->flags & ~(LIMA_BO_FLAG_HEAP))
+	if (args->flags & ~(LIMA_BO_FLAG_HEAP | LIMA_BO_FLAG_FORCE_VA))
 		return -EINVAL;
 
 	if (args->size == 0)
 		return -EINVAL;
 
-	return lima_gem_create_handle(dev, file, args->size, args->flags, &args->handle);
+	if (args->flags & LIMA_BO_FLAG_FORCE_VA) {
+		u64 max = (u64)args->va + (u64)args->size;
+
+		if (max > LIMA_VA_RESERVE_START)
+			return -EINVAL;
+
+		if (!IS_ALIGNED(args->va, PAGE_SIZE))
+			return -EINVAL;
+	} else {
+		if (args->va)
+			return -EINVAL;
+	}
+
+	return lima_gem_create_handle(dev, file, args->size, args->flags,
+				      &args->handle, args->va);
 }
 
 static int lima_ioctl_gem_info(struct drm_device *dev, void *data, struct drm_file *file)
@@ -253,6 +264,7 @@ DEFINE_DRM_GEM_FOPS(lima_drm_driver_fops);
  * Changelog:
  *
  * - 1.1.0 - add heap buffer support
+ * - 1.2.0 - add force va support
  */
 
 static struct drm_driver lima_drm_driver = {
@@ -264,9 +276,9 @@ static struct drm_driver lima_drm_driver = {
 	.fops               = &lima_drm_driver_fops,
 	.name               = "lima",
 	.desc               = "lima DRM",
-	.date               = "20191231",
+	.date               = "20200215",
 	.major              = 1,
-	.minor              = 1,
+	.minor              = 2,
 	.patchlevel         = 0,
 
 	.gem_create_object  = lima_gem_create_object,
diff --git a/drivers/gpu/drm/lima/lima_gem.c b/drivers/gpu/drm/lima/lima_gem.c
index 5404e0d668db..c28d90756584 100644
--- a/drivers/gpu/drm/lima/lima_gem.c
+++ b/drivers/gpu/drm/lima/lima_gem.c
@@ -95,7 +95,7 @@ int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm)
 }
 
 int lima_gem_create_handle(struct drm_device *dev, struct drm_file *file,
-			   u32 size, u32 flags, u32 *handle)
+			   u32 size, u32 flags, u32 *handle, u32 va)
 {
 	int err;
 	gfp_t mask;
@@ -116,8 +116,11 @@ int lima_gem_create_handle(struct drm_device *dev, struct drm_file *file,
 	mask |= __GFP_DMA32;
 	mapping_set_gfp_mask(obj->filp->f_mapping, mask);
 
+	bo = to_lima_bo(obj);
+	bo->flags = flags;
+	bo->force_va = va;
+
 	if (is_heap) {
-		bo = to_lima_bo(obj);
 		err = lima_heap_alloc(bo, NULL);
 		if (err)
 			goto out;
diff --git a/drivers/gpu/drm/lima/lima_gem.h b/drivers/gpu/drm/lima/lima_gem.h
index ccea06142f4b..2a6db0c0be89 100644
--- a/drivers/gpu/drm/lima/lima_gem.h
+++ b/drivers/gpu/drm/lima/lima_gem.h
@@ -15,6 +15,8 @@ struct lima_bo {
 	struct mutex lock;
 	struct list_head va;
 
+	u32 flags;
+	u32 force_va;
 	size_t heap_size;
 };
 
@@ -37,7 +39,7 @@ static inline struct dma_resv *lima_bo_resv(struct lima_bo *bo)
 int lima_heap_alloc(struct lima_bo *bo, struct lima_vm *vm);
 struct drm_gem_object *lima_gem_create_object(struct drm_device *dev, size_t size);
 int lima_gem_create_handle(struct drm_device *dev, struct drm_file *file,
-			   u32 size, u32 flags, u32 *handle);
+			   u32 size, u32 flags, u32 *handle, u32 va);
 int lima_gem_get_info(struct drm_file *file, u32 handle, u32 *va, u64 *offset);
 int lima_gem_submit(struct drm_file *file, struct lima_submit *submit);
 int lima_gem_wait(struct drm_file *file, u32 handle, u32 op, s64 timeout_ns);
diff --git a/drivers/gpu/drm/lima/lima_vm.c b/drivers/gpu/drm/lima/lima_vm.c
index 5b92fb82674a..be0510032538 100644
--- a/drivers/gpu/drm/lima/lima_vm.c
+++ b/drivers/gpu/drm/lima/lima_vm.c
@@ -3,6 +3,7 @@
 
 #include <linux/slab.h>
 #include <linux/dma-mapping.h>
+#include <drm/lima_drm.h>
 
 #include "lima_device.h"
 #include "lima_vm.h"
@@ -93,6 +94,7 @@ int lima_vm_bo_add(struct lima_vm *vm, struct lima_bo *bo, bool create)
 	struct lima_bo_va *bo_va;
 	struct sg_dma_page_iter sg_iter;
 	int offset = 0, err;
+	u64 start, end;
 
 	mutex_lock(&bo->lock);
 
@@ -120,7 +122,16 @@ int lima_vm_bo_add(struct lima_vm *vm, struct lima_bo *bo, bool create)
 
 	mutex_lock(&vm->lock);
 
-	err = drm_mm_insert_node(&vm->mm, &bo_va->node, lima_bo_size(bo));
+	if (bo->flags & LIMA_BO_FLAG_FORCE_VA) {
+		start = bo->force_va;
+		end = start + lima_bo_size(bo);
+	} else {
+		start = 0;
+		end = U64_MAX;
+	}
+
+	err = drm_mm_insert_node_in_range(&vm->mm, &bo_va->node, lima_bo_size(bo),
+					  0, 0, start, end, 0);
 	if (err)
 		goto err_out1;
 
diff --git a/include/uapi/drm/lima_drm.h b/include/uapi/drm/lima_drm.h
index 1ec58d652a5a..3e699bb78394 100644
--- a/include/uapi/drm/lima_drm.h
+++ b/include/uapi/drm/lima_drm.h
@@ -37,7 +37,12 @@ struct drm_lima_get_param {
  * due to lack of heap memory. size field of heap buffer is an up bound of
  * the backup memory which can be set to a fairly large value.
  */
-#define LIMA_BO_FLAG_HEAP  (1 << 0)
+#define LIMA_BO_FLAG_HEAP        (1 << 0)
+/*
+ * force buffer GPU virtual address to be drm_lima_gem_create.va, this is
+ * used to replay some task with fixed GPU virtual address
+ */
+#define LIMA_BO_FLAG_FORCE_VA    (1 << 1)
 
 /**
  * create a buffer for used by GPU
@@ -46,7 +51,7 @@ struct drm_lima_gem_create {
 	__u32 size;    /* in, buffer size */
 	__u32 flags;   /* in, buffer flags */
 	__u32 handle;  /* out, GEM buffer handle */
-	__u32 pad;     /* pad, must be zero */
+	__u32 va;      /* in, buffer va */
 };
 
 /**
-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 3/5] drm/lima: save task info dump when task fail

[kbuild test robot]

[-- Attachment #1: Type: text/plain, Size: 6602 bytes --]

Hi Qiang,

I love your patch! Yet something to improve:

[auto build test ERROR on drm-tip/drm-tip]
[cannot apply to drm-exynos/exynos-drm-next drm-intel/for-linux-next tegra-drm/drm/tegra/for-next linus/master v5.6-rc2 next-20200221]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Qiang-Yu/drm-lima-add-error-debug-functionality/20200223-054634
base:   git://anongit.freedesktop.org/drm/drm-tip drm-tip
config: mips-allyesconfig (attached as .config)
compiler: mips-linux-gcc (GCC) 7.5.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.5.0 make.cross ARCH=mips 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   drivers/gpu/drm/lima/lima_sched.c: In function 'lima_sched_build_error_task_list':
>> drivers/gpu/drm/lima/lima_sched.c:347:11: error: implicit declaration of function 'vmap'; did you mean 'bmap'? [-Werror=implicit-function-declaration]
       data = vmap(bo->base.pages, bo->heap_size >> PAGE_SHIFT,
              ^~~~
              bmap
>> drivers/gpu/drm/lima/lima_sched.c:348:9: error: 'VM_MAP' undeclared (first use in this function); did you mean 'VM_MPX'?
            VM_MAP, pgprot_writecombine(PAGE_KERNEL));
            ^~~~~~
            VM_MPX
   drivers/gpu/drm/lima/lima_sched.c:348:9: note: each undeclared identifier is reported only once for each function it appears in
>> drivers/gpu/drm/lima/lima_sched.c:356:4: error: implicit declaration of function 'vunmap'; did you mean 'kunmap'? [-Werror=implicit-function-declaration]
       vunmap(data);
       ^~~~~~
       kunmap
   cc1: some warnings being treated as errors

vim +347 drivers/gpu/drm/lima/lima_sched.c

   258	
   259	static void lima_sched_build_error_task_list(struct lima_sched_task *task)
   260	{
   261		struct lima_sched_error_task *et;
   262		struct lima_sched_pipe *pipe = to_lima_pipe(task->base.sched);
   263		struct lima_ip *ip = pipe->processor[0];
   264		int pipe_id = ip->id == lima_ip_gp ? lima_pipe_gp : lima_pipe_pp;
   265		struct lima_device *dev = ip->dev;
   266		struct lima_sched_context *sched_ctx =
   267			container_of(task->base.entity, struct lima_sched_context, base);
   268		struct lima_ctx *ctx =
   269			container_of(sched_ctx, struct lima_ctx, context[pipe_id]);
   270		struct lima_dump_task *dt;
   271		struct lima_dump_chunk *chunk;
   272		struct lima_dump_chunk_pid *pid_chunk;
   273		struct lima_dump_chunk_buffer *buffer_chunk;
   274		uint32_t size, task_size, mem_size;
   275		int i;
   276	
   277		mutex_lock(&dev->error_task_list_lock);
   278	
   279		if (dev->dump.num_tasks >= lima_max_error_tasks) {
   280			dev_info(dev->dev, "fail to save task state: error task list is full\n");
   281			goto out;
   282		}
   283	
   284		/* frame chunk */
   285		size = sizeof(struct lima_dump_chunk) + pipe->frame_size;
   286		/* process name chunk */
   287		size += sizeof(struct lima_dump_chunk) + sizeof(ctx->pname);
   288		/* pid chunk */
   289		size += sizeof(struct lima_dump_chunk);
   290		/* buffer chunks */
   291		for (i = 0; i < task->num_bos; i++) {
   292			struct lima_bo *bo = task->bos[i];
   293	
   294			size += sizeof(struct lima_dump_chunk);
   295			size += bo->heap_size ? bo->heap_size : lima_bo_size(bo);
   296		}
   297	
   298		task_size = size + sizeof(struct lima_dump_task);
   299		mem_size = task_size + sizeof(*et);
   300		et = kvmalloc(mem_size, GFP_KERNEL);
   301		if (!et) {
   302			dev_err(dev->dev, "fail to alloc task dump buffer of size %x\n",
   303				mem_size);
   304			goto out;
   305		}
   306	
   307		et->data = et + 1;
   308		et->size = task_size;
   309	
   310		dt = et->data;
   311		memset(dt, 0, sizeof(*dt));
   312		dt->id = pipe_id;
   313		dt->size = size;
   314	
   315		chunk = (struct lima_dump_chunk *)(dt + 1);
   316		memset(chunk, 0, sizeof(*chunk));
   317		chunk->id = LIMA_DUMP_CHUNK_FRAME;
   318		chunk->size = pipe->frame_size;
   319		memcpy(chunk + 1, task->frame, pipe->frame_size);
   320		dt->num_chunks++;
   321	
   322		chunk = (void *)(chunk + 1) + chunk->size;
   323		memset(chunk, 0, sizeof(*chunk));
   324		chunk->id = LIMA_DUMP_CHUNK_PROCESS_NAME;
   325		chunk->size = sizeof(ctx->pname);
   326		memcpy(chunk + 1, ctx->pname, sizeof(ctx->pname));
   327		dt->num_chunks++;
   328	
   329		pid_chunk = (void *)(chunk + 1) + chunk->size;
   330		memset(pid_chunk, 0, sizeof(*pid_chunk));
   331		pid_chunk->id = LIMA_DUMP_CHUNK_PROCESS_ID;
   332		pid_chunk->pid = ctx->pid;
   333		dt->num_chunks++;
   334	
   335		buffer_chunk = (void *)(pid_chunk + 1) + pid_chunk->size;
   336		for (i = 0; i < task->num_bos; i++) {
   337			struct lima_bo *bo = task->bos[i];
   338			void *data;
   339	
   340			memset(buffer_chunk, 0, sizeof(*buffer_chunk));
   341		        buffer_chunk->id = LIMA_DUMP_CHUNK_BUFFER;
   342			buffer_chunk->va = lima_vm_get_va(task->vm, bo);
   343	
   344			if (bo->heap_size) {
   345				buffer_chunk->size = bo->heap_size;
   346	
 > 347				data = vmap(bo->base.pages, bo->heap_size >> PAGE_SHIFT,
 > 348					    VM_MAP, pgprot_writecombine(PAGE_KERNEL));
   349				if (!data) {
   350					kvfree(et);
   351					goto out;
   352				}
   353	
   354				memcpy(buffer_chunk + 1, data, buffer_chunk->size);
   355	
 > 356				vunmap(data);
   357			} else {
   358				buffer_chunk->size = lima_bo_size(bo);
   359	
   360				data = drm_gem_shmem_vmap(&bo->base.base);
   361				if (IS_ERR_OR_NULL(data)) {
   362					kvfree(et);
   363					goto out;
   364				}
   365	
   366				memcpy(buffer_chunk + 1, data, buffer_chunk->size);
   367	
   368				drm_gem_shmem_vunmap(&bo->base.base, data);
   369			}
   370	
   371			buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
   372			dt->num_chunks++;
   373		}
   374	
   375		list_add(&et->list, &dev->error_task_list);
   376		dev->dump.size += et->size;
   377		dev->dump.num_tasks++;
   378	
   379		dev_info(dev->dev, "save error task state success\n");
   380	
   381	out:
   382		mutex_unlock(&dev->error_task_list_lock);
   383	}
   384	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 64305 bytes --]

[-- Attachment #3: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 3/5] drm/lima: save task info dump when task fail

[Vasily Khoruzhick]

On Fri, Feb 21, 2020 at 6:43 PM Qiang Yu <yuq825@gmail.com> wrote:
>
> Save all information to start a task which can be exported to user
> for debug usage. Dump file data format is specified in lima_dump.h

Looks like lima_sched.c is missing #include <linux/vmalloc.h> so
kbuild bot complains

> Signed-off-by: Qiang Yu <yuq825@gmail.com>
> ---
>  drivers/gpu/drm/lima/lima_device.c |  13 +++
>  drivers/gpu/drm/lima/lima_device.h |   8 ++
>  drivers/gpu/drm/lima/lima_dump.h   |  77 +++++++++++++++++
>  drivers/gpu/drm/lima/lima_sched.c  | 128 +++++++++++++++++++++++++++++
>  drivers/gpu/drm/lima/lima_sched.h  |   7 ++
>  5 files changed, 233 insertions(+)
>  create mode 100644 drivers/gpu/drm/lima/lima_dump.h
>
> diff --git a/drivers/gpu/drm/lima/lima_device.c b/drivers/gpu/drm/lima/lima_device.c
> index 19829b543024..42a00171fea5 100644
> --- a/drivers/gpu/drm/lima/lima_device.c
> +++ b/drivers/gpu/drm/lima/lima_device.c
> @@ -344,6 +344,12 @@ int lima_device_init(struct lima_device *ldev)
>         if (err)
>                 goto err_out5;
>
> +       ldev->dump.magic = LIMA_DUMP_MAGIC;
> +       ldev->dump.version_major = LIMA_DUMP_MAJOR;
> +       ldev->dump.version_minor = LIMA_DUMP_MINOR;
> +       INIT_LIST_HEAD(&ldev->error_task_list);
> +       mutex_init(&ldev->error_task_list_lock);
> +
>         dev_info(ldev->dev, "bus rate = %lu\n", clk_get_rate(ldev->clk_bus));
>         dev_info(ldev->dev, "mod rate = %lu", clk_get_rate(ldev->clk_gpu));
>
> @@ -370,6 +376,13 @@ int lima_device_init(struct lima_device *ldev)
>  void lima_device_fini(struct lima_device *ldev)
>  {
>         int i;
> +       struct lima_sched_error_task *et, *tmp;
> +
> +       list_for_each_entry_safe(et, tmp, &ldev->error_task_list, list) {
> +               list_del(&et->list);
> +               kvfree(et);
> +       }
> +       mutex_destroy(&ldev->error_task_list_lock);
>
>         lima_fini_pp_pipe(ldev);
>         lima_fini_gp_pipe(ldev);
> diff --git a/drivers/gpu/drm/lima/lima_device.h b/drivers/gpu/drm/lima/lima_device.h
> index 31158d86271c..f17173f47f26 100644
> --- a/drivers/gpu/drm/lima/lima_device.h
> +++ b/drivers/gpu/drm/lima/lima_device.h
> @@ -6,8 +6,11 @@
>
>  #include <drm/drm_device.h>
>  #include <linux/delay.h>
> +#include <linux/list.h>
> +#include <linux/mutex.h>
>
>  #include "lima_sched.h"
> +#include "lima_dump.h"
>
>  enum lima_gpu_id {
>         lima_gpu_mali400 = 0,
> @@ -94,6 +97,11 @@ struct lima_device {
>
>         u32 *dlbu_cpu;
>         dma_addr_t dlbu_dma;
> +
> +       /* debug info */
> +       struct lima_dump_head dump;
> +       struct list_head error_task_list;
> +       struct mutex error_task_list_lock;
>  };
>
>  static inline struct lima_device *
> diff --git a/drivers/gpu/drm/lima/lima_dump.h b/drivers/gpu/drm/lima/lima_dump.h
> new file mode 100644
> index 000000000000..ca243d99c51b
> --- /dev/null
> +++ b/drivers/gpu/drm/lima/lima_dump.h
> @@ -0,0 +1,77 @@
> +/* SPDX-License-Identifier: GPL-2.0 OR MIT */
> +/* Copyright 2020 Qiang Yu <yuq825@gmail.com> */
> +
> +#ifndef __LIMA_DUMP_H__
> +#define __LIMA_DUMP_H__
> +
> +#include <linux/types.h>
> +
> +/**
> + * dump file format for all the information to start a lima task
> + *
> + * top level format
> + * | magic code "LIMA" | format version | num tasks | data size |
> + * | reserved | reserved | reserved | reserved |
> + * | task 1 ID | task 1 size | num chunks | reserved | task 1 data |
> + * | task 2 ID | task 2 size | num chunks | reserved | task 2 data |
> + * ...
> + *
> + * task data format
> + * | chunk 1 ID | chunk 1 size | reserved | reserved | chunk 1 data |
> + * | chunk 2 ID | chunk 2 size | reserved | reserved | chunk 2 data |
> + * ...
> + *
> + */
> +
> +#define LIMA_DUMP_MAJOR 1
> +#define LIMA_DUMP_MINOR 0
> +
> +#define LIMA_DUMP_MAGIC 0x414d494c
> +
> +struct lima_dump_head {
> +       __u32 magic;
> +       __u16 version_major;
> +       __u16 version_minor;
> +       __u32 num_tasks;
> +       __u32 size;
> +       __u32 reserved[4];
> +};
> +
> +#define LIMA_DUMP_TASK_GP   0
> +#define LIMA_DUMP_TASK_PP   1
> +#define LIMA_DUMP_TASK_NUM  2
> +
> +struct lima_dump_task {
> +       __u32 id;
> +       __u32 size;
> +       __u32 num_chunks;
> +       __u32 reserved;
> +};
> +
> +#define LIMA_DUMP_CHUNK_FRAME         0
> +#define LIMA_DUMP_CHUNK_BUFFER        1
> +#define LIMA_DUMP_CHUNK_PROCESS_NAME  2
> +#define LIMA_DUMP_CHUNK_PROCESS_ID    3
> +#define LIMA_DUMP_CHUNK_NUM           4
> +
> +struct lima_dump_chunk {
> +       __u32 id;
> +       __u32 size;
> +       __u32 reserved[2];
> +};
> +
> +struct lima_dump_chunk_buffer {
> +       __u32 id;
> +       __u32 size;
> +       __u32 va;
> +       __u32 reserved;
> +};
> +
> +struct lima_dump_chunk_pid {
> +       __u32 id;
> +       __u32 size;
> +       __u32 pid;
> +       __u32 reserved;
> +};
> +
> +#endif
> diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
> index 3886999b4533..cd1bf3ad9bb5 100644
> --- a/drivers/gpu/drm/lima/lima_sched.c
> +++ b/drivers/gpu/drm/lima/lima_sched.c
> @@ -256,6 +256,132 @@ static struct dma_fence *lima_sched_run_job(struct drm_sched_job *job)
>         return task->fence;
>  }
>
> +static void lima_sched_build_error_task_list(struct lima_sched_task *task)
> +{
> +       struct lima_sched_error_task *et;
> +       struct lima_sched_pipe *pipe = to_lima_pipe(task->base.sched);
> +       struct lima_ip *ip = pipe->processor[0];
> +       int pipe_id = ip->id == lima_ip_gp ? lima_pipe_gp : lima_pipe_pp;
> +       struct lima_device *dev = ip->dev;
> +       struct lima_sched_context *sched_ctx =
> +               container_of(task->base.entity, struct lima_sched_context, base);
> +       struct lima_ctx *ctx =
> +               container_of(sched_ctx, struct lima_ctx, context[pipe_id]);
> +       struct lima_dump_task *dt;
> +       struct lima_dump_chunk *chunk;
> +       struct lima_dump_chunk_pid *pid_chunk;
> +       struct lima_dump_chunk_buffer *buffer_chunk;
> +       uint32_t size, task_size, mem_size;
> +       int i;
> +
> +       mutex_lock(&dev->error_task_list_lock);
> +
> +       if (dev->dump.num_tasks >= lima_max_error_tasks) {
> +               dev_info(dev->dev, "fail to save task state: error task list is full\n");
> +               goto out;
> +       }
> +
> +       /* frame chunk */
> +       size = sizeof(struct lima_dump_chunk) + pipe->frame_size;
> +       /* process name chunk */
> +       size += sizeof(struct lima_dump_chunk) + sizeof(ctx->pname);
> +       /* pid chunk */
> +       size += sizeof(struct lima_dump_chunk);
> +       /* buffer chunks */
> +       for (i = 0; i < task->num_bos; i++) {
> +               struct lima_bo *bo = task->bos[i];
> +
> +               size += sizeof(struct lima_dump_chunk);
> +               size += bo->heap_size ? bo->heap_size : lima_bo_size(bo);
> +       }
> +
> +       task_size = size + sizeof(struct lima_dump_task);
> +       mem_size = task_size + sizeof(*et);
> +       et = kvmalloc(mem_size, GFP_KERNEL);
> +       if (!et) {
> +               dev_err(dev->dev, "fail to alloc task dump buffer of size %x\n",
> +                       mem_size);
> +               goto out;
> +       }
> +
> +       et->data = et + 1;
> +       et->size = task_size;
> +
> +       dt = et->data;
> +       memset(dt, 0, sizeof(*dt));
> +       dt->id = pipe_id;
> +       dt->size = size;
> +
> +       chunk = (struct lima_dump_chunk *)(dt + 1);
> +       memset(chunk, 0, sizeof(*chunk));
> +       chunk->id = LIMA_DUMP_CHUNK_FRAME;
> +       chunk->size = pipe->frame_size;
> +       memcpy(chunk + 1, task->frame, pipe->frame_size);
> +       dt->num_chunks++;
> +
> +       chunk = (void *)(chunk + 1) + chunk->size;
> +       memset(chunk, 0, sizeof(*chunk));
> +       chunk->id = LIMA_DUMP_CHUNK_PROCESS_NAME;
> +       chunk->size = sizeof(ctx->pname);
> +       memcpy(chunk + 1, ctx->pname, sizeof(ctx->pname));
> +       dt->num_chunks++;
> +
> +       pid_chunk = (void *)(chunk + 1) + chunk->size;
> +       memset(pid_chunk, 0, sizeof(*pid_chunk));
> +       pid_chunk->id = LIMA_DUMP_CHUNK_PROCESS_ID;
> +       pid_chunk->pid = ctx->pid;
> +       dt->num_chunks++;
> +
> +       buffer_chunk = (void *)(pid_chunk + 1) + pid_chunk->size;
> +       for (i = 0; i < task->num_bos; i++) {
> +               struct lima_bo *bo = task->bos[i];
> +               void *data;
> +
> +               memset(buffer_chunk, 0, sizeof(*buffer_chunk));
> +               buffer_chunk->id = LIMA_DUMP_CHUNK_BUFFER;
> +               buffer_chunk->va = lima_vm_get_va(task->vm, bo);
> +
> +               if (bo->heap_size) {
> +                       buffer_chunk->size = bo->heap_size;
> +
> +                       data = vmap(bo->base.pages, bo->heap_size >> PAGE_SHIFT,
> +                                   VM_MAP, pgprot_writecombine(PAGE_KERNEL));
> +                       if (!data) {
> +                               kvfree(et);
> +                               goto out;
> +                       }
> +
> +                       memcpy(buffer_chunk + 1, data, buffer_chunk->size);
> +
> +                       vunmap(data);
> +               } else {
> +                       buffer_chunk->size = lima_bo_size(bo);
> +
> +                       data = drm_gem_shmem_vmap(&bo->base.base);
> +                       if (IS_ERR_OR_NULL(data)) {
> +                               kvfree(et);
> +                               goto out;
> +                       }
> +
> +                       memcpy(buffer_chunk + 1, data, buffer_chunk->size);
> +
> +                       drm_gem_shmem_vunmap(&bo->base.base, data);
> +               }
> +
> +               buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
> +               dt->num_chunks++;
> +       }
> +
> +       list_add(&et->list, &dev->error_task_list);
> +       dev->dump.size += et->size;
> +       dev->dump.num_tasks++;
> +
> +       dev_info(dev->dev, "save error task state success\n");
> +
> +out:
> +       mutex_unlock(&dev->error_task_list_lock);
> +}
> +
>  static void lima_sched_timedout_job(struct drm_sched_job *job)
>  {
>         struct lima_sched_pipe *pipe = to_lima_pipe(job->sched);
> @@ -268,6 +394,8 @@ static void lima_sched_timedout_job(struct drm_sched_job *job)
>
>         drm_sched_increase_karma(&task->base);
>
> +       lima_sched_build_error_task_list(task);
> +
>         pipe->task_error(pipe);
>
>         if (pipe->bcast_mmu)
> diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
> index d64393fb50a9..e29f5e3b675b 100644
> --- a/drivers/gpu/drm/lima/lima_sched.h
> +++ b/drivers/gpu/drm/lima/lima_sched.h
> @@ -5,9 +5,16 @@
>  #define __LIMA_SCHED_H__
>
>  #include <drm/gpu_scheduler.h>
> +#include <linux/list.h>
>
>  struct lima_vm;
>
> +struct lima_sched_error_task {
> +       struct list_head list;
> +       void *data;
> +       uint32_t size;
> +};
> +
>  struct lima_sched_task {
>         struct drm_sched_job base;
>
> --
> 2.17.1
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 0/5] drm/lima: add error debug functionality

[Andreas Baierl]

I could successfully use the output with 
https://gitlab.freedesktop.org/lima/lima.dump

So you can add my
Tested-by: Andreas Baierl <ichgeh@imkreisrum.de>


Am 22.02.2020 um 03:42 schrieb Qiang Yu:
> Save task error state when it fail and export to user by
> sysfs as a binary file which can be dumped and replayed
> by lima_dump tool at:
> https://gitlab.freedesktop.org/lima/lima_dump
>
> Qiang Yu (5):
>    drm/lima: save process info for debug usage
>    drm/lima: add max_error_tasks module parameter
>    drm/lima: save task info dump when task fail
>    drm/lima: add error sysfs to export error task dump
>    drm/lima: add LIMA_BO_FLAG_FORCE_VA
>
>   drivers/gpu/drm/lima/lima_ctx.c    |   3 +
>   drivers/gpu/drm/lima/lima_ctx.h    |   5 ++
>   drivers/gpu/drm/lima/lima_device.c |  13 +++
>   drivers/gpu/drm/lima/lima_device.h |   8 ++
>   drivers/gpu/drm/lima/lima_drv.c    | 123 +++++++++++++++++++++++++--
>   drivers/gpu/drm/lima/lima_drv.h    |   1 +
>   drivers/gpu/drm/lima/lima_dump.h   |  77 +++++++++++++++++
>   drivers/gpu/drm/lima/lima_gem.c    |   7 +-
>   drivers/gpu/drm/lima/lima_gem.h    |   4 +-
>   drivers/gpu/drm/lima/lima_sched.c  | 128 +++++++++++++++++++++++++++++
>   drivers/gpu/drm/lima/lima_sched.h  |   7 ++
>   drivers/gpu/drm/lima/lima_vm.c     |  13 ++-
>   include/uapi/drm/lima_drm.h        |   9 +-
>   13 files changed, 385 insertions(+), 13 deletions(-)
>   create mode 100644 drivers/gpu/drm/lima/lima_dump.h
>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v2 3/6] drm/lima: save task info dump when task fail

[Qiang Yu]

Save all information to start a task which can be exported to user
for debug usage. Dump file data format is specified in lima_dump.h

v2:
Add include header to address build robot complain.

Signed-off-by: Qiang Yu <yuq825@gmail.com>
---
 drivers/gpu/drm/lima/lima_device.c |  13 +++
 drivers/gpu/drm/lima/lima_device.h |   8 ++
 drivers/gpu/drm/lima/lima_dump.h   |  77 +++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.c  | 129 +++++++++++++++++++++++++++++
 drivers/gpu/drm/lima/lima_sched.h  |   7 ++
 5 files changed, 234 insertions(+)
 create mode 100644 drivers/gpu/drm/lima/lima_dump.h

diff --git a/drivers/gpu/drm/lima/lima_device.c b/drivers/gpu/drm/lima/lima_device.c
index 19829b543024..42a00171fea5 100644
--- a/drivers/gpu/drm/lima/lima_device.c
+++ b/drivers/gpu/drm/lima/lima_device.c
@@ -344,6 +344,12 @@ int lima_device_init(struct lima_device *ldev)
 	if (err)
 		goto err_out5;
 
+	ldev->dump.magic = LIMA_DUMP_MAGIC;
+	ldev->dump.version_major = LIMA_DUMP_MAJOR;
+	ldev->dump.version_minor = LIMA_DUMP_MINOR;
+	INIT_LIST_HEAD(&ldev->error_task_list);
+	mutex_init(&ldev->error_task_list_lock);
+
 	dev_info(ldev->dev, "bus rate = %lu\n", clk_get_rate(ldev->clk_bus));
 	dev_info(ldev->dev, "mod rate = %lu", clk_get_rate(ldev->clk_gpu));
 
@@ -370,6 +376,13 @@ int lima_device_init(struct lima_device *ldev)
 void lima_device_fini(struct lima_device *ldev)
 {
 	int i;
+	struct lima_sched_error_task *et, *tmp;
+
+	list_for_each_entry_safe(et, tmp, &ldev->error_task_list, list) {
+		list_del(&et->list);
+		kvfree(et);
+	}
+	mutex_destroy(&ldev->error_task_list_lock);
 
 	lima_fini_pp_pipe(ldev);
 	lima_fini_gp_pipe(ldev);
diff --git a/drivers/gpu/drm/lima/lima_device.h b/drivers/gpu/drm/lima/lima_device.h
index 31158d86271c..f17173f47f26 100644
--- a/drivers/gpu/drm/lima/lima_device.h
+++ b/drivers/gpu/drm/lima/lima_device.h
@@ -6,8 +6,11 @@
 
 #include <drm/drm_device.h>
 #include <linux/delay.h>
+#include <linux/list.h>
+#include <linux/mutex.h>
 
 #include "lima_sched.h"
+#include "lima_dump.h"
 
 enum lima_gpu_id {
 	lima_gpu_mali400 = 0,
@@ -94,6 +97,11 @@ struct lima_device {
 
 	u32 *dlbu_cpu;
 	dma_addr_t dlbu_dma;
+
+	/* debug info */
+	struct lima_dump_head dump;
+	struct list_head error_task_list;
+	struct mutex error_task_list_lock;
 };
 
 static inline struct lima_device *
diff --git a/drivers/gpu/drm/lima/lima_dump.h b/drivers/gpu/drm/lima/lima_dump.h
new file mode 100644
index 000000000000..ca243d99c51b
--- /dev/null
+++ b/drivers/gpu/drm/lima/lima_dump.h
@@ -0,0 +1,77 @@
+/* SPDX-License-Identifier: GPL-2.0 OR MIT */
+/* Copyright 2020 Qiang Yu <yuq825@gmail.com> */
+
+#ifndef __LIMA_DUMP_H__
+#define __LIMA_DUMP_H__
+
+#include <linux/types.h>
+
+/**
+ * dump file format for all the information to start a lima task
+ *
+ * top level format
+ * | magic code "LIMA" | format version | num tasks | data size |
+ * | reserved | reserved | reserved | reserved |
+ * | task 1 ID | task 1 size | num chunks | reserved | task 1 data |
+ * | task 2 ID | task 2 size | num chunks | reserved | task 2 data |
+ * ...
+ *
+ * task data format
+ * | chunk 1 ID | chunk 1 size | reserved | reserved | chunk 1 data |
+ * | chunk 2 ID | chunk 2 size | reserved | reserved | chunk 2 data |
+ * ...
+ *
+ */
+
+#define LIMA_DUMP_MAJOR 1
+#define LIMA_DUMP_MINOR 0
+
+#define LIMA_DUMP_MAGIC 0x414d494c
+
+struct lima_dump_head {
+	__u32 magic;
+	__u16 version_major;
+	__u16 version_minor;
+	__u32 num_tasks;
+	__u32 size;
+	__u32 reserved[4];
+};
+
+#define LIMA_DUMP_TASK_GP   0
+#define LIMA_DUMP_TASK_PP   1
+#define LIMA_DUMP_TASK_NUM  2
+
+struct lima_dump_task {
+	__u32 id;
+	__u32 size;
+	__u32 num_chunks;
+	__u32 reserved;
+};
+
+#define LIMA_DUMP_CHUNK_FRAME         0
+#define LIMA_DUMP_CHUNK_BUFFER        1
+#define LIMA_DUMP_CHUNK_PROCESS_NAME  2
+#define LIMA_DUMP_CHUNK_PROCESS_ID    3
+#define LIMA_DUMP_CHUNK_NUM           4
+
+struct lima_dump_chunk {
+	__u32 id;
+	__u32 size;
+	__u32 reserved[2];
+};
+
+struct lima_dump_chunk_buffer {
+	__u32 id;
+	__u32 size;
+	__u32 va;
+	__u32 reserved;
+};
+
+struct lima_dump_chunk_pid {
+	__u32 id;
+	__u32 size;
+	__u32 pid;
+	__u32 reserved;
+};
+
+#endif
diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c
index 3886999b4533..f295479e3733 100644
--- a/drivers/gpu/drm/lima/lima_sched.c
+++ b/drivers/gpu/drm/lima/lima_sched.c
@@ -4,6 +4,7 @@
 #include <linux/kthread.h>
 #include <linux/slab.h>
 #include <linux/xarray.h>
+#include <linux/vmalloc.h>
 
 #include "lima_drv.h"
 #include "lima_sched.h"
@@ -256,6 +257,132 @@ static struct dma_fence *lima_sched_run_job(struct drm_sched_job *job)
 	return task->fence;
 }
 
+static void lima_sched_build_error_task_list(struct lima_sched_task *task)
+{
+	struct lima_sched_error_task *et;
+	struct lima_sched_pipe *pipe = to_lima_pipe(task->base.sched);
+	struct lima_ip *ip = pipe->processor[0];
+	int pipe_id = ip->id == lima_ip_gp ? lima_pipe_gp : lima_pipe_pp;
+	struct lima_device *dev = ip->dev;
+	struct lima_sched_context *sched_ctx =
+		container_of(task->base.entity, struct lima_sched_context, base);
+	struct lima_ctx *ctx =
+		container_of(sched_ctx, struct lima_ctx, context[pipe_id]);
+	struct lima_dump_task *dt;
+	struct lima_dump_chunk *chunk;
+	struct lima_dump_chunk_pid *pid_chunk;
+	struct lima_dump_chunk_buffer *buffer_chunk;
+	uint32_t size, task_size, mem_size;
+	int i;
+
+	mutex_lock(&dev->error_task_list_lock);
+
+	if (dev->dump.num_tasks >= lima_max_error_tasks) {
+		dev_info(dev->dev, "fail to save task state: error task list is full\n");
+		goto out;
+	}
+
+	/* frame chunk */
+	size = sizeof(struct lima_dump_chunk) + pipe->frame_size;
+	/* process name chunk */
+	size += sizeof(struct lima_dump_chunk) + sizeof(ctx->pname);
+	/* pid chunk */
+	size += sizeof(struct lima_dump_chunk);
+	/* buffer chunks */
+	for (i = 0; i < task->num_bos; i++) {
+		struct lima_bo *bo = task->bos[i];
+
+		size += sizeof(struct lima_dump_chunk);
+		size += bo->heap_size ? bo->heap_size : lima_bo_size(bo);
+	}
+
+	task_size = size + sizeof(struct lima_dump_task);
+	mem_size = task_size + sizeof(*et);
+	et = kvmalloc(mem_size, GFP_KERNEL);
+	if (!et) {
+		dev_err(dev->dev, "fail to alloc task dump buffer of size %x\n",
+			mem_size);
+		goto out;
+	}
+
+	et->data = et + 1;
+	et->size = task_size;
+
+	dt = et->data;
+	memset(dt, 0, sizeof(*dt));
+	dt->id = pipe_id;
+	dt->size = size;
+
+	chunk = (struct lima_dump_chunk *)(dt + 1);
+	memset(chunk, 0, sizeof(*chunk));
+	chunk->id = LIMA_DUMP_CHUNK_FRAME;
+	chunk->size = pipe->frame_size;
+	memcpy(chunk + 1, task->frame, pipe->frame_size);
+	dt->num_chunks++;
+
+	chunk = (void *)(chunk + 1) + chunk->size;
+	memset(chunk, 0, sizeof(*chunk));
+	chunk->id = LIMA_DUMP_CHUNK_PROCESS_NAME;
+	chunk->size = sizeof(ctx->pname);
+	memcpy(chunk + 1, ctx->pname, sizeof(ctx->pname));
+	dt->num_chunks++;
+
+	pid_chunk = (void *)(chunk + 1) + chunk->size;
+	memset(pid_chunk, 0, sizeof(*pid_chunk));
+	pid_chunk->id = LIMA_DUMP_CHUNK_PROCESS_ID;
+	pid_chunk->pid = ctx->pid;
+	dt->num_chunks++;
+
+	buffer_chunk = (void *)(pid_chunk + 1) + pid_chunk->size;
+	for (i = 0; i < task->num_bos; i++) {
+		struct lima_bo *bo = task->bos[i];
+		void *data;
+
+		memset(buffer_chunk, 0, sizeof(*buffer_chunk));
+	        buffer_chunk->id = LIMA_DUMP_CHUNK_BUFFER;
+		buffer_chunk->va = lima_vm_get_va(task->vm, bo);
+
+		if (bo->heap_size) {
+			buffer_chunk->size = bo->heap_size;
+
+			data = vmap(bo->base.pages, bo->heap_size >> PAGE_SHIFT,
+				    VM_MAP, pgprot_writecombine(PAGE_KERNEL));
+			if (!data) {
+				kvfree(et);
+				goto out;
+			}
+
+			memcpy(buffer_chunk + 1, data, buffer_chunk->size);
+
+			vunmap(data);
+		} else {
+			buffer_chunk->size = lima_bo_size(bo);
+
+			data = drm_gem_shmem_vmap(&bo->base.base);
+			if (IS_ERR_OR_NULL(data)) {
+				kvfree(et);
+				goto out;
+			}
+
+			memcpy(buffer_chunk + 1, data, buffer_chunk->size);
+
+			drm_gem_shmem_vunmap(&bo->base.base, data);
+		}
+
+		buffer_chunk = (void *)(buffer_chunk + 1) + buffer_chunk->size;
+		dt->num_chunks++;
+	}
+
+	list_add(&et->list, &dev->error_task_list);
+	dev->dump.size += et->size;
+	dev->dump.num_tasks++;
+
+	dev_info(dev->dev, "save error task state success\n");
+
+out:
+	mutex_unlock(&dev->error_task_list_lock);
+}
+
 static void lima_sched_timedout_job(struct drm_sched_job *job)
 {
 	struct lima_sched_pipe *pipe = to_lima_pipe(job->sched);
@@ -268,6 +395,8 @@ static void lima_sched_timedout_job(struct drm_sched_job *job)
 
 	drm_sched_increase_karma(&task->base);
 
+	lima_sched_build_error_task_list(task);
+
 	pipe->task_error(pipe);
 
 	if (pipe->bcast_mmu)
diff --git a/drivers/gpu/drm/lima/lima_sched.h b/drivers/gpu/drm/lima/lima_sched.h
index d64393fb50a9..e29f5e3b675b 100644
--- a/drivers/gpu/drm/lima/lima_sched.h
+++ b/drivers/gpu/drm/lima/lima_sched.h
@@ -5,9 +5,16 @@
 #define __LIMA_SCHED_H__
 
 #include <drm/gpu_scheduler.h>
+#include <linux/list.h>
 
 struct lima_vm;
 
+struct lima_sched_error_task {
+	struct list_head list;
+	void *data;
+	uint32_t size;
+};
+
 struct lima_sched_task {
 	struct drm_sched_job base;
 
-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 0/5] drm/lima: add error debug functionality

[Vasily Khoruzhick]

On Fri, Feb 21, 2020 at 6:43 PM Qiang Yu <yuq825@gmail.com> wrote:
>
> Save task error state when it fail and export to user by
> sysfs as a binary file which can be dumped and replayed
> by lima_dump tool at:
> https://gitlab.freedesktop.org/lima/lima_dump

With v2 of Patch 3, series is:

Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>


> Qiang Yu (5):
>   drm/lima: save process info for debug usage
>   drm/lima: add max_error_tasks module parameter
>   drm/lima: save task info dump when task fail
>   drm/lima: add error sysfs to export error task dump
>   drm/lima: add LIMA_BO_FLAG_FORCE_VA
>
>  drivers/gpu/drm/lima/lima_ctx.c    |   3 +
>  drivers/gpu/drm/lima/lima_ctx.h    |   5 ++
>  drivers/gpu/drm/lima/lima_device.c |  13 +++
>  drivers/gpu/drm/lima/lima_device.h |   8 ++
>  drivers/gpu/drm/lima/lima_drv.c    | 123 +++++++++++++++++++++++++--
>  drivers/gpu/drm/lima/lima_drv.h    |   1 +
>  drivers/gpu/drm/lima/lima_dump.h   |  77 +++++++++++++++++
>  drivers/gpu/drm/lima/lima_gem.c    |   7 +-
>  drivers/gpu/drm/lima/lima_gem.h    |   4 +-
>  drivers/gpu/drm/lima/lima_sched.c  | 128 +++++++++++++++++++++++++++++
>  drivers/gpu/drm/lima/lima_sched.h  |   7 ++
>  drivers/gpu/drm/lima/lima_vm.c     |  13 ++-
>  include/uapi/drm/lima_drm.h        |   9 +-
>  13 files changed, 385 insertions(+), 13 deletions(-)
>  create mode 100644 drivers/gpu/drm/lima/lima_dump.h
>
> --
> 2.17.1
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH 0/5] drm/lima: add error debug functionality

[Qiang Yu]

Thanks for review and test. Applied patch 1~4 to drm-misc-next.
As patch 5, due to related with non-continuous command stream,
I'd like to apply it when non-continuous command stream has been
implemented which may need a more common VA interface.

Thanks,
Qiang

On Sun, Mar 15, 2020 at 12:32 PM Vasily Khoruzhick <anarsoul@gmail.com> wrote:
>
> On Fri, Feb 21, 2020 at 6:43 PM Qiang Yu <yuq825@gmail.com> wrote:
> >
> > Save task error state when it fail and export to user by
> > sysfs as a binary file which can be dumped and replayed
> > by lima_dump tool at:
> > https://gitlab.freedesktop.org/lima/lima_dump
>
> With v2 of Patch 3, series is:
>
> Reviewed-by: Vasily Khoruzhick <anarsoul@gmail.com>
>
>
> > Qiang Yu (5):
> >   drm/lima: save process info for debug usage
> >   drm/lima: add max_error_tasks module parameter
> >   drm/lima: save task info dump when task fail
> >   drm/lima: add error sysfs to export error task dump
> >   drm/lima: add LIMA_BO_FLAG_FORCE_VA
> >
> >  drivers/gpu/drm/lima/lima_ctx.c    |   3 +
> >  drivers/gpu/drm/lima/lima_ctx.h    |   5 ++
> >  drivers/gpu/drm/lima/lima_device.c |  13 +++
> >  drivers/gpu/drm/lima/lima_device.h |   8 ++
> >  drivers/gpu/drm/lima/lima_drv.c    | 123 +++++++++++++++++++++++++--
> >  drivers/gpu/drm/lima/lima_drv.h    |   1 +
> >  drivers/gpu/drm/lima/lima_dump.h   |  77 +++++++++++++++++
> >  drivers/gpu/drm/lima/lima_gem.c    |   7 +-
> >  drivers/gpu/drm/lima/lima_gem.h    |   4 +-
> >  drivers/gpu/drm/lima/lima_sched.c  | 128 +++++++++++++++++++++++++++++
> >  drivers/gpu/drm/lima/lima_sched.h  |   7 ++
> >  drivers/gpu/drm/lima/lima_vm.c     |  13 ++-
> >  include/uapi/drm/lima_drm.h        |   9 +-
> >  13 files changed, 385 insertions(+), 13 deletions(-)
> >  create mode 100644 drivers/gpu/drm/lima/lima_dump.h
> >
> > --
> > 2.17.1
> >
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel