PRT support for amdgpu v3

PRT support for amdgpu v3

[Christian König]

Hi guys,

ok I finally found time to write an unit test for this and hammered out the last few bugs.

Seems to work fine on my Tonga now. Please note that this set is based on "fix race in GEM VA map IOCTL v2", without that patch you will run into a NULL pointer dereference during PRT mapping.

Going to send out the unit test in a minute.

Regards,
Christian.

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/6] drm/amdgpu: add support for BO_VAs without BO v2

[Christian König]

From: Christian König <christian.koenig@amd.com>

For PRT support we need mappings which aren't backed by any memory.

v2: fix parameter checking

Signed-off-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 20 ++++++++++++++------
 1 file changed, 14 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 8e6030d..64f04c7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1117,7 +1117,7 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev,
 	struct fence *exclusive;
 	int r;
 
-	if (clear) {
+	if (clear || !bo_va->bo) {
 		mem = NULL;
 		nodes = NULL;
 		exclusive = NULL;
@@ -1134,9 +1134,15 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev,
 		exclusive = reservation_object_get_excl(bo_va->bo->tbo.resv);
 	}
 
-	flags = amdgpu_ttm_tt_pte_flags(adev, bo_va->bo->tbo.ttm, mem);
-	gtt_flags = (amdgpu_ttm_is_bound(bo_va->bo->tbo.ttm) &&
-		adev == amdgpu_ttm_adev(bo_va->bo->tbo.bdev)) ? flags : 0;
+	if (bo_va->bo) {
+		flags = amdgpu_ttm_tt_pte_flags(adev, bo_va->bo->tbo.ttm, mem);
+		gtt_flags = (amdgpu_ttm_is_bound(bo_va->bo->tbo.ttm) &&
+			adev == amdgpu_ttm_adev(bo_va->bo->tbo.bdev)) ?
+			flags : 0;
+	} else {
+		flags = 0x0;
+		gtt_flags = ~0x0;
+	}
 
 	spin_lock(&vm->status_lock);
 	if (!list_empty(&bo_va->vm_status))
@@ -1271,7 +1277,8 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
 	INIT_LIST_HEAD(&bo_va->invalids);
 	INIT_LIST_HEAD(&bo_va->vm_status);
 
-	list_add_tail(&bo_va->bo_list, &bo->va);
+	if (bo)
+		list_add_tail(&bo_va->bo_list, &bo->va);
 
 	return bo_va;
 }
@@ -1309,7 +1316,8 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
 
 	/* make sure object fit at this offset */
 	eaddr = saddr + size - 1;
-	if ((saddr >= eaddr) || (offset + size > amdgpu_bo_size(bo_va->bo)))
+	if (saddr >= eaddr ||
+	    (bo_va->bo && offset + size > amdgpu_bo_size(bo_va->bo)))
 		return -EINVAL;
 
 	last_pfn = eaddr / AMDGPU_GPU_PAGE_SIZE;
-- 
2.5.0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 2/6] drm/amdgpu: add basic PRT support

[Christian König]

From: Christian König <christian.koenig@amd.com>

Future hardware generations can handle PRT flags on a per page basis,
but current hardware can only turn it on globally.

Add the basic handling for both, a global callback to enable/disable
triggered by setting a per mapping flag.

Signed-off-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h    |   2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 101 +++++++++++++++++++++++++++++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |   6 ++
 3 files changed, 98 insertions(+), 11 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 402a895..34a971a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -296,6 +296,8 @@ struct amdgpu_gart_funcs {
 			   uint32_t gpu_page_idx, /* pte/pde to update */
 			   uint64_t addr, /* addr to write into pte/pde */
 			   uint32_t flags); /* access flags */
+	/* enable/disable PRT support */
+	void (*set_prt)(struct amdgpu_device *adev, bool enable);
 };
 
 /* provided by the ih block */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 64f04c7..bc32239 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -69,6 +69,12 @@ struct amdgpu_pte_update_params {
 	bool shadow;
 };
 
+/* Helper to disable partial resident texture feature from a fence callback */
+struct amdgpu_prt_cb {
+	struct amdgpu_device *adev;
+	struct fence_cb cb;
+};
+
 /**
  * amdgpu_vm_num_pde - return the number of page directory entries
  *
@@ -989,11 +995,8 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
 		goto error_free;
 
 	amdgpu_bo_fence(vm->page_directory, f, true);
-	if (fence) {
-		fence_put(*fence);
-		*fence = fence_get(f);
-	}
-	fence_put(f);
+	fence_put(*fence);
+	*fence = f;
 	return 0;
 
 error_free:
@@ -1177,6 +1180,61 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev,
 }
 
 /**
+ * amdgpu_vm_update_prt_state - update the global PRT state
+ */
+static void amdgpu_vm_update_prt_state(struct amdgpu_device *adev)
+{
+	unsigned long flags;
+	bool enable;
+
+	spin_lock_irqsave(&adev->vm_manager.prt_lock, flags);
+	enable = !!atomic_read(&adev->vm_manager.num_prt_mappings);
+	adev->gart.gart_funcs->set_prt(adev, enable);
+	spin_unlock_irqrestore(&adev->vm_manager.prt_lock, flags);
+}
+
+/**
+ * amdgpu_vm_prt - callback for updating the PRT status
+ */
+static void amdgpu_vm_prt_cb(struct fence *fence, struct fence_cb *_cb)
+{
+	struct amdgpu_prt_cb *cb = container_of(_cb, struct amdgpu_prt_cb, cb);
+
+	amdgpu_vm_update_prt_state(cb->adev);
+	kfree(cb);
+}
+
+/**
+ * amdgpu_vm_free_mapping - free a mapping
+ *
+ * @adev: amdgpu_device pointer
+ * @vm: requested vm
+ * @mapping: mapping to be freed
+ * @fence: fence of the unmap operation
+ *
+ * Free a mapping and make sure we decrease the PRT usage count if applicable.
+ */
+static void amdgpu_vm_free_mapping(struct amdgpu_device *adev,
+				   struct amdgpu_vm *vm,
+				   struct amdgpu_bo_va_mapping *mapping,
+				   struct fence *fence)
+{
+	if ((mapping->flags & AMDGPU_PTE_PRT) &&
+	    atomic_dec_return(&adev->vm_manager.num_prt_mappings) == 0) {
+		struct amdgpu_prt_cb *cb = kmalloc(sizeof(struct amdgpu_prt_cb),
+						   GFP_KERNEL);
+
+		cb->adev = adev;
+		if (!fence || fence_add_callback(fence, &cb->cb,
+						 amdgpu_vm_prt_cb)) {
+			amdgpu_vm_update_prt_state(adev);
+			kfree(cb);
+		}
+	}
+	kfree(mapping);
+}
+
+/**
  * amdgpu_vm_clear_freed - clear freed BOs in the PT
  *
  * @adev: amdgpu_device pointer
@@ -1191,6 +1249,7 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
 			  struct amdgpu_vm *vm)
 {
 	struct amdgpu_bo_va_mapping *mapping;
+	struct fence *fence = NULL;
 	int r;
 
 	while (!list_empty(&vm->freed)) {
@@ -1199,12 +1258,15 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
 		list_del(&mapping->list);
 
 		r = amdgpu_vm_bo_split_mapping(adev, NULL, 0, NULL, vm, mapping,
-					       0, 0, NULL);
-		kfree(mapping);
-		if (r)
+					       0, 0, &fence);
+		amdgpu_vm_free_mapping(adev, vm, mapping, fence);
+		if (r) {
+			fence_put(fence);
 			return r;
+		}
 
 	}
+	fence_put(fence);
 	return 0;
 
 }
@@ -1314,6 +1376,15 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
 	    size == 0 || size & AMDGPU_GPU_PAGE_MASK)
 		return -EINVAL;
 
+	if (flags & AMDGPU_PTE_PRT) {
+		/* Check if we have PRT hardware support */
+		if (!adev->gart.gart_funcs->set_prt)
+			return -EINVAL;
+
+		if (atomic_inc_return(&adev->vm_manager.num_prt_mappings) == 1)
+			amdgpu_vm_update_prt_state(adev);
+	}
+
 	/* make sure object fit at this offset */
 	eaddr = saddr + size - 1;
 	if (saddr >= eaddr ||
@@ -1400,7 +1471,7 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
 	list_del(&mapping->list);
 	interval_tree_remove(&mapping->it, &vm->va);
 	trace_amdgpu_vm_bo_unmap(bo_va, mapping);
-	kfree(mapping);
+	amdgpu_vm_free_mapping(adev, vm, mapping, NULL);
 
 error:
 	return r;
@@ -1452,7 +1523,8 @@ int amdgpu_vm_bo_unmap(struct amdgpu_device *adev,
 	if (valid)
 		list_add(&mapping->list, &vm->freed);
 	else
-		kfree(mapping);
+		amdgpu_vm_free_mapping(adev, vm, mapping,
+				       bo_va->last_pt_update);
 
 	return 0;
 }
@@ -1488,7 +1560,8 @@ void amdgpu_vm_bo_rmv(struct amdgpu_device *adev,
 	list_for_each_entry_safe(mapping, next, &bo_va->invalids, list) {
 		list_del(&mapping->list);
 		interval_tree_remove(&mapping->it, &vm->va);
-		kfree(mapping);
+		amdgpu_vm_free_mapping(adev, vm, mapping,
+				       bo_va->last_pt_update);
 	}
 
 	fence_put(bo_va->last_pt_update);
@@ -1625,9 +1698,13 @@ void amdgpu_vm_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)
 		kfree(mapping);
 	}
 	list_for_each_entry_safe(mapping, tmp, &vm->freed, list) {
+		if (mapping->flags & AMDGPU_PTE_PRT)
+			continue;
+
 		list_del(&mapping->list);
 		kfree(mapping);
 	}
+	amdgpu_vm_clear_freed(adev, vm);
 
 	for (i = 0; i < amdgpu_vm_num_pdes(adev); i++) {
 		struct amdgpu_bo *pt = vm->page_tables[i].bo;
@@ -1672,6 +1749,8 @@ void amdgpu_vm_manager_init(struct amdgpu_device *adev)
 
 	atomic_set(&adev->vm_manager.vm_pte_next_ring, 0);
 	atomic64_set(&adev->vm_manager.client_counter, 0);
+	spin_lock_init(&adev->vm_manager.prt_lock);
+	atomic_set(&adev->vm_manager.num_prt_mappings, 0);
 }
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
index 1f99715..4d26e9b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
@@ -65,6 +65,8 @@ struct amdgpu_bo_list_entry;
 
 #define AMDGPU_PTE_FRAG(x)	((x & 0x1f) << 7)
 
+#define AMDGPU_PTE_PRT		(1UL << 63)
+
 /* How to programm VM fault handling */
 #define AMDGPU_VM_FAULT_STOP_NEVER	0
 #define AMDGPU_VM_FAULT_STOP_FIRST	1
@@ -159,6 +161,10 @@ struct amdgpu_vm_manager {
 	atomic_t				vm_pte_next_ring;
 	/* client id counter */
 	atomic64_t				client_counter;
+
+	/* partial resident texture handling */
+	spinlock_t				prt_lock;
+	atomic_t				num_prt_mappings;
 };
 
 void amdgpu_vm_manager_init(struct amdgpu_device *adev);
-- 
2.5.0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 3/6] drm/amdgpu: IOCTL interface for PRT support v4

[Christian König]

From: Junwei Zhang <Jerry.Zhang@amd.com>

Till GFX8 we can only enable PRT support globally, but with the next hardware
generation we can do this on a per page basis.

Keep the interface consistent by adding PRT mappings and enable
support globally on current hardware when the first mapping is made.

v2: disable PRT support delayed and on all error paths
v3: PRT and other permissions are mutal exclusive,
    PRT mappings don't need a BO.
v4: update PRT mappings durign CS as well, make va_flags 64bit

Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h     |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c  | 16 +++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 62 ++++++++++++++++++++-------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 10 ++++++
 include/uapi/drm/amdgpu_drm.h           |  2 ++
 5 files changed, 64 insertions(+), 27 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 34a971a..99ca5e8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -703,6 +703,7 @@ void amdgpu_ctx_mgr_fini(struct amdgpu_ctx_mgr *mgr);
 
 struct amdgpu_fpriv {
 	struct amdgpu_vm	vm;
+	struct amdgpu_bo_va	*prt_va;
 	struct mutex		bo_list_lock;
 	struct idr		bo_list_handles;
 	struct amdgpu_ctx_mgr	ctx_mgr;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 6e948e4..433254b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -758,10 +758,11 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser, int error, bo
 	amdgpu_bo_unref(&parser->uf_entry.robj);
 }
 
-static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p,
-				   struct amdgpu_vm *vm)
+static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p)
 {
 	struct amdgpu_device *adev = p->adev;
+	struct amdgpu_fpriv *fpriv = p->filp->driver_priv;
+	struct amdgpu_vm *vm = &fpriv->vm;
 	struct amdgpu_bo_va *bo_va;
 	struct amdgpu_bo *bo;
 	int i, r;
@@ -778,6 +779,15 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p,
 	if (r)
 		return r;
 
+	r = amdgpu_vm_bo_update(adev, fpriv->prt_va, false);
+	if (r)
+		return r;
+
+	r = amdgpu_sync_fence(adev, &p->job->sync,
+			      fpriv->prt_va->last_pt_update);
+	if (r)
+		return r;
+
 	if (amdgpu_sriov_vf(adev)) {
 		struct fence *f;
 		bo_va = vm->csa_bo_va;
@@ -854,7 +864,7 @@ static int amdgpu_cs_ib_vm_chunk(struct amdgpu_device *adev,
 	if (p->job->vm) {
 		p->job->vm_pd_addr = amdgpu_bo_gpu_offset(vm->page_directory);
 
-		r = amdgpu_bo_vm_update_pte(p, vm);
+		r = amdgpu_bo_vm_update_pte(p);
 		if (r)
 			return r;
 	}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index 1dc59aa..7f530f2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -540,6 +540,12 @@ static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
 int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
 			  struct drm_file *filp)
 {
+	const uint32_t valid_flags = AMDGPU_VM_DELAY_UPDATE |
+		AMDGPU_VM_PAGE_READABLE | AMDGPU_VM_PAGE_WRITEABLE |
+		AMDGPU_VM_PAGE_EXECUTABLE;
+	const uint32_t prt_flags = AMDGPU_VM_DELAY_UPDATE |
+		AMDGPU_VM_PAGE_PRT;
+
 	struct drm_amdgpu_gem_va *args = data;
 	struct drm_gem_object *gobj;
 	struct amdgpu_device *adev = dev->dev_private;
@@ -550,7 +556,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
 	struct ttm_validate_buffer tv;
 	struct ww_acquire_ctx ticket;
 	struct list_head list;
-	uint32_t invalid_flags, va_flags = 0;
+	uint64_t va_flags = 0;
 	int r = 0;
 
 	if (!adev->vm_manager.enabled)
@@ -564,11 +570,9 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
-	invalid_flags = ~(AMDGPU_VM_DELAY_UPDATE | AMDGPU_VM_PAGE_READABLE |
-			AMDGPU_VM_PAGE_WRITEABLE | AMDGPU_VM_PAGE_EXECUTABLE);
-	if ((args->flags & invalid_flags)) {
-		dev_err(&dev->pdev->dev, "invalid flags 0x%08X vs 0x%08X\n",
-			args->flags, invalid_flags);
+	if ((args->flags & ~valid_flags) && (args->flags & ~prt_flags)) {
+		dev_err(&dev->pdev->dev, "invalid flags combination 0x%08X\n",
+			args->flags);
 		return -EINVAL;
 	}
 
@@ -582,28 +586,34 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
 		return -EINVAL;
 	}
 
-	gobj = drm_gem_object_lookup(filp, args->handle);
-	if (gobj == NULL)
-		return -ENOENT;
-	abo = gem_to_amdgpu_bo(gobj);
 	INIT_LIST_HEAD(&list);
-	tv.bo = &abo->tbo;
-	tv.shared = false;
-	list_add(&tv.head, &list);
+	if (!(args->flags & AMDGPU_VM_PAGE_PRT)) {
+		gobj = drm_gem_object_lookup(filp, args->handle);
+		if (gobj == NULL)
+			return -ENOENT;
+		abo = gem_to_amdgpu_bo(gobj);
+		tv.bo = &abo->tbo;
+		tv.shared = false;
+		list_add(&tv.head, &list);
+	} else {
+		gobj = NULL;
+		abo = NULL;
+	}
 
 	amdgpu_vm_get_pd_bo(&fpriv->vm, &list, &vm_pd);
 
 	r = ttm_eu_reserve_buffers(&ticket, &list, true, NULL);
-	if (r) {
-		drm_gem_object_unreference_unlocked(gobj);
-		return r;
-	}
+	if (r)
+		goto error_unref;
 
-	bo_va = amdgpu_vm_bo_find(&fpriv->vm, abo);
-	if (!bo_va) {
-		ttm_eu_backoff_reservation(&ticket, &list);
-		drm_gem_object_unreference_unlocked(gobj);
-		return -ENOENT;
+	if (abo) {
+		bo_va = amdgpu_vm_bo_find(&fpriv->vm, abo);
+		if (!bo_va) {
+			r = -ENOENT;
+			goto error_backoff;
+		}
+	} else {
+		bo_va = fpriv->prt_va;
 	}
 
 	switch (args->operation) {
@@ -614,6 +624,8 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
 			va_flags |= AMDGPU_PTE_WRITEABLE;
 		if (args->flags & AMDGPU_VM_PAGE_EXECUTABLE)
 			va_flags |= AMDGPU_PTE_EXECUTABLE;
+		if (args->flags & AMDGPU_VM_PAGE_PRT)
+			va_flags |= AMDGPU_PTE_PRT;
 		r = amdgpu_vm_bo_map(adev, bo_va, args->va_address,
 				     args->offset_in_bo, args->map_size,
 				     va_flags);
@@ -624,11 +636,13 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
 	default:
 		break;
 	}
-	if (!r && !(args->flags & AMDGPU_VM_DELAY_UPDATE) &&
-	    !amdgpu_vm_debug)
+	if (!r && !(args->flags & AMDGPU_VM_DELAY_UPDATE) && !amdgpu_vm_debug)
 		amdgpu_gem_va_update_vm(adev, bo_va, &list, args->operation);
+
+error_backoff:
 	ttm_eu_backoff_reservation(&ticket, &list);
 
+error_unref:
 	drm_gem_object_unreference_unlocked(gobj);
 	return r;
 }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index 215f73b..d5f9d6a4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -656,6 +656,14 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
 		goto out_suspend;
 	}
 
+	fpriv->prt_va = amdgpu_vm_bo_add(adev, &fpriv->vm, NULL);
+	if (!fpriv->prt_va) {
+		r = -ENOMEM;
+		amdgpu_vm_fini(adev, &fpriv->vm);
+		kfree(fpriv);
+		goto out_suspend;
+	}
+
 	if (amdgpu_sriov_vf(adev)) {
 		r = amdgpu_map_static_csa(adev, &fpriv->vm);
 		if (r)
@@ -700,6 +708,8 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
 	amdgpu_uvd_free_handles(adev, file_priv);
 	amdgpu_vce_free_handles(adev, file_priv);
 
+	amdgpu_vm_bo_rmv(adev, fpriv->prt_va);
+
 	if (amdgpu_sriov_vf(adev)) {
 		/* TODO: how to handle reserve failure */
 		BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, false));
diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
index 2cf8df8..07e3710 100644
--- a/include/uapi/drm/amdgpu_drm.h
+++ b/include/uapi/drm/amdgpu_drm.h
@@ -363,6 +363,8 @@ struct drm_amdgpu_gem_op {
 #define AMDGPU_VM_PAGE_WRITEABLE	(1 << 2)
 /* executable mapping, new for VI */
 #define AMDGPU_VM_PAGE_EXECUTABLE	(1 << 3)
+/* partially resident texture */
+#define AMDGPU_VM_PAGE_PRT		(1 << 4)
 
 struct drm_amdgpu_gem_va {
 	/** GEM object handle */
-- 
2.5.0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 3/6] drm/amdgpu: IOCTL interface for PRT support v4

[Nicolai Hähnle]

On 08.02.2017 16:04, Christian König wrote:
> From: Junwei Zhang <Jerry.Zhang@amd.com>
>
> Till GFX8 we can only enable PRT support globally, but with the next hardware
> generation we can do this on a per page basis.
>
> Keep the interface consistent by adding PRT mappings and enable
> support globally on current hardware when the first mapping is made.
>
> v2: disable PRT support delayed and on all error paths
> v3: PRT and other permissions are mutal exclusive,
>     PRT mappings don't need a BO.
> v4: update PRT mappings durign CS as well, make va_flags 64bit

*during

Series looks good to me:

Reviewed-by: Nicolai Hähnle <nicolai.haehnle@amd.com>



> Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
> Signed-off-by: Christian König <christian.koenig@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu.h     |  1 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c  | 16 +++++++--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 62 ++++++++++++++++++++-------------
>  drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 10 ++++++
>  include/uapi/drm/amdgpu_drm.h           |  2 ++
>  5 files changed, 64 insertions(+), 27 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 34a971a..99ca5e8 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -703,6 +703,7 @@ void amdgpu_ctx_mgr_fini(struct amdgpu_ctx_mgr *mgr);
>
>  struct amdgpu_fpriv {
>  	struct amdgpu_vm	vm;
> +	struct amdgpu_bo_va	*prt_va;
>  	struct mutex		bo_list_lock;
>  	struct idr		bo_list_handles;
>  	struct amdgpu_ctx_mgr	ctx_mgr;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> index 6e948e4..433254b 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
> @@ -758,10 +758,11 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser, int error, bo
>  	amdgpu_bo_unref(&parser->uf_entry.robj);
>  }
>
> -static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p,
> -				   struct amdgpu_vm *vm)
> +static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p)
>  {
>  	struct amdgpu_device *adev = p->adev;
> +	struct amdgpu_fpriv *fpriv = p->filp->driver_priv;
> +	struct amdgpu_vm *vm = &fpriv->vm;
>  	struct amdgpu_bo_va *bo_va;
>  	struct amdgpu_bo *bo;
>  	int i, r;
> @@ -778,6 +779,15 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p,
>  	if (r)
>  		return r;
>
> +	r = amdgpu_vm_bo_update(adev, fpriv->prt_va, false);
> +	if (r)
> +		return r;
> +
> +	r = amdgpu_sync_fence(adev, &p->job->sync,
> +			      fpriv->prt_va->last_pt_update);
> +	if (r)
> +		return r;
> +
>  	if (amdgpu_sriov_vf(adev)) {
>  		struct fence *f;
>  		bo_va = vm->csa_bo_va;
> @@ -854,7 +864,7 @@ static int amdgpu_cs_ib_vm_chunk(struct amdgpu_device *adev,
>  	if (p->job->vm) {
>  		p->job->vm_pd_addr = amdgpu_bo_gpu_offset(vm->page_directory);
>
> -		r = amdgpu_bo_vm_update_pte(p, vm);
> +		r = amdgpu_bo_vm_update_pte(p);
>  		if (r)
>  			return r;
>  	}
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> index 1dc59aa..7f530f2 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
> @@ -540,6 +540,12 @@ static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev,
>  int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
>  			  struct drm_file *filp)
>  {
> +	const uint32_t valid_flags = AMDGPU_VM_DELAY_UPDATE |
> +		AMDGPU_VM_PAGE_READABLE | AMDGPU_VM_PAGE_WRITEABLE |
> +		AMDGPU_VM_PAGE_EXECUTABLE;
> +	const uint32_t prt_flags = AMDGPU_VM_DELAY_UPDATE |
> +		AMDGPU_VM_PAGE_PRT;
> +
>  	struct drm_amdgpu_gem_va *args = data;
>  	struct drm_gem_object *gobj;
>  	struct amdgpu_device *adev = dev->dev_private;
> @@ -550,7 +556,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
>  	struct ttm_validate_buffer tv;
>  	struct ww_acquire_ctx ticket;
>  	struct list_head list;
> -	uint32_t invalid_flags, va_flags = 0;
> +	uint64_t va_flags = 0;
>  	int r = 0;
>
>  	if (!adev->vm_manager.enabled)
> @@ -564,11 +570,9 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
>  		return -EINVAL;
>  	}
>
> -	invalid_flags = ~(AMDGPU_VM_DELAY_UPDATE | AMDGPU_VM_PAGE_READABLE |
> -			AMDGPU_VM_PAGE_WRITEABLE | AMDGPU_VM_PAGE_EXECUTABLE);
> -	if ((args->flags & invalid_flags)) {
> -		dev_err(&dev->pdev->dev, "invalid flags 0x%08X vs 0x%08X\n",
> -			args->flags, invalid_flags);
> +	if ((args->flags & ~valid_flags) && (args->flags & ~prt_flags)) {
> +		dev_err(&dev->pdev->dev, "invalid flags combination 0x%08X\n",
> +			args->flags);
>  		return -EINVAL;
>  	}
>
> @@ -582,28 +586,34 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
>  		return -EINVAL;
>  	}
>
> -	gobj = drm_gem_object_lookup(filp, args->handle);
> -	if (gobj == NULL)
> -		return -ENOENT;
> -	abo = gem_to_amdgpu_bo(gobj);
>  	INIT_LIST_HEAD(&list);
> -	tv.bo = &abo->tbo;
> -	tv.shared = false;
> -	list_add(&tv.head, &list);
> +	if (!(args->flags & AMDGPU_VM_PAGE_PRT)) {
> +		gobj = drm_gem_object_lookup(filp, args->handle);
> +		if (gobj == NULL)
> +			return -ENOENT;
> +		abo = gem_to_amdgpu_bo(gobj);
> +		tv.bo = &abo->tbo;
> +		tv.shared = false;
> +		list_add(&tv.head, &list);
> +	} else {
> +		gobj = NULL;
> +		abo = NULL;
> +	}
>
>  	amdgpu_vm_get_pd_bo(&fpriv->vm, &list, &vm_pd);
>
>  	r = ttm_eu_reserve_buffers(&ticket, &list, true, NULL);
> -	if (r) {
> -		drm_gem_object_unreference_unlocked(gobj);
> -		return r;
> -	}
> +	if (r)
> +		goto error_unref;
>
> -	bo_va = amdgpu_vm_bo_find(&fpriv->vm, abo);
> -	if (!bo_va) {
> -		ttm_eu_backoff_reservation(&ticket, &list);
> -		drm_gem_object_unreference_unlocked(gobj);
> -		return -ENOENT;
> +	if (abo) {
> +		bo_va = amdgpu_vm_bo_find(&fpriv->vm, abo);
> +		if (!bo_va) {
> +			r = -ENOENT;
> +			goto error_backoff;
> +		}
> +	} else {
> +		bo_va = fpriv->prt_va;
>  	}
>
>  	switch (args->operation) {
> @@ -614,6 +624,8 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
>  			va_flags |= AMDGPU_PTE_WRITEABLE;
>  		if (args->flags & AMDGPU_VM_PAGE_EXECUTABLE)
>  			va_flags |= AMDGPU_PTE_EXECUTABLE;
> +		if (args->flags & AMDGPU_VM_PAGE_PRT)
> +			va_flags |= AMDGPU_PTE_PRT;
>  		r = amdgpu_vm_bo_map(adev, bo_va, args->va_address,
>  				     args->offset_in_bo, args->map_size,
>  				     va_flags);
> @@ -624,11 +636,13 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data,
>  	default:
>  		break;
>  	}
> -	if (!r && !(args->flags & AMDGPU_VM_DELAY_UPDATE) &&
> -	    !amdgpu_vm_debug)
> +	if (!r && !(args->flags & AMDGPU_VM_DELAY_UPDATE) && !amdgpu_vm_debug)
>  		amdgpu_gem_va_update_vm(adev, bo_va, &list, args->operation);
> +
> +error_backoff:
>  	ttm_eu_backoff_reservation(&ticket, &list);
>
> +error_unref:
>  	drm_gem_object_unreference_unlocked(gobj);
>  	return r;
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> index 215f73b..d5f9d6a4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> @@ -656,6 +656,14 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
>  		goto out_suspend;
>  	}
>
> +	fpriv->prt_va = amdgpu_vm_bo_add(adev, &fpriv->vm, NULL);
> +	if (!fpriv->prt_va) {
> +		r = -ENOMEM;
> +		amdgpu_vm_fini(adev, &fpriv->vm);
> +		kfree(fpriv);
> +		goto out_suspend;
> +	}
> +
>  	if (amdgpu_sriov_vf(adev)) {
>  		r = amdgpu_map_static_csa(adev, &fpriv->vm);
>  		if (r)
> @@ -700,6 +708,8 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
>  	amdgpu_uvd_free_handles(adev, file_priv);
>  	amdgpu_vce_free_handles(adev, file_priv);
>
> +	amdgpu_vm_bo_rmv(adev, fpriv->prt_va);
> +
>  	if (amdgpu_sriov_vf(adev)) {
>  		/* TODO: how to handle reserve failure */
>  		BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, false));
> diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_drm.h
> index 2cf8df8..07e3710 100644
> --- a/include/uapi/drm/amdgpu_drm.h
> +++ b/include/uapi/drm/amdgpu_drm.h
> @@ -363,6 +363,8 @@ struct drm_amdgpu_gem_op {
>  #define AMDGPU_VM_PAGE_WRITEABLE	(1 << 2)
>  /* executable mapping, new for VI */
>  #define AMDGPU_VM_PAGE_EXECUTABLE	(1 << 3)
> +/* partially resident texture */
> +#define AMDGPU_VM_PAGE_PRT		(1 << 4)
>
>  struct drm_amdgpu_gem_va {
>  	/** GEM object handle */
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 4/6] drm/amdgpu: implement PRT for GFX6 v2

[Christian König]

From: Christian König <christian.koenig@amd.com>

Enable/disable the handling globally for now and
print a warning when we enable it for the first time.

v2: write to the correct register, adjust bits to that hw generation
v3: fix compilation, add the missing register bit definitions

Signed-off-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h                |  1 +
 drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c              | 55 ++++++++++++++++++++++
 .../drm/amd/include/asic_reg/gmc/gmc_6_0_sh_mask.h |  4 ++
 3 files changed, 60 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 99ca5e8..d8516dd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -571,6 +571,7 @@ struct amdgpu_mc {
 	uint32_t		vram_type;
 	uint32_t                srbm_soft_reset;
 	struct amdgpu_mode_mc_save save;
+	bool			prt_warning;
 };
 
 /*
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
index e2b0b16..b9b5c24 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
@@ -398,6 +398,60 @@ static void gmc_v6_0_set_fault_enable_default(struct amdgpu_device *adev,
 	WREG32(mmVM_CONTEXT1_CNTL, tmp);
 }
 
+ /**
+   + * gmc_v8_0_set_prt - set PRT VM fault
+   + *
+   + * @adev: amdgpu_device pointer
+   + * @enable: enable/disable VM fault handling for PRT
+   +*/
+static void gmc_v6_0_set_prt(struct amdgpu_device *adev, bool enable)
+{
+	u32 tmp;
+
+	if (enable && !adev->mc.prt_warning) {
+		dev_warn(adev->dev, "Disabling VM faults because of PRT request!\n");
+		adev->mc.prt_warning = true;
+	}
+
+	tmp = RREG32(mmVM_PRT_CNTL);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    CB_DISABLE_FAULT_ON_UNMAPPED_ACCESS,
+			    enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    TC_DISABLE_FAULT_ON_UNMAPPED_ACCESS,
+			    enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    L2_CACHE_STORE_INVALID_ENTRIES,
+			    enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    L1_TLB_STORE_INVALID_ENTRIES,
+			    enable);
+	WREG32(mmVM_PRT_CNTL, tmp);
+
+	if (enable) {
+		uint32_t low = AMDGPU_VA_RESERVED_SIZE >> AMDGPU_GPU_PAGE_SHIFT;
+		uint32_t high = adev->vm_manager.max_pfn;
+
+		WREG32(mmVM_PRT_APERTURE0_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE1_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE2_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE3_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE0_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE1_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE2_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE3_HIGH_ADDR, high);
+	} else {
+		WREG32(mmVM_PRT_APERTURE0_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE1_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE2_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE3_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE0_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE1_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE2_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE3_HIGH_ADDR, 0x0);
+	}
+}
+
 static int gmc_v6_0_gart_enable(struct amdgpu_device *adev)
 {
 	int r, i;
@@ -1080,6 +1134,7 @@ static const struct amd_ip_funcs gmc_v6_0_ip_funcs = {
 static const struct amdgpu_gart_funcs gmc_v6_0_gart_funcs = {
 	.flush_gpu_tlb = gmc_v6_0_gart_flush_gpu_tlb,
 	.set_pte_pde = gmc_v6_0_gart_set_pte_pde,
+	.set_prt = gmc_v6_0_set_prt,
 };
 
 static const struct amdgpu_irq_src_funcs gmc_v6_0_irq_funcs = {
diff --git a/drivers/gpu/drm/amd/include/asic_reg/gmc/gmc_6_0_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/gmc/gmc_6_0_sh_mask.h
index 0f6c6c8..7155312 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/gmc/gmc_6_0_sh_mask.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/gmc/gmc_6_0_sh_mask.h
@@ -11891,5 +11891,9 @@
 #define VM_PRT_CNTL__L1_TLB_STORE_INVALID_ENTRIES__SHIFT 0x00000003
 #define VM_PRT_CNTL__L2_CACHE_STORE_INVALID_ENTRIES_MASK 0x00000004L
 #define VM_PRT_CNTL__L2_CACHE_STORE_INVALID_ENTRIES__SHIFT 0x00000002
+#define VM_PRT_CNTL__CB_DISABLE_FAULT_ON_UNMAPPED_ACCESS_MASK 0x00000001L
+#define VM_PRT_CNTL__CB_DISABLE_FAULT_ON_UNMAPPED_ACCESS__SHIFT 0x00000000
+#define VM_PRT_CNTL__TC_DISABLE_FAULT_ON_UNMAPPED_ACCESS_MASK 0x00000002L
+#define VM_PRT_CNTL__TC_DISABLE_FAULT_ON_UNMAPPED_ACCESS__SHIFT 0x00000001
 
 #endif
-- 
2.5.0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 5/6] drm/amdgpu: implement PRT for GFX7 v2

[Christian König]

From: Christian König <christian.koenig@amd.com>

Enable/disable the handling globally for now and
print a warning when we enable it for the first time.

v2: set correct register

Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 57 +++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index 8d05e0c..4b38d06 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -481,6 +481,62 @@ static void gmc_v7_0_set_fault_enable_default(struct amdgpu_device *adev,
 }
 
 /**
+ * gmc_v7_0_set_prt - set PRT VM fault
+ *
+ * @adev: amdgpu_device pointer
+ * @enable: enable/disable VM fault handling for PRT
+ */
+static void gmc_v7_0_set_prt(struct amdgpu_device *adev, bool enable)
+{
+	uint32_t tmp;
+
+	if (enable && !adev->mc.prt_warning) {
+		dev_warn(adev->dev, "Disabling VM faults because of PRT request!\n");
+		adev->mc.prt_warning = true;
+	}
+
+	tmp = RREG32(mmVM_PRT_CNTL);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    CB_DISABLE_READ_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    CB_DISABLE_WRITE_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    TC_DISABLE_READ_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    TC_DISABLE_WRITE_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    L2_CACHE_STORE_INVALID_ENTRIES, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    L1_TLB_STORE_INVALID_ENTRIES, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    MASK_PDE0_FAULT, enable);
+	WREG32(mmVM_PRT_CNTL, tmp);
+
+	if (enable) {
+		uint32_t low = AMDGPU_VA_RESERVED_SIZE >> AMDGPU_GPU_PAGE_SHIFT;
+		uint32_t high = adev->vm_manager.max_pfn;
+
+		WREG32(mmVM_PRT_APERTURE0_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE1_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE2_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE3_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE0_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE1_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE2_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE3_HIGH_ADDR, high);
+	} else {
+		WREG32(mmVM_PRT_APERTURE0_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE1_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE2_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE3_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE0_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE1_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE2_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE3_HIGH_ADDR, 0x0);
+	}
+}
+
+/**
  * gmc_v7_0_gart_enable - gart enable
  *
  * @adev: amdgpu_device pointer
@@ -1259,6 +1315,7 @@ static const struct amd_ip_funcs gmc_v7_0_ip_funcs = {
 static const struct amdgpu_gart_funcs gmc_v7_0_gart_funcs = {
 	.flush_gpu_tlb = gmc_v7_0_gart_flush_gpu_tlb,
 	.set_pte_pde = gmc_v7_0_gart_set_pte_pde,
+	.set_prt = gmc_v7_0_set_prt,
 };
 
 static const struct amdgpu_irq_src_funcs gmc_v7_0_irq_funcs = {
-- 
2.5.0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 6/6] drm/amdgpu: implement PRT for GFX8 v2

[Christian König]

From: Christian König <christian.koenig@amd.com>

Enable/disable the handling globally for now and
print a warning when we enable it for the first time.

v2: set correct register

Signed-off-by: Junwei Zhang <Jerry.Zhang@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 57 +++++++++++++++++++++++++++++++++++
 1 file changed, 57 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 7669b32..a68b850 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -595,6 +595,62 @@ static void gmc_v8_0_set_fault_enable_default(struct amdgpu_device *adev,
 }
 
 /**
+ * gmc_v8_0_set_prt - set PRT VM fault
+ *
+ * @adev: amdgpu_device pointer
+ * @enable: enable/disable VM fault handling for PRT
+*/
+static void gmc_v8_0_set_prt(struct amdgpu_device *adev, bool enable)
+{
+	u32 tmp;
+
+	if (enable && !adev->mc.prt_warning) {
+		dev_warn(adev->dev, "Disabling VM faults because of PRT request!\n");
+		adev->mc.prt_warning = true;
+	}
+
+	tmp = RREG32(mmVM_PRT_CNTL);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    CB_DISABLE_READ_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    CB_DISABLE_WRITE_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    TC_DISABLE_READ_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    TC_DISABLE_WRITE_FAULT_ON_UNMAPPED_ACCESS, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    L2_CACHE_STORE_INVALID_ENTRIES, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    L1_TLB_STORE_INVALID_ENTRIES, enable);
+	tmp = REG_SET_FIELD(tmp, VM_PRT_CNTL,
+			    MASK_PDE0_FAULT, enable);
+	WREG32(mmVM_PRT_CNTL, tmp);
+
+	if (enable) {
+		uint32_t low = AMDGPU_VA_RESERVED_SIZE >> AMDGPU_GPU_PAGE_SHIFT;
+		uint32_t high = adev->vm_manager.max_pfn;
+
+		WREG32(mmVM_PRT_APERTURE0_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE1_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE2_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE3_LOW_ADDR, low);
+		WREG32(mmVM_PRT_APERTURE0_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE1_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE2_HIGH_ADDR, high);
+		WREG32(mmVM_PRT_APERTURE3_HIGH_ADDR, high);
+	} else {
+		WREG32(mmVM_PRT_APERTURE0_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE1_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE2_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE3_LOW_ADDR, 0xfffffff);
+		WREG32(mmVM_PRT_APERTURE0_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE1_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE2_HIGH_ADDR, 0x0);
+		WREG32(mmVM_PRT_APERTURE3_HIGH_ADDR, 0x0);
+	}
+}
+
+/**
  * gmc_v8_0_gart_enable - gart enable
  *
  * @adev: amdgpu_device pointer
@@ -1485,6 +1541,7 @@ static const struct amd_ip_funcs gmc_v8_0_ip_funcs = {
 static const struct amdgpu_gart_funcs gmc_v8_0_gart_funcs = {
 	.flush_gpu_tlb = gmc_v8_0_gart_flush_gpu_tlb,
 	.set_pte_pde = gmc_v8_0_gart_set_pte_pde,
+	.set_prt = gmc_v8_0_set_prt,
 };
 
 static const struct amdgpu_irq_src_funcs gmc_v8_0_irq_funcs = {
-- 
2.5.0

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: PRT support for amdgpu v3

[Nicolai Hähnle]

On 08.02.2017 16:04, Christian König wrote:
> Hi guys,
>
> ok I finally found time to write an unit test for this and hammered out the last few bugs.
>
> Seems to work fine on my Tonga now. Please note that this set is based on "fix race in GEM VA map IOCTL v2", without that patch you will run into a NULL pointer dereference during PRT mapping.

I can confirm that it works with my Mesa series as well: I get the 
"Disabling VM faults" warning message, and indeed accessing outside the 
committed region does not cause VM faults.

Cheers,
Nicolai

>
> Going to send out the unit test in a minute.
>
> Regards,
> Christian.
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: PRT support for amdgpu v3

[Christian König]

Am 09.02.2017 um 11:11 schrieb Nicolai Hähnle:
> On 08.02.2017 16:04, Christian König wrote:
>> Hi guys,
>>
>> ok I finally found time to write an unit test for this and hammered 
>> out the last few bugs.
>>
>> Seems to work fine on my Tonga now. Please note that this set is 
>> based on "fix race in GEM VA map IOCTL v2", without that patch you 
>> will run into a NULL pointer dereference during PRT mapping.
>
> I can confirm that it works with my Mesa series as well: I get the 
> "Disabling VM faults" warning message, and indeed accessing outside 
> the committed region does not cause VM faults.

Great! Anybody who wants to give me some RBs on the patches?

I would like to commit those to the internal branches.

Regards,
Christian.

>
> Cheers,
> Nicolai
>
>>
>> Going to send out the unit test in a minute.
>>
>> Regards,
>> Christian.
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: PRT support for amdgpu v3

[Bas Nieuwenhuizen]

Tested on amd-staging-4.9 + these patches, and it works for me too.

Thanks,
Bas Nieuwenhuizen

On Wed, Feb 8, 2017, at 16:04, Christian König wrote:
> Hi guys,
> 
> ok I finally found time to write an unit test for this and hammered out
> the last few bugs.
> 
> Seems to work fine on my Tonga now. Please note that this set is based on
> "fix race in GEM VA map IOCTL v2", without that patch you will run into a
> NULL pointer dereference during PRT mapping.
> 
> Going to send out the unit test in a minute.
> 
> Regards,
> Christian.
> 
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: PRT support for amdgpu v3

[Nicolai Hähnle]

[-- Attachment #1: Type: text/plain, Size: 931 bytes --]

Hi,

Some more testing uncovered a bug in cleanup paths. When the application 
segfaults while PRT mappings exist, I get a WARN_ON (which seems fairly 
straightforward) and occasionally also an RCU error warning -- see the 
attached dmesg logs.

Regular application shutdown works fine, though.

Cheers,
Nicolai

On 08.02.2017 16:04, Christian König wrote:
> Hi guys,
>
> ok I finally found time to write an unit test for this and hammered out the last few bugs.
>
> Seems to work fine on my Tonga now. Please note that this set is based on "fix race in GEM VA map IOCTL v2", without that patch you will run into a NULL pointer dereference during PRT mapping.
>
> Going to send out the unit test in a minute.
>
> Regards,
> Christian.
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>


[-- Attachment #2: dmesg --]
[-- Type: text/plain, Size: 8190 bytes --]

Feb 12 12:07:46 cassiopeia kernel: [36172.512767] arb_sparse_buff[15881]: segfault at 10 ip 00007f295fdda14a sp 00007ffce6dd2860 error 4 in radeonsi_dri.so[7f295f57e000+b54000]
Feb 12 12:07:46 cassiopeia kernel: [36172.688430] ------------[ cut here ]------------
Feb 12 12:07:46 cassiopeia kernel: [36172.688689] WARNING: CPU: 3 PID: 15886 at drivers/gpu/drm/amd/amdgpu/amdgpu_object.c:1000 amdgpu_bo_gpu_offset+0xe0/0x1d0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.688692] Modules linked in: snd_usb_audio snd_usbmidi_lib btrfs xor raid6_pq binfmt_misc edac_mce_amd edac_core nls_iso8859_1 dm_crypt kvm_amd kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel input_leds joydev aesni_intel aes_x86_64 glue_helper lrw gf128mul ablk_helper cryptd snd_hda_codec_realtek snd_hda_codec_generic serio_raw snd_hda_codec_hdmi fam15h_power snd_hda_intel snd_hda_codec k10temp snd_hda_core snd_hwdep snd_pcm i2c_piix4 snd_seq_midi snd_seq_midi_event snd_rawmidi snd_seq snd_seq_device snd_timer snd soundcore tpm_infineon eeepc_wmi asus_wmi video mac_hid sparse_keymap mxm_wmi shpchp wmi parport_pc ppdev lp parport autofs4 amdkfd amd_iommu_v2 amdgpu i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops psmouse ttm drm ahci r8169 libahci mii fjes
Feb 12 12:07:46 cassiopeia kernel: [36172.688896]  hid_generic usbhid hid
Feb 12 12:07:46 cassiopeia kernel: [36172.688915] CPU: 3 PID: 15886 Comm: si_shader:3 Tainted: G    B           4.9.0-amd-staging-4.9-prt #133
Feb 12 12:07:46 cassiopeia kernel: [36172.688920] Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 LE R2.0, BIOS 2601 03/24/2015
Feb 12 12:07:46 cassiopeia kernel: [36172.688925]  ffff8804d952f3e0 ffffffffa06b93a7 0000000000000000 0000000000000000
Feb 12 12:07:46 cassiopeia kernel: [36172.688940]  ffff8804d952f428 ffffffffa00d25b1 ffff880560a06440 000003e800000000
Feb 12 12:07:46 cassiopeia kernel: [36172.688953]  ffff88065d4dd468 ffff88065d4dd51c ffff88065d4dcd28 0000000000001fff
Feb 12 12:07:46 cassiopeia kernel: [36172.688966] Call Trace:
Feb 12 12:07:46 cassiopeia kernel: [36172.688980]  [<ffffffffa06b93a7>] dump_stack+0x86/0xcf
Feb 12 12:07:46 cassiopeia kernel: [36172.688991]  [<ffffffffa00d25b1>] __warn+0x111/0x130
Feb 12 12:07:46 cassiopeia kernel: [36172.689002]  [<ffffffffa00d279d>] warn_slowpath_null+0x1d/0x20
Feb 12 12:07:46 cassiopeia kernel: [36172.689234]  [<ffffffffc0505d70>] amdgpu_bo_gpu_offset+0xe0/0x1d0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.689474]  [<ffffffffc052a073>] amdgpu_vm_update_ptes.isra.9+0x103/0x2c0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.689484]  [<ffffffffa035b6f5>] ? kmem_cache_alloc+0x195/0x270
Feb 12 12:07:46 cassiopeia kernel: [36172.689723]  [<ffffffffc052a351>] amdgpu_vm_frag_ptes+0x121/0x140 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.689963]  [<ffffffffc052abf6>] amdgpu_vm_bo_split_mapping+0x7d6/0xa00 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.689974]  [<ffffffffa00db778>] ? do_group_exit+0x98/0x160
Feb 12 12:07:46 cassiopeia kernel: [36172.690213]  [<ffffffffc052a420>] ? amdgpu_vm_free_mapping.isra.10+0xb0/0xb0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.690221]  [<ffffffffa00f0b49>] ? get_signal+0x3a9/0xb90
Feb 12 12:07:46 cassiopeia kernel: [36172.690230]  [<ffffffffa015fa06>] ? trace_hardirqs_on_caller+0x16/0x280
Feb 12 12:07:46 cassiopeia kernel: [36172.690238]  [<ffffffffa015fc7d>] ? trace_hardirqs_on+0xd/0x10
Feb 12 12:07:46 cassiopeia kernel: [36172.690487]  [<ffffffffc0529d10>] ? amdgpu_vm_do_copy_ptes+0x1c0/0x1c0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.690496]  [<ffffffffa035d03a>] ? kfree+0xea/0x2a0
Feb 12 12:07:46 cassiopeia kernel: [36172.690736]  [<ffffffffc052d83a>] amdgpu_vm_clear_freed+0x11a/0x200 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.690976]  [<ffffffffc052d720>] ? amdgpu_vm_bo_update+0x750/0x750 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.690986]  [<ffffffffa0360d99>] ? kasan_slab_free+0x89/0xc0
Feb 12 12:07:46 cassiopeia kernel: [36172.691225]  [<ffffffffc052f289>] ? amdgpu_vm_fini+0x259/0x430 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.691464]  [<ffffffffc052f2bc>] amdgpu_vm_fini+0x28c/0x430 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.691716]  [<ffffffffc052f030>] ? amdgpu_vm_init+0x4d0/0x4d0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.691975]  [<ffffffffc04eb053>] amdgpu_driver_postclose_kms+0x233/0x3b0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.691987]  [<ffffffffa06bb010>] ? idr_layer_rcu_free+0x20/0x20
Feb 12 12:07:46 cassiopeia kernel: [36172.692216]  [<ffffffffc04eae20>] ? amdgpu_driver_open_kms+0x1e0/0x1e0 [amdgpu]
Feb 12 12:07:46 cassiopeia kernel: [36172.692226]  [<ffffffffa0e0594e>] ? mutex_unlock+0xe/0x10
Feb 12 12:07:46 cassiopeia kernel: [36172.692313]  [<ffffffffc0330a92>] ? drm_master_release+0xf2/0x230 [drm]
Feb 12 12:07:46 cassiopeia kernel: [36172.692395]  [<ffffffffc0337892>] drm_release+0x432/0x5c0 [drm]
Feb 12 12:07:46 cassiopeia kernel: [36172.692406]  [<ffffffffa039d1a7>] __fput+0x177/0x350
Feb 12 12:07:46 cassiopeia kernel: [36172.692415]  [<ffffffffa039d3ce>] ____fput+0xe/0x10
Feb 12 12:07:46 cassiopeia kernel: [36172.692423]  [<ffffffffa010b2d0>] task_work_run+0xa0/0xc0
Feb 12 12:07:46 cassiopeia kernel: [36172.692431]  [<ffffffffa00da7ed>] do_exit+0x48d/0x1320
Feb 12 12:07:46 cassiopeia kernel: [36172.692440]  [<ffffffffa0160190>] ? debug_check_no_locks_freed+0x1c0/0x1c0
Feb 12 12:07:46 cassiopeia kernel: [36172.692449]  [<ffffffffa00da360>] ? mm_update_next_owner+0x330/0x330
Feb 12 12:07:46 cassiopeia kernel: [36172.692456]  [<ffffffffa0361ea4>] ? qlist_free_all+0x34/0xc0
Feb 12 12:07:46 cassiopeia kernel: [36172.692464]  [<ffffffffa035c6dd>] ? __slab_free+0x1ed/0x340
Feb 12 12:07:46 cassiopeia kernel: [36172.692473]  [<ffffffffa035ea07>] ? ___cache_free+0x97/0xb0
Feb 12 12:07:46 cassiopeia kernel: [36172.692480]  [<ffffffffa0361eb9>] ? qlist_free_all+0x49/0xc0
Feb 12 12:07:46 cassiopeia kernel: [36172.692487]  [<ffffffffa0362306>] ? quarantine_reduce+0x1a6/0x1e0
Feb 12 12:07:46 cassiopeia kernel: [36172.692495]  [<ffffffffa03607da>] ? kasan_kmalloc+0xca/0xe0
Feb 12 12:07:46 cassiopeia kernel: [36172.692504]  [<ffffffffa035b8d8>] ? kmem_cache_alloc_node+0x108/0x290
Feb 12 12:07:46 cassiopeia kernel: [36172.692512]  [<ffffffffa0c0ec6e>] ? __alloc_skb+0xae/0x310
Feb 12 12:07:46 cassiopeia kernel: [36172.692519]  [<ffffffffa0c0fcb7>] ? alloc_skb_with_frags+0x77/0x2b0
Feb 12 12:07:46 cassiopeia kernel: [36172.692528]  [<ffffffffa01899e6>] ? debug_lockdep_rcu_enabled+0x26/0x40
Feb 12 12:07:46 cassiopeia kernel: [36172.692537]  [<ffffffffa00db778>] do_group_exit+0x98/0x160
Feb 12 12:07:46 cassiopeia kernel: [36172.692545]  [<ffffffffa00f0b49>] get_signal+0x3a9/0xb90
Feb 12 12:07:46 cassiopeia kernel: [36172.692556]  [<ffffffffa004f41b>] do_signal+0x8b/0xc50
Feb 12 12:07:46 cassiopeia kernel: [36172.692564]  [<ffffffffa0160190>] ? debug_check_no_locks_freed+0x1c0/0x1c0
Feb 12 12:07:46 cassiopeia kernel: [36172.692573]  [<ffffffffa004f390>] ? setup_sigcontext+0x280/0x280
Feb 12 12:07:46 cassiopeia kernel: [36172.692582]  [<ffffffffa01899e6>] ? debug_lockdep_rcu_enabled+0x26/0x40
Feb 12 12:07:46 cassiopeia kernel: [36172.692590]  [<ffffffffa03a624e>] ? __set_task_comm+0x2e/0x190
Feb 12 12:07:46 cassiopeia kernel: [36172.692600]  [<ffffffffa092e010>] ? proc_ptrace_connector+0x1d0/0x1d0
Feb 12 12:07:46 cassiopeia kernel: [36172.692608]  [<ffffffffa01bfe33>] ? SyS_futex+0xe3/0x280
Feb 12 12:07:46 cassiopeia kernel: [36172.692615]  [<ffffffffa03a6293>] ? __set_task_comm+0x73/0x190
Feb 12 12:07:46 cassiopeia kernel: [36172.692623]  [<ffffffffa01bfd50>] ? do_futex+0x1030/0x1030
Feb 12 12:07:46 cassiopeia kernel: [36172.692631]  [<ffffffffa00fb255>] ? SyS_prctl+0x145/0x700
Feb 12 12:07:46 cassiopeia kernel: [36172.692641]  [<ffffffffa0004511>] exit_to_usermode_loop+0x91/0xe0
Feb 12 12:07:46 cassiopeia kernel: [36172.692650]  [<ffffffffa0005880>] syscall_return_slowpath+0x110/0x120
Feb 12 12:07:46 cassiopeia kernel: [36172.692658]  [<ffffffffa0e0a926>] entry_SYSCALL_64_fastpath+0xc4/0xc6
Feb 12 12:07:46 cassiopeia kernel: [36172.692741] ---[ end trace 3894dbe499888456 ]---

[-- Attachment #3: dmesg.2 --]
[-- Type: text/plain, Size: 14046 bytes --]

Feb 12 12:15:54 cassiopeia kernel: [  160.750710] arb_sparse_buff[3979]: segfault at 10 ip 00007f328a71e14a sp 00007fffad17e090 error 4 in radeonsi_dri.so[7f3289ec2000+b54000]
Feb 12 12:15:55 cassiopeia kernel: [  160.957647] 
Feb 12 12:15:55 cassiopeia kernel: [  160.957653] ===============================
Feb 12 12:15:55 cassiopeia kernel: [  160.957656] [ INFO: suspicious RCU usage. ]
Feb 12 12:15:55 cassiopeia kernel: [  160.957660] 4.9.0-amd-staging-4.9-prt #133 Not tainted
Feb 12 12:15:55 cassiopeia kernel: [  160.957662] -------------------------------
Feb 12 12:15:55 cassiopeia kernel: [  160.957666] ./include/linux/reservation.h:162 suspicious rcu_dereference_protected() usage!
Feb 12 12:15:55 cassiopeia kernel: [  160.957668] 
Feb 12 12:15:55 cassiopeia kernel: [  160.957668] other info that might help us debug this:
Feb 12 12:15:55 cassiopeia kernel: [  160.957668] 
Feb 12 12:15:55 cassiopeia kernel: [  160.957672] 
Feb 12 12:15:55 cassiopeia kernel: [  160.957672] rcu_scheduler_active = 1, debug_locks = 0
Feb 12 12:15:55 cassiopeia kernel: [  160.957676] 1 lock held by arb_sparse_buff/3979:
Feb 12 12:15:55 cassiopeia kernel: [  160.957679]  #0:  (drm_global_mutex){+.+.+.}, at: [<ffffffffc03774c6>] drm_release+0x66/0x5c0 [drm]
Feb 12 12:15:55 cassiopeia kernel: [  160.957723] 
Feb 12 12:15:55 cassiopeia kernel: [  160.957723] stack backtrace:
Feb 12 12:15:55 cassiopeia kernel: [  160.957728] CPU: 5 PID: 3979 Comm: arb_sparse_buff Not tainted 4.9.0-amd-staging-4.9-prt #133
Feb 12 12:15:55 cassiopeia kernel: [  160.957732] Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 LE R2.0, BIOS 2601 03/24/2015
Feb 12 12:15:55 cassiopeia kernel: [  160.957735]  ffff88067c4df4c0 ffffffffa66b93a7 ffff88069bf8cb40 0000000000000001
Feb 12 12:15:55 cassiopeia kernel: [  160.957742]  ffff88067c4df4f0 ffffffffa615dbdf ffff8806dd1ae948 0000000000000000
Feb 12 12:15:55 cassiopeia kernel: [  160.957748]  ffff8807064f0000 0000000000000000 ffff88067c4df550 ffffffffc0534119
Feb 12 12:15:55 cassiopeia kernel: [  160.957753] Call Trace:
Feb 12 12:15:55 cassiopeia kernel: [  160.957761]  [<ffffffffa66b93a7>] dump_stack+0x86/0xcf
Feb 12 12:15:55 cassiopeia kernel: [  160.957767]  [<ffffffffa615dbdf>] lockdep_rcu_suspicious+0xef/0x120
Feb 12 12:15:55 cassiopeia kernel: [  160.957895]  [<ffffffffc0534119>] amdgpu_sync_resv+0x2e9/0x300 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.957989]  [<ffffffffc05f6455>] ? amdgpu_job_alloc_with_ib+0x65/0x90 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958073]  [<ffffffffc052a848>] amdgpu_vm_bo_split_mapping+0x428/0xa00 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958079]  [<ffffffffa60db778>] ? do_group_exit+0x98/0x160
Feb 12 12:15:55 cassiopeia kernel: [  160.958162]  [<ffffffffc052a420>] ? amdgpu_vm_free_mapping.isra.10+0xb0/0xb0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958167]  [<ffffffffa615fb77>] ? trace_hardirqs_on_caller+0x187/0x280
Feb 12 12:15:55 cassiopeia kernel: [  160.958171]  [<ffffffffa615fc7d>] ? trace_hardirqs_on+0xd/0x10
Feb 12 12:15:55 cassiopeia kernel: [  160.958174]  [<ffffffffa615f1ec>] ? mark_lock+0xcc/0x810
Feb 12 12:15:55 cassiopeia kernel: [  160.958257]  [<ffffffffc0529d10>] ? amdgpu_vm_do_copy_ptes+0x1c0/0x1c0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958261]  [<ffffffffa615f9be>] ? mark_held_locks+0x8e/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.958266]  [<ffffffffa6362103>] ? quarantine_put+0xd3/0x130
Feb 12 12:15:55 cassiopeia kernel: [  160.958350]  [<ffffffffc052d83a>] amdgpu_vm_clear_freed+0x11a/0x200 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958433]  [<ffffffffc052d720>] ? amdgpu_vm_bo_update+0x750/0x750 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958438]  [<ffffffffa6360d99>] ? kasan_slab_free+0x89/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.958520]  [<ffffffffc052f289>] ? amdgpu_vm_fini+0x259/0x430 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958603]  [<ffffffffc052f2bc>] amdgpu_vm_fini+0x28c/0x430 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958686]  [<ffffffffc052f030>] ? amdgpu_vm_init+0x4d0/0x4d0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958765]  [<ffffffffc04eb053>] amdgpu_driver_postclose_kms+0x233/0x3b0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958770]  [<ffffffffa66bb010>] ? idr_layer_rcu_free+0x20/0x20
Feb 12 12:15:55 cassiopeia kernel: [  160.958849]  [<ffffffffc04eae20>] ? amdgpu_driver_open_kms+0x1e0/0x1e0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.958854]  [<ffffffffa6e0594e>] ? mutex_unlock+0xe/0x10
Feb 12 12:15:55 cassiopeia kernel: [  160.958882]  [<ffffffffc0370a92>] ? drm_master_release+0xf2/0x230 [drm]
Feb 12 12:15:55 cassiopeia kernel: [  160.958911]  [<ffffffffc0377892>] drm_release+0x432/0x5c0 [drm]
Feb 12 12:15:55 cassiopeia kernel: [  160.958916]  [<ffffffffa639d1a7>] __fput+0x177/0x350
Feb 12 12:15:55 cassiopeia kernel: [  160.958921]  [<ffffffffa639d3ce>] ____fput+0xe/0x10
Feb 12 12:15:55 cassiopeia kernel: [  160.958925]  [<ffffffffa610b2d0>] task_work_run+0xa0/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.958929]  [<ffffffffa60da7ed>] do_exit+0x48d/0x1320
Feb 12 12:15:55 cassiopeia kernel: [  160.958933]  [<ffffffffa6160190>] ? debug_check_no_locks_freed+0x1c0/0x1c0
Feb 12 12:15:55 cassiopeia kernel: [  160.958937]  [<ffffffffa60da360>] ? mm_update_next_owner+0x330/0x330
Feb 12 12:15:55 cassiopeia kernel: [  160.958942]  [<ffffffffa6189af6>] ? rcu_read_lock_sched_held+0x76/0x80
Feb 12 12:15:55 cassiopeia kernel: [  160.958946]  [<ffffffffa61899aa>] ? debug_lockdep_rcu_enabled.part.4+0x1a/0x30
Feb 12 12:15:55 cassiopeia kernel: [  160.958951]  [<ffffffffa60db778>] do_group_exit+0x98/0x160
Feb 12 12:15:55 cassiopeia kernel: [  160.958955]  [<ffffffffa60f0b49>] get_signal+0x3a9/0xb90
Feb 12 12:15:55 cassiopeia kernel: [  160.958959]  [<ffffffffa60ee00f>] ? force_sig_info+0x14f/0x170
Feb 12 12:15:55 cassiopeia kernel: [  160.958964]  [<ffffffffa604f41b>] do_signal+0x8b/0xc50
Feb 12 12:15:55 cassiopeia kernel: [  160.958968]  [<ffffffffa6178f6e>] ? vprintk_default+0x3e/0x60
Feb 12 12:15:55 cassiopeia kernel: [  160.958973]  [<ffffffffa62b56ee>] ? printk+0xa1/0xc8
Feb 12 12:15:55 cassiopeia kernel: [  160.958977]  [<ffffffffa604f390>] ? setup_sigcontext+0x280/0x280
Feb 12 12:15:55 cassiopeia kernel: [  160.958981]  [<ffffffffa615996f>] ? up_read+0x1f/0x40
Feb 12 12:15:55 cassiopeia kernel: [  160.958985]  [<ffffffffa6178f6e>] ? vprintk_default+0x3e/0x60
Feb 12 12:15:55 cassiopeia kernel: [  160.958989]  [<ffffffffa615f953>] ? mark_held_locks+0x23/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.958994]  [<ffffffffa60044e6>] ? exit_to_usermode_loop+0x66/0xe0
Feb 12 12:15:55 cassiopeia kernel: [  160.958998]  [<ffffffffa6004511>] exit_to_usermode_loop+0x91/0xe0
Feb 12 12:15:55 cassiopeia kernel: [  160.959002]  [<ffffffffa600575f>] prepare_exit_to_usermode+0x7f/0x90
Feb 12 12:15:55 cassiopeia kernel: [  160.959006]  [<ffffffffa6e0b274>] retint_user+0x8/0x20
Feb 12 12:15:55 cassiopeia kernel: [  160.959024] ------------[ cut here ]------------
Feb 12 12:15:55 cassiopeia kernel: [  160.959108] WARNING: CPU: 5 PID: 3979 at drivers/gpu/drm/amd/amdgpu/amdgpu_object.c:1000 amdgpu_bo_gpu_offset+0xe0/0x1d0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959111] Modules linked in: joydev input_leds hid_generic usbhid hid binfmt_misc nls_iso8859_1 edac_mce_amd edac_core dm_crypt kvm_amd kvm irqbypass crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 glue_helper lrw gf128mul ablk_helper cryptd serio_raw snd_hda_codec_realtek k10temp fam15h_power snd_hda_codec_generic eeepc_wmi asus_wmi video sparse_keymap snd_hda_codec_hdmi i2c_piix4 snd_hda_intel mxm_wmi snd_hda_codec snd_hda_core snd_hwdep snd_pcm snd_seq_midi snd_seq_midi_event snd_rawmidi tpm_infineon snd_seq snd_seq_device snd_timer mac_hid snd soundcore wmi shpchp parport_pc ppdev lp parport autofs4 amdkfd amd_iommu_v2 amdgpu i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops psmouse ttm drm ahci libahci r8169 mii fjes
Feb 12 12:15:55 cassiopeia kernel: [  160.959196] CPU: 5 PID: 3979 Comm: arb_sparse_buff Not tainted 4.9.0-amd-staging-4.9-prt #133
Feb 12 12:15:55 cassiopeia kernel: [  160.959200] Hardware name: To be filled by O.E.M. To be filled by O.E.M./M5A97 LE R2.0, BIOS 2601 03/24/2015
Feb 12 12:15:55 cassiopeia kernel: [  160.959202]  ffff88067c4df3e0 ffffffffa66b93a7 0000000000000000 0000000000000000
Feb 12 12:15:55 cassiopeia kernel: [  160.959208]  ffff88067c4df428 ffffffffa60d25b1 ffff88069bf8cb40 000003e800000000
Feb 12 12:15:55 cassiopeia kernel: [  160.959214]  ffff8806dd1ac1a8 ffff8806dd1ac25c ffff8806dd1ae948 0000000000001fff
Feb 12 12:15:55 cassiopeia kernel: [  160.959219] Call Trace:
Feb 12 12:15:55 cassiopeia kernel: [  160.959224]  [<ffffffffa66b93a7>] dump_stack+0x86/0xcf
Feb 12 12:15:55 cassiopeia kernel: [  160.959229]  [<ffffffffa60d25b1>] __warn+0x111/0x130
Feb 12 12:15:55 cassiopeia kernel: [  160.959234]  [<ffffffffa60d279d>] warn_slowpath_null+0x1d/0x20
Feb 12 12:15:55 cassiopeia kernel: [  160.959316]  [<ffffffffc0505d70>] amdgpu_bo_gpu_offset+0xe0/0x1d0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959401]  [<ffffffffc052a073>] amdgpu_vm_update_ptes.isra.9+0x103/0x2c0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959416]  [<ffffffffa635b6f5>] ? kmem_cache_alloc+0x195/0x270
Feb 12 12:15:55 cassiopeia kernel: [  160.959511]  [<ffffffffc052a351>] amdgpu_vm_frag_ptes+0x121/0x140 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959597]  [<ffffffffc052abf6>] amdgpu_vm_bo_split_mapping+0x7d6/0xa00 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959602]  [<ffffffffa60db778>] ? do_group_exit+0x98/0x160
Feb 12 12:15:55 cassiopeia kernel: [  160.959687]  [<ffffffffc052a420>] ? amdgpu_vm_free_mapping.isra.10+0xb0/0xb0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959691]  [<ffffffffa615fb77>] ? trace_hardirqs_on_caller+0x187/0x280
Feb 12 12:15:55 cassiopeia kernel: [  160.959695]  [<ffffffffa615fc7d>] ? trace_hardirqs_on+0xd/0x10
Feb 12 12:15:55 cassiopeia kernel: [  160.959699]  [<ffffffffa615f1ec>] ? mark_lock+0xcc/0x810
Feb 12 12:15:55 cassiopeia kernel: [  160.959783]  [<ffffffffc0529d10>] ? amdgpu_vm_do_copy_ptes+0x1c0/0x1c0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959787]  [<ffffffffa615f9be>] ? mark_held_locks+0x8e/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.959791]  [<ffffffffa6362103>] ? quarantine_put+0xd3/0x130
Feb 12 12:15:55 cassiopeia kernel: [  160.959876]  [<ffffffffc052d83a>] amdgpu_vm_clear_freed+0x11a/0x200 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959961]  [<ffffffffc052d720>] ? amdgpu_vm_bo_update+0x750/0x750 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.959965]  [<ffffffffa6360d99>] ? kasan_slab_free+0x89/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.960050]  [<ffffffffc052f289>] ? amdgpu_vm_fini+0x259/0x430 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.960135]  [<ffffffffc052f2bc>] amdgpu_vm_fini+0x28c/0x430 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.960220]  [<ffffffffc052f030>] ? amdgpu_vm_init+0x4d0/0x4d0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.960301]  [<ffffffffc04eb053>] amdgpu_driver_postclose_kms+0x233/0x3b0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.960306]  [<ffffffffa66bb010>] ? idr_layer_rcu_free+0x20/0x20
Feb 12 12:15:55 cassiopeia kernel: [  160.960387]  [<ffffffffc04eae20>] ? amdgpu_driver_open_kms+0x1e0/0x1e0 [amdgpu]
Feb 12 12:15:55 cassiopeia kernel: [  160.960391]  [<ffffffffa6e0594e>] ? mutex_unlock+0xe/0x10
Feb 12 12:15:55 cassiopeia kernel: [  160.960420]  [<ffffffffc0370a92>] ? drm_master_release+0xf2/0x230 [drm]
Feb 12 12:15:55 cassiopeia kernel: [  160.960450]  [<ffffffffc0377892>] drm_release+0x432/0x5c0 [drm]
Feb 12 12:15:55 cassiopeia kernel: [  160.960455]  [<ffffffffa639d1a7>] __fput+0x177/0x350
Feb 12 12:15:55 cassiopeia kernel: [  160.960459]  [<ffffffffa639d3ce>] ____fput+0xe/0x10
Feb 12 12:15:55 cassiopeia kernel: [  160.960463]  [<ffffffffa610b2d0>] task_work_run+0xa0/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.960467]  [<ffffffffa60da7ed>] do_exit+0x48d/0x1320
Feb 12 12:15:55 cassiopeia kernel: [  160.960471]  [<ffffffffa6160190>] ? debug_check_no_locks_freed+0x1c0/0x1c0
Feb 12 12:15:55 cassiopeia kernel: [  160.960475]  [<ffffffffa60da360>] ? mm_update_next_owner+0x330/0x330
Feb 12 12:15:55 cassiopeia kernel: [  160.960479]  [<ffffffffa6189af6>] ? rcu_read_lock_sched_held+0x76/0x80
Feb 12 12:15:55 cassiopeia kernel: [  160.960483]  [<ffffffffa61899aa>] ? debug_lockdep_rcu_enabled.part.4+0x1a/0x30
Feb 12 12:15:55 cassiopeia kernel: [  160.960488]  [<ffffffffa60db778>] do_group_exit+0x98/0x160
Feb 12 12:15:55 cassiopeia kernel: [  160.960491]  [<ffffffffa60f0b49>] get_signal+0x3a9/0xb90
Feb 12 12:15:55 cassiopeia kernel: [  160.960495]  [<ffffffffa60ee00f>] ? force_sig_info+0x14f/0x170
Feb 12 12:15:55 cassiopeia kernel: [  160.960500]  [<ffffffffa604f41b>] do_signal+0x8b/0xc50
Feb 12 12:15:55 cassiopeia kernel: [  160.960503]  [<ffffffffa6178f6e>] ? vprintk_default+0x3e/0x60
Feb 12 12:15:55 cassiopeia kernel: [  160.960507]  [<ffffffffa62b56ee>] ? printk+0xa1/0xc8
Feb 12 12:15:55 cassiopeia kernel: [  160.960511]  [<ffffffffa604f390>] ? setup_sigcontext+0x280/0x280
Feb 12 12:15:55 cassiopeia kernel: [  160.960515]  [<ffffffffa615996f>] ? up_read+0x1f/0x40
Feb 12 12:15:55 cassiopeia kernel: [  160.960519]  [<ffffffffa6178f6e>] ? vprintk_default+0x3e/0x60
Feb 12 12:15:55 cassiopeia kernel: [  160.960523]  [<ffffffffa615f953>] ? mark_held_locks+0x23/0xc0
Feb 12 12:15:55 cassiopeia kernel: [  160.960527]  [<ffffffffa60044e6>] ? exit_to_usermode_loop+0x66/0xe0
Feb 12 12:15:55 cassiopeia kernel: [  160.960531]  [<ffffffffa6004511>] exit_to_usermode_loop+0x91/0xe0
Feb 12 12:15:55 cassiopeia kernel: [  160.960536]  [<ffffffffa600575f>] prepare_exit_to_usermode+0x7f/0x90
Feb 12 12:15:55 cassiopeia kernel: [  160.960539]  [<ffffffffa6e0b274>] retint_user+0x8/0x20
Feb 12 12:15:55 cassiopeia kernel: [  160.960580] ---[ end trace b633027e38c2b434 ]---

[-- Attachment #4: Type: text/plain, Size: 154 bytes --]

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: PRT support for amdgpu v3

[Christian König]

Hi Nicolai,

that one should be fixed by "drm/amdgpu: fix PRT cleanup order in the 
VM". Please test and/or review.

Thanks,
Christian.

Am 12.02.2017 um 12:36 schrieb Nicolai Hähnle:
> Hi,
>
> Some more testing uncovered a bug in cleanup paths. When the 
> application segfaults while PRT mappings exist, I get a WARN_ON (which 
> seems fairly straightforward) and occasionally also an RCU error 
> warning -- see the attached dmesg logs.
>
> Regular application shutdown works fine, though.
>
> Cheers,
> Nicolai
>
> On 08.02.2017 16:04, Christian König wrote:
>> Hi guys,
>>
>> ok I finally found time to write an unit test for this and hammered 
>> out the last few bugs.
>>
>> Seems to work fine on my Tonga now. Please note that this set is 
>> based on "fix race in GEM VA map IOCTL v2", without that patch you 
>> will run into a NULL pointer dereference during PRT mapping.
>>
>> Going to send out the unit test in a minute.
>>
>> Regards,
>> Christian.
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
>>
>

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx