[PATCH v5 00/27] RFC Support hot device unplug in amdgpu

[PATCH v5 00/27] RFC Support hot device unplug in amdgpu

[Andrey Grodzovsky]

Until now extracting a card either by physical extraction (e.g. eGPU with 
thunderbolt connection or by emulation through  syfs -> /sys/bus/pci/devices/device_id/remove) 
would cause random crashes in user apps. The random crashes in apps were 
mostly due to the app having mapped a device backed BO into its address 
space was still trying to access the BO while the backing device was gone.
To answer this first problem Christian suggested to fix the handling of mapped 
memory in the clients when the device goes away by forcibly unmap all buffers the 
user processes has by clearing their respective VMAs mapping the device BOs. 
Then when the VMAs try to fill in the page tables again we check in the fault 
handlerif the device is removed and if so, return an error. This will generate a 
SIGBUS to the application which can then cleanly terminate.This indeed was done 
but this in turn created a problem of kernel OOPs were the OOPSes were due to the 
fact that while the app was terminating because of the SIGBUSit would trigger use 
after free in the driver by calling to accesses device structures that were already 
released from the pci remove sequence.This was handled by introducing a 'flush' 
sequence during device removal were we wait for drm file reference to drop to 0 
meaning all user clients directly using this device terminated.

v2:
Based on discussions in the mailing list with Daniel and Pekka [1] and based on the document 
produced by Pekka from those discussions [2] the whole approach with returning SIGBUS and 
waiting for all user clients having CPU mapping of device BOs to die was dropped. 
Instead as per the document suggestion the device structures are kept alive until 
the last reference to the device is dropped by user client and in the meanwhile all existing and new CPU mappings of the BOs 
belonging to the device directly or by dma-buf import are rerouted to per user 
process dummy rw page.Also, I skipped the 'Requirements for KMS UAPI' section of [2] 
since i am trying to get the minimal set of requirements that still give useful solution 
to work and this is the'Requirements for Render and Cross-Device UAPI' section and so my 
test case is removing a secondary device, which is render only and is not involved 
in KMS.

v3:
More updates following comments from v2 such as removing loop to find DRM file when rerouting 
page faults to dummy page,getting rid of unnecessary sysfs handling refactoring and moving 
prevention of GPU recovery post device unplug from amdgpu to scheduler layer. 
On top of that added unplug support for the IOMMU enabled system.

v4:
Drop last sysfs hack and use sysfs default attribute.
Guard against write accesses after device removal to avoid modifying released memory.
Update dummy pages handling to on demand allocation and release through drm managed framework.
Add return value to scheduler job TO handler (by Luben Tuikov) and use this in amdgpu for prevention 
of GPU recovery post device unplug
Also rebase on top of drm-misc-mext instead of amd-staging-drm-next

v5:
The most significant in this series is the improved protection from kernel driver accessing MMIO ranges that were allocated
for the device once the device is gone. To do this, first a patch 'drm/amdgpu: Unmap all MMIO mappings' is introduced.
This patch unamps all MMIO mapped into the kernel address space in the form of BARs and kernel BOs with CPU visible VRAM mappings.
This way it helped to discover multiple such access points because a page fault would be immediately generated on access. Most of them
were solved by moving HW fini code into pci_remove stage (patch drm/amdgpu: Add early fini callback) and for some who 
were harder to unwind drm_dev_enter/exit scoping was used. In addition all the IOCTLs and all background work and timers 
are now protected with drm_dev_enter/exit at their root in an attempt that after drm_dev_unplug is finished none of them 
run anymore and the pci_remove thread is the only thread executing which might touch the HW. To prevent deadlocks in such 
case against threads stuck on various HW or SW fences patches 'drm/amdgpu: Finalise device fences on device remove'  
and drm/amdgpu: Add rw_sem to pushing job into sched queue' take care of force signaling all such existing fences 
and rejecting any newly added ones.

With these patches I am able to gracefully remove the secondary card using sysfs remove hook while glxgears is running off of secondary 
card (DRI_PRIME=1) without kernel oopses or hangs and keep working with the primary card or soft reset the device without hangs or oopses.
Also as per Daniel's comment I added 3 tests to IGT [4] to core_hotunplug test suite - remove device while commands are submitted, 
exported BO and exported fence (not pushed yet).
Also now it's possible to plug back the device after unplug 
Also some users now can successfully use those patches with eGPU boxes[3].




TODOs for followup work:
Convert AMDGPU code to use devm (for hw stuff) and drmm (for sw stuff and allocations) (Daniel)
Add support for 'Requirements for KMS UAPI' section of [2] - unplugging primary, display connected card.

[1] - Discussions during v4 of the patchset https://lists.freedesktop.org/archives/amd-gfx/2021-January/058595.html
[2] - drm/doc: device hot-unplug for userspace https://www.spinics.net/lists/dri-devel/msg259755.html
[3] - Related gitlab ticket https://gitlab.freedesktop.org/drm/amd/-/issues/1081
[4] - https://gitlab.freedesktop.org/agrodzov/igt-gpu-tools/-/commits/master

Andrey Grodzovsky (27):
  drm/ttm: Remap all page faults to per process dummy page.
  drm/ttm: Expose ttm_tt_unpopulate for driver use
  drm/amdgpu: Split amdgpu_device_fini into early and late
  drm/amdkfd: Split kfd suspend from devie exit
  drm/amdgpu: Add early fini callback
  drm/amdgpu: Handle IOMMU enabled case.
  drm/amdgpu: Remap all page faults to per process dummy page.
  PCI: add support for dev_groups to struct pci_device_driver
  dmr/amdgpu: Move some sysfs attrs creation to default_attr
  drm/amdgpu: Guard against write accesses after device removal
  drm/sched: Make timeout timer rearm conditional.
  drm/amdgpu: Prevent any job recoveries after device is unplugged.
  drm/amdgpu: When filizing the fence driver. stop scheduler first.
  drm/amdgpu: Fix hang on device removal.
  drm/scheduler: Fix hang when sched_entity released
  drm/amdgpu: Unmap all MMIO mappings
  drm/amdgpu: Add rw_sem to pushing job into sched queue
  drm/sched: Expose drm_sched_entity_kill_jobs
  drm/amdgpu: Finilise device fences on device remove.
  drm: Scope all DRM IOCTLs  with drm_dev_enter/exit
  drm/amdgpu: Add support for hot-unplug feature at DRM level.
  drm/amd/display: Scope all DM queued work with drm_dev_enter/exit
  drm/amd/powerplay: Scope all PM queued work with drm_dev_enter/exit
  drm/amdkfd: Scope all KFD queued work with drm_dev_enter/exit
  drm/amdgpu: Scope all amdgpu queued work with drm_dev_enter/exit
  drm/amd/display: Remove superflous drm_mode_config_cleanup
  drm/amdgpu: Verify DMA opearations from device are done

 drivers/gpu/drm/amd/amdgpu/amdgpu.h           |  18 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c    |  13 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c  |  17 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c        |  13 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    | 353 ++++++++++++++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c       |  34 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c     |  34 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c      |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h      |   1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c       |   9 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c   |  25 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c        | 228 +++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c       |  61 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h       |   3 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_job.c       |  33 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c      |  28 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c       |  12 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |  41 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h    |   7 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       | 115 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h       |   3 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c       |  56 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c      |  70 ++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h      |  52 +--
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  21 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c       |  74 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c       |  45 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c       |  83 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c        |   7 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c  |  14 +-
 drivers/gpu/drm/amd/amdgpu/cik_ih.c           |   3 +-
 drivers/gpu/drm/amd/amdgpu/cz_ih.c            |   3 +-
 drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c         |  10 +-
 drivers/gpu/drm/amd/amdgpu/iceland_ih.c       |   3 +-
 drivers/gpu/drm/amd/amdgpu/navi10_ih.c        |   5 +-
 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c        |  44 +--
 drivers/gpu/drm/amd/amdgpu/psp_v12_0.c        |   8 +-
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c         |   8 +-
 drivers/gpu/drm/amd/amdgpu/si_ih.c            |   3 +-
 drivers/gpu/drm/amd/amdgpu/tonga_ih.c         |   3 +-
 drivers/gpu/drm/amd/amdgpu/vce_v4_0.c         |  26 +-
 drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c         |  22 +-
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c        |   5 +-
 drivers/gpu/drm/amd/amdgpu/vega20_ih.c        |   2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_device.c       |   3 +-
 drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c    |  14 +-
 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c |  13 +-
 .../amd/display/amdgpu_dm/amdgpu_dm_hdcp.c    | 124 +++---
 .../drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c |  24 +-
 drivers/gpu/drm/amd/include/amd_shared.h      |   2 +
 drivers/gpu/drm/amd/pm/amdgpu_dpm.c           |  44 ++-
 .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c |   2 +
 drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  26 +-
 drivers/gpu/drm/drm_ioctl.c                   |  15 +-
 drivers/gpu/drm/scheduler/sched_entity.c      |   6 +-
 drivers/gpu/drm/scheduler/sched_main.c        |  35 +-
 drivers/gpu/drm/ttm/ttm_bo_vm.c               |  79 +++-
 drivers/gpu/drm/ttm/ttm_tt.c                  |   1 +
 drivers/pci/pci-driver.c                      |   1 +
 include/drm/drm_drv.h                         |   6 +
 include/drm/gpu_scheduler.h                   |   1 +
 include/drm/ttm/ttm_bo_api.h                  |   2 +
 include/linux/pci.h                           |   3 +
 64 files changed, 1388 insertions(+), 633 deletions(-)

-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 01/27] drm/ttm: Remap all page faults to per process dummy page.

[Andrey Grodzovsky]

On device removal reroute all CPU mappings to dummy page.

v3:
Remove loop to find DRM file and instead access it
by vma->vm_file->private_data. Move dummy page installation
into a separate function.

v4:
Map the entire BOs VA space into on demand allocated dummy page
on the first fault for that BO.

v5: Remove duplicate return.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/ttm/ttm_bo_vm.c | 79 ++++++++++++++++++++++++++++++++-
 include/drm/ttm/ttm_bo_api.h    |  2 +
 2 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index b31b18058965..8b8300551a7f 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -34,6 +34,8 @@
 #include <drm/ttm/ttm_bo_driver.h>
 #include <drm/ttm/ttm_placement.h>
 #include <drm/drm_vma_manager.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_managed.h>
 #include <linux/mm.h>
 #include <linux/pfn_t.h>
 #include <linux/rbtree.h>
@@ -380,19 +382,94 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf,
 }
 EXPORT_SYMBOL(ttm_bo_vm_fault_reserved);
 
+static void ttm_bo_release_dummy_page(struct drm_device *dev, void *res)
+{
+	struct page *dummy_page = (struct page *)res;
+
+	__free_page(dummy_page);
+}
+
+vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot)
+{
+	struct vm_area_struct *vma = vmf->vma;
+	struct ttm_buffer_object *bo = vma->vm_private_data;
+	struct drm_device *ddev = bo->base.dev;
+	vm_fault_t ret = VM_FAULT_NOPAGE;
+	unsigned long address = vma->vm_start;
+	unsigned long num_prefault = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
+	unsigned long pfn;
+	struct page *page;
+	int i;
+
+	/*
+	 * Wait for buffer data in transit, due to a pipelined
+	 * move.
+	 */
+	ret = ttm_bo_vm_fault_idle(bo, vmf);
+	if (unlikely(ret != 0))
+		return ret;
+
+	/* Allocate new dummy page to map all the VA range in this VMA to it*/
+	page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+	if (!page)
+		return VM_FAULT_OOM;
+
+	pfn = page_to_pfn(page);
+
+	/*
+	 * Prefault the entire VMA range right away to avoid further faults
+	 */
+	for (i = 0; i < num_prefault; ++i) {
+
+		if (unlikely(address >= vma->vm_end))
+			break;
+
+		if (vma->vm_flags & VM_MIXEDMAP)
+			ret = vmf_insert_mixed_prot(vma, address,
+						    __pfn_to_pfn_t(pfn, PFN_DEV),
+						    prot);
+		else
+			ret = vmf_insert_pfn_prot(vma, address, pfn, prot);
+
+		/* Never error on prefaulted PTEs */
+		if (unlikely((ret & VM_FAULT_ERROR))) {
+			if (i == 0)
+				return VM_FAULT_NOPAGE;
+			else
+				break;
+		}
+
+		address += PAGE_SIZE;
+	}
+
+	/* Set the page to be freed using drmm release action */
+	if (drmm_add_action_or_reset(ddev, ttm_bo_release_dummy_page, page))
+		return VM_FAULT_OOM;
+
+	return ret;
+}
+EXPORT_SYMBOL(ttm_bo_vm_dummy_page);
+
 vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf)
 {
 	struct vm_area_struct *vma = vmf->vma;
 	pgprot_t prot;
 	struct ttm_buffer_object *bo = vma->vm_private_data;
+	struct drm_device *ddev = bo->base.dev;
 	vm_fault_t ret;
+	int idx;
 
 	ret = ttm_bo_vm_reserve(bo, vmf);
 	if (ret)
 		return ret;
 
 	prot = vma->vm_page_prot;
-	ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1);
+	if (drm_dev_enter(ddev, &idx)) {
+		ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1);
+		drm_dev_exit(idx);
+	} else {
+		ret = ttm_bo_vm_dummy_page(vmf, prot);
+	}
 	if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
 		return ret;
 
diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h
index 3587f660e8f4..dbb00e495cb4 100644
--- a/include/drm/ttm/ttm_bo_api.h
+++ b/include/drm/ttm/ttm_bo_api.h
@@ -635,4 +635,6 @@ int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr,
 		     void *buf, int len, int write);
 bool ttm_bo_delayed_delete(struct ttm_device *bdev, bool remove_all);
 
+vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot);
+
 #endif
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 02/27] drm/ttm: Expose ttm_tt_unpopulate for driver use

[Andrey Grodzovsky]

It's needed to drop iommu backed pages on device unplug
before device's IOMMU group is released.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/ttm/ttm_tt.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c
index 48c407cff112..f2ce1b372096 100644
--- a/drivers/gpu/drm/ttm/ttm_tt.c
+++ b/drivers/gpu/drm/ttm/ttm_tt.c
@@ -431,3 +431,4 @@ void ttm_tt_mgr_init(unsigned long num_pages, unsigned long num_dma32_pages)
 	if (!ttm_dma32_pages_limit)
 		ttm_dma32_pages_limit = num_dma32_pages;
 }
+EXPORT_SYMBOL(ttm_tt_unpopulate);
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late

[Andrey Grodzovsky]

Some of the stuff in amdgpu_device_fini such as HW interrupts
disable and pending fences finilization must be done right away on
pci_remove while most of the stuff which relates to finilizing and
releasing driver data structures can be kept until
drm_driver.release hook is called, i.e. when the last device
reference is dropped.

v4: Change functions prefix early->hw and late->sw

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 ++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  7 ++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c  | 15 ++++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    | 26 +++++++++++++---------
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h    |  3 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 12 +++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c    |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h   |  3 ++-
 drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  2 +-
 drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  2 +-
 drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  2 +-
 drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  2 +-
 drivers/gpu/drm/amd/amdgpu/si_ih.c         |  2 +-
 drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  2 +-
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  2 +-
 drivers/gpu/drm/amd/amdgpu/vega20_ih.c     |  2 +-
 17 files changed, 79 insertions(+), 36 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 1af2fa1591fd..fddb82897e5d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
 
 int amdgpu_device_init(struct amdgpu_device *adev,
 		       uint32_t flags);
-void amdgpu_device_fini(struct amdgpu_device *adev);
+void amdgpu_device_fini_hw(struct amdgpu_device *adev);
+void amdgpu_device_fini_sw(struct amdgpu_device *adev);
+
 int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
 
 void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
@@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev);
 int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
 void amdgpu_driver_postclose_kms(struct drm_device *dev,
 				 struct drm_file *file_priv);
+void amdgpu_driver_release_kms(struct drm_device *dev);
+
 int amdgpu_device_ip_suspend(struct amdgpu_device *adev);
 int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);
 int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 6447cd6ca5a8..8d22b79fc1cd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev,
  * Tear down the driver info (all asics).
  * Called at driver shutdown.
  */
-void amdgpu_device_fini(struct amdgpu_device *adev)
+void amdgpu_device_fini_hw(struct amdgpu_device *adev)
 {
 	dev_info(adev->dev, "amdgpu: finishing device.\n");
 	flush_delayed_work(&adev->delayed_init_work);
 	adev->shutdown = true;
 
-	kfree(adev->pci_state);
-
 	/* make sure IB test finished before entering exclusive mode
 	 * to avoid preemption on IB test
 	 * */
@@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
 		else
 			drm_atomic_helper_shutdown(adev_to_drm(adev));
 	}
-	amdgpu_fence_driver_fini(adev);
+	amdgpu_fence_driver_fini_hw(adev);
+
 	if (adev->pm_sysfs_en)
 		amdgpu_pm_sysfs_fini(adev);
+	if (adev->ucode_sysfs_en)
+		amdgpu_ucode_sysfs_fini(adev);
+	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
+
+
 	amdgpu_fbdev_fini(adev);
+
+	amdgpu_irq_fini_hw(adev);
+}
+
+void amdgpu_device_fini_sw(struct amdgpu_device *adev)
+{
 	amdgpu_device_ip_fini(adev);
+	amdgpu_fence_driver_fini_sw(adev);
 	release_firmware(adev->firmware.gpu_info_fw);
 	adev->firmware.gpu_info_fw = NULL;
 	adev->accel_working = false;
@@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
 	adev->rmmio = NULL;
 	amdgpu_device_doorbell_fini(adev);
 
-	if (adev->ucode_sysfs_en)
-		amdgpu_ucode_sysfs_fini(adev);
-
-	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
 	if (IS_ENABLED(CONFIG_PERF_EVENTS))
 		amdgpu_pmu_fini(adev);
 	if (adev->mman.discovery_bin)
 		amdgpu_discovery_fini(adev);
+
+	kfree(adev->pci_state);
+
 }
 
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 671ec1002230..54cb5ee2f563 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev)
 {
 	struct drm_device *dev = pci_get_drvdata(pdev);
 
-#ifdef MODULE
-	if (THIS_MODULE->state != MODULE_STATE_GOING)
-#endif
-		DRM_ERROR("Hotplug removal is not supported\n");
 	drm_dev_unplug(dev);
 	amdgpu_driver_unload_kms(dev);
+
 	pci_disable_device(pdev);
-	pci_set_drvdata(pdev, NULL);
 }
 
 static void
@@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = {
 	.dumb_create = amdgpu_mode_dumb_create,
 	.dumb_map_offset = amdgpu_mode_dumb_mmap,
 	.fops = &amdgpu_driver_kms_fops,
+	.release = &amdgpu_driver_release_kms,
 
 	.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
 	.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index 8e0a5650d383..34d51e962799 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
  *
  * Tear down the fence driver for all possible rings (all asics).
  */
-void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
+void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 {
 	unsigned i, j;
 	int r;
@@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
 		if (!ring->no_scheduler)
 			drm_sched_fini(&ring->sched);
 		del_timer_sync(&ring->fence_drv.fallback_timer);
+	}
+}
+
+void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev)
+{
+	unsigned int i, j;
+
+	for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
+		struct amdgpu_ring *ring = adev->rings[i];
+
+		if (!ring || !ring->fence_drv.initialized)
+			continue;
+
 		for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j)
 			dma_fence_put(ring->fence_drv.fences[j]);
 		kfree(ring->fence_drv.fences);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index afbbec82a289..63e815c27585 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -49,6 +49,7 @@
 #include <drm/drm_irq.h>
 #include <drm/drm_vblank.h>
 #include <drm/amdgpu_drm.h>
+#include <drm/drm_drv.h>
 #include "amdgpu.h"
 #include "amdgpu_ih.h"
 #include "atom.h"
@@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
 	return 0;
 }
 
+
+void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
+{
+	if (adev->irq.installed) {
+		drm_irq_uninstall(&adev->ddev);
+		adev->irq.installed = false;
+		if (adev->irq.msi_enabled)
+			pci_free_irq_vectors(adev->pdev);
+
+		if (!amdgpu_device_has_dc_support(adev))
+			flush_work(&adev->hotplug_work);
+	}
+}
+
 /**
  * amdgpu_irq_fini - shut down interrupt handling
  *
@@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
  * functionality, shuts down vblank, hotplug and reset interrupt handling,
  * turns off interrupts from all sources (all ASICs).
  */
-void amdgpu_irq_fini(struct amdgpu_device *adev)
+void amdgpu_irq_fini_sw(struct amdgpu_device *adev)
 {
 	unsigned i, j;
 
-	if (adev->irq.installed) {
-		drm_irq_uninstall(adev_to_drm(adev));
-		adev->irq.installed = false;
-		if (adev->irq.msi_enabled)
-			pci_free_irq_vectors(adev->pdev);
-		if (!amdgpu_device_has_dc_support(adev))
-			flush_work(&adev->hotplug_work);
-	}
-
 	for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) {
 		if (!adev->irq.client[i].sources)
 			continue;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
index ac527e5deae6..392a7324e2b1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
@@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev);
 irqreturn_t amdgpu_irq_handler(int irq, void *arg);
 
 int amdgpu_irq_init(struct amdgpu_device *adev);
-void amdgpu_irq_fini(struct amdgpu_device *adev);
+void amdgpu_irq_fini_sw(struct amdgpu_device *adev);
+void amdgpu_irq_fini_hw(struct amdgpu_device *adev);
 int amdgpu_irq_add_id(struct amdgpu_device *adev,
 		      unsigned client_id, unsigned src_id,
 		      struct amdgpu_irq_src *source);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index 64beb3399604..1af3fba7bfd4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -29,6 +29,7 @@
 #include "amdgpu.h"
 #include <drm/drm_debugfs.h>
 #include <drm/amdgpu_drm.h>
+#include <drm/drm_drv.h>
 #include "amdgpu_uvd.h"
 #include "amdgpu_vce.h"
 #include "atom.h"
@@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev)
 	}
 
 	amdgpu_acpi_fini(adev);
-	amdgpu_device_fini(adev);
+	amdgpu_device_fini_hw(adev);
 }
 
 void amdgpu_register_gpu_instance(struct amdgpu_device *adev)
@@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
 	pm_runtime_put_autosuspend(dev->dev);
 }
 
+
+void amdgpu_driver_release_kms(struct drm_device *dev)
+{
+	struct amdgpu_device *adev = drm_to_adev(dev);
+
+	amdgpu_device_fini_sw(adev);
+	pci_set_drvdata(adev->pdev, NULL);
+}
+
 /*
  * VBlank related functions.
  */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index 1fb2a91ad30a..c0a16eac4923 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev)
 	if (!con)
 		return 0;
 
+
 	/* Need disable ras on all IPs here before ip [hw/sw]fini */
 	amdgpu_ras_disable_all_features(adev, 0);
 	amdgpu_ras_recovery_fini(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
index 56acec1075ac..0f195f7bf797 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
@@ -107,7 +107,8 @@ struct amdgpu_fence_driver {
 };
 
 int amdgpu_fence_driver_init(struct amdgpu_device *adev);
-void amdgpu_fence_driver_fini(struct amdgpu_device *adev);
+void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev);
+void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev);
 void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
 
 int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
index d3745711d55f..183d44a6583c 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
@@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
index 307c01301c87..d32743949003 100644
--- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
@@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
index cc957471f31e..da96c6013477 100644
--- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
@@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index f4e4040bbd25..5eea4550b856 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
index 51880f6ef634..751307f3252c 100644
--- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
@@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 
 	return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
index 249fcbee7871..973d80ec7f6c 100644
--- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
@@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
index 88626d83e07b..2d0094c276ca 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
@@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
index 5a3c867d5881..9059b21b079f 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
@@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
-	amdgpu_irq_fini(adev);
+	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 04/27] drm/amdkfd: Split kfd suspend from devie exit

[Andrey Grodzovsky]

Helps to expdite HW related stuff to amdgpu_pci_remove

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_device.c    | 3 ++-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
index c5343a5eecbe..9edb35ba181b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -169,7 +169,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)
 	}
 }
 
-void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev)
+void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev)
 {
 	if (adev->kfd.dev) {
 		kgd2kfd_device_exit(adev->kfd.dev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index a81d9cacf9b8..c51001602a68 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -126,7 +126,7 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev,
 			const void *ih_ring_entry);
 void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev);
 void amdgpu_amdkfd_device_init(struct amdgpu_device *adev);
-void amdgpu_amdkfd_device_fini(struct amdgpu_device *adev);
+void amdgpu_amdkfd_device_fini_sw(struct amdgpu_device *adev);
 int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine,
 				uint32_t vmid, uint64_t gpu_addr,
 				uint32_t *ib_cmd, uint32_t ib_len);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 72c893fff61a..1bb8bc6d85f5 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -833,10 +833,11 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
 	return kfd->init_complete;
 }
 
+
+
 void kgd2kfd_device_exit(struct kfd_dev *kfd)
 {
 	if (kfd->init_complete) {
-		kgd2kfd_suspend(kfd, false);
 		device_queue_manager_uninit(kfd->dqm);
 		kfd_interrupt_exit(kfd);
 		kfd_topology_remove_device(kfd);
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 05/27] drm/amdgpu: Add early fini callback

[Andrey Grodzovsky]

Use it to call disply code dependent on device->drv_data
before it's set to NULL on device unplug

v5: Move HW finilization into this callback to prevent MMIO accesses
    post cpi remove.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    | 59 +++++++++++++------
 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 +++-
 drivers/gpu/drm/amd/include/amd_shared.h      |  2 +
 3 files changed, 52 insertions(+), 21 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 8d22b79fc1cd..46d646c40338 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -2536,34 +2536,26 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
 	return 0;
 }
 
-/**
- * amdgpu_device_ip_fini - run fini for hardware IPs
- *
- * @adev: amdgpu_device pointer
- *
- * Main teardown pass for hardware IPs.  The list of all the hardware
- * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks
- * are run.  hw_fini tears down the hardware associated with each IP
- * and sw_fini tears down any software state associated with each IP.
- * Returns 0 on success, negative error code on failure.
- */
-static int amdgpu_device_ip_fini(struct amdgpu_device *adev)
+static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev)
 {
 	int i, r;
 
-	if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done)
-		amdgpu_virt_release_ras_err_handler_data(adev);
+	for (i = 0; i < adev->num_ip_blocks; i++) {
+		if (!adev->ip_blocks[i].version->funcs->early_fini)
+			continue;
 
-	amdgpu_ras_pre_fini(adev);
+		r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev);
+		if (r) {
+			DRM_DEBUG("early_fini of IP block <%s> failed %d\n",
+				  adev->ip_blocks[i].version->funcs->name, r);
+		}
+	}
 
-	if (adev->gmc.xgmi.num_physical_nodes > 1)
-		amdgpu_xgmi_remove_device(adev);
+	amdgpu_amdkfd_suspend(adev, false);
 
 	amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
 	amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
 
-	amdgpu_amdkfd_device_fini(adev);
-
 	/* need to disable SMC first */
 	for (i = 0; i < adev->num_ip_blocks; i++) {
 		if (!adev->ip_blocks[i].status.hw)
@@ -2594,6 +2586,33 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev)
 		adev->ip_blocks[i].status.hw = false;
 	}
 
+	return 0;
+}
+
+/**
+ * amdgpu_device_ip_fini - run fini for hardware IPs
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Main teardown pass for hardware IPs.  The list of all the hardware
+ * IPs that make up the asic is walked and the hw_fini and sw_fini callbacks
+ * are run.  hw_fini tears down the hardware associated with each IP
+ * and sw_fini tears down any software state associated with each IP.
+ * Returns 0 on success, negative error code on failure.
+ */
+static int amdgpu_device_ip_fini(struct amdgpu_device *adev)
+{
+	int i, r;
+
+	if (amdgpu_sriov_vf(adev) && adev->virt.ras_init_done)
+		amdgpu_virt_release_ras_err_handler_data(adev);
+
+	amdgpu_ras_pre_fini(adev);
+
+	if (adev->gmc.xgmi.num_physical_nodes > 1)
+		amdgpu_xgmi_remove_device(adev);
+
+	amdgpu_amdkfd_device_fini_sw(adev);
 
 	for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
 		if (!adev->ip_blocks[i].status.sw)
@@ -3624,6 +3643,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
 	amdgpu_fbdev_fini(adev);
 
 	amdgpu_irq_fini_hw(adev);
+
+	amdgpu_device_ip_fini_early(adev);
 }
 
 void amdgpu_device_fini_sw(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 55e39b462a5e..c0b9abb773a4 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -1170,6 +1170,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev)
 	return -EINVAL;
 }
 
+static int amdgpu_dm_early_fini(void *handle)
+{
+	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+
+	amdgpu_dm_audio_fini(adev);
+
+	return 0;
+}
+
 static void amdgpu_dm_fini(struct amdgpu_device *adev)
 {
 	int i;
@@ -1178,8 +1187,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev)
 		drm_encoder_cleanup(&adev->dm.mst_encoders[i].base);
 	}
 
-	amdgpu_dm_audio_fini(adev);
-
 	amdgpu_dm_destroy_drm_device(&adev->dm);
 
 #ifdef CONFIG_DRM_AMD_DC_HDCP
@@ -2194,6 +2201,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = {
 	.late_init = dm_late_init,
 	.sw_init = dm_sw_init,
 	.sw_fini = dm_sw_fini,
+	.early_fini = amdgpu_dm_early_fini,
 	.hw_init = dm_hw_init,
 	.hw_fini = dm_hw_fini,
 	.suspend = dm_suspend,
diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h
index 43ed6291b2b8..1ad56da486e4 100644
--- a/drivers/gpu/drm/amd/include/amd_shared.h
+++ b/drivers/gpu/drm/amd/include/amd_shared.h
@@ -240,6 +240,7 @@ enum amd_dpm_forced_level;
  * @late_init: sets up late driver/hw state (post hw_init) - Optional
  * @sw_init: sets up driver state, does not configure hw
  * @sw_fini: tears down driver state, does not configure hw
+ * @early_fini: tears down stuff before dev detached from driver
  * @hw_init: sets up the hw state
  * @hw_fini: tears down the hw state
  * @late_fini: final cleanup
@@ -268,6 +269,7 @@ struct amd_ip_funcs {
 	int (*late_init)(void *handle);
 	int (*sw_init)(void *handle);
 	int (*sw_fini)(void *handle);
+	int (*early_fini)(void *handle);
 	int (*hw_init)(void *handle);
 	int (*hw_fini)(void *handle);
 	void (*late_fini)(void *handle);
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Andrey Grodzovsky]

Handle all DMA IOMMU gropup related dependencies before the
group is removed.

v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
 drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
 drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
 drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
 drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
 drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
 drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
 drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
 14 files changed, 56 insertions(+), 16 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index fddb82897e5d..30a24db5f4d1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1054,6 +1054,8 @@ struct amdgpu_device {
 
 	bool                            in_pci_err_recovery;
 	struct pci_saved_state          *pci_state;
+
+	struct list_head                device_bo_list;
 };
 
 static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 46d646c40338..91594ddc2459 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -70,6 +70,7 @@
 #include <drm/task_barrier.h>
 #include <linux/pm_runtime.h>
 
+
 MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
 MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
 MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
@@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = {
 	NULL
 };
 
-
 /**
  * amdgpu_device_init - initialize the driver
  *
@@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
 
 	INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
 
+	INIT_LIST_HEAD(&adev->device_bo_list);
+
 	adev->gfx.gfx_off_req_count = 1;
 	adev->pm.ac_power = power_supply_is_system_supplied() > 0;
 
@@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
 	return r;
 }
 
+static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
+{
+	struct amdgpu_bo *bo = NULL;
+
+	/*
+	 * Unmaps all DMA mappings before device will be removed from it's
+	 * IOMMU group otherwise in case of IOMMU enabled system a crash
+	 * will happen.
+	 */
+
+	spin_lock(&adev->mman.bdev.lru_lock);
+	while (!list_empty(&adev->device_bo_list)) {
+		bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);
+		list_del_init(&bo->bo);
+		spin_unlock(&adev->mman.bdev.lru_lock);
+		if (bo->tbo.ttm)
+			ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
+		spin_lock(&adev->mman.bdev.lru_lock);
+	}
+	spin_unlock(&adev->mman.bdev.lru_lock);
+}
+
 /**
  * amdgpu_device_fini - tear down the driver
  *
@@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
 		amdgpu_ucode_sysfs_fini(adev);
 	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
 
-
 	amdgpu_fbdev_fini(adev);
 
 	amdgpu_irq_fini_hw(adev);
 
 	amdgpu_device_ip_fini_early(adev);
+
+	amdgpu_clear_dma_mappings(adev);
+
+	amdgpu_gart_dummy_page_fini(adev);
 }
 
 void amdgpu_device_fini_sw(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
index fde2d899b2c4..49cdcaf8512d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
@@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
  *
  * Frees the dummy page used by the driver (all asics).
  */
-static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
+void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
 {
 	if (!adev->dummy_page_addr)
 		return;
@@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
 	vfree(adev->gart.pages);
 	adev->gart.pages = NULL;
 #endif
-	amdgpu_gart_dummy_page_fini(adev);
 }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
index afa2e2877d87..5678d9c105ab 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
@@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev);
 void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
 int amdgpu_gart_init(struct amdgpu_device *adev);
 void amdgpu_gart_fini(struct amdgpu_device *adev);
+void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
 int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
 		       int pages);
 int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index 63e815c27585..a922154953a7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
 		if (!amdgpu_device_has_dc_support(adev))
 			flush_work(&adev->hotplug_work);
 	}
+
+	if (adev->irq.ih_soft.ring)
+		amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
+	if (adev->irq.ih.ring)
+		amdgpu_ih_ring_fini(adev, &adev->irq.ih);
+	if (adev->irq.ih1.ring)
+		amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
+	if (adev->irq.ih2.ring)
+		amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
 }
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 485f249d063a..62d829f5e62c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
 		list_del_init(&bo->shadow_list);
 		mutex_unlock(&adev->shadow_list_lock);
 	}
-	amdgpu_bo_unref(&bo->parent);
 
+
+	spin_lock(&adev->mman.bdev.lru_lock);
+	list_del(&bo->bo);
+	spin_unlock(&adev->mman.bdev.lru_lock);
+
+	amdgpu_bo_unref(&bo->parent);
 	kfree(bo->metadata);
 	kfree(bo);
 }
@@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 	if (bp->type == ttm_bo_type_device)
 		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
 
+	INIT_LIST_HEAD(&bo->bo);
+
+	spin_lock(&adev->mman.bdev.lru_lock);
+	list_add_tail(&bo->bo, &adev->device_bo_list);
+	spin_unlock(&adev->mman.bdev.lru_lock);
+
 	return 0;
 
 fail_unreserve:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index 9ac37569823f..5ae8555ef275 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -110,6 +110,8 @@ struct amdgpu_bo {
 	struct list_head		shadow_list;
 
 	struct kgd_mem                  *kfd_bo;
+
+	struct list_head		bo;
 };
 
 static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
index 183d44a6583c..df385ffc9768 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
@@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
 	amdgpu_irq_fini_sw(adev);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
 	return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
index d32743949003..b8c47e0cf37a 100644
--- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
@@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
 	amdgpu_irq_fini_sw(adev);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
 	return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
index da96c6013477..ddfe4eaeea05 100644
--- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
@@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
 	amdgpu_irq_fini_sw(adev);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
 	return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index 5eea4550b856..e171a9e78544 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
 
 	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 
 	return 0;
 }
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
index 751307f3252c..9a24f17a5750 100644
--- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
@@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
 	amdgpu_irq_fini_sw(adev);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 
 	return 0;
 }
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
index 973d80ec7f6c..b08905d1c00f 100644
--- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
@@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
 
 	amdgpu_irq_fini_sw(adev);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 	amdgpu_irq_remove_domain(adev);
 
 	return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
index 2d0094c276ca..8c8abc00f710 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
@@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
 
 	amdgpu_irq_fini_sw(adev);
 	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
-	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
 
 	return 0;
 }
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 07/27] drm/amdgpu: Remap all page faults to per process dummy page.

[Andrey Grodzovsky]

On device removal reroute all CPU mappings to dummy page
per drm_file instance or imported GEM object.

v4:
Update for modified ttm_bo_vm_dummy_page

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++-----
 1 file changed, 16 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index a785acc09f20..93163b220e46 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -49,6 +49,7 @@
 
 #include <drm/drm_debugfs.h>
 #include <drm/amdgpu_drm.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_object.h"
@@ -1982,18 +1983,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable)
 static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf)
 {
 	struct ttm_buffer_object *bo = vmf->vma->vm_private_data;
+	struct drm_device *ddev = bo->base.dev;
 	vm_fault_t ret;
+	int idx;
 
 	ret = ttm_bo_vm_reserve(bo, vmf);
 	if (ret)
 		return ret;
 
-	ret = amdgpu_bo_fault_reserve_notify(bo);
-	if (ret)
-		goto unlock;
+	if (drm_dev_enter(ddev, &idx)) {
+		ret = amdgpu_bo_fault_reserve_notify(bo);
+		if (ret) {
+			drm_dev_exit(idx);
+			goto unlock;
+		}
 
-	ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot,
-				       TTM_BO_VM_NUM_PREFAULT, 1);
+		 ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot,
+						TTM_BO_VM_NUM_PREFAULT, 1);
+
+		 drm_dev_exit(idx);
+	} else {
+		ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot);
+	}
 	if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
 		return ret;
 
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 08/27] PCI: add support for dev_groups to struct pci_device_driver

[Andrey Grodzovsky]

This is exact copy of 'USB: add support for dev_groups to
struct usb_device_driver' patch by Greg but just for
the PCI case.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
 drivers/pci/pci-driver.c | 1 +
 include/linux/pci.h      | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
index ec44a79e951a..3a72352aa5cf 100644
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner,
 	drv->driver.owner = owner;
 	drv->driver.mod_name = mod_name;
 	drv->driver.groups = drv->groups;
+	drv->driver.dev_groups = drv->dev_groups;
 
 	spin_lock_init(&drv->dynids.lock);
 	INIT_LIST_HEAD(&drv->dynids.list);
diff --git a/include/linux/pci.h b/include/linux/pci.h
index 86c799c97b77..b57755b03009 100644
--- a/include/linux/pci.h
+++ b/include/linux/pci.h
@@ -858,6 +858,8 @@ struct module;
  *		number of VFs to enable via sysfs "sriov_numvfs" file.
  * @err_handler: See Documentation/PCI/pci-error-recovery.rst
  * @groups:	Sysfs attribute groups.
+ * @dev_groups: Attributes attached to the device that will be
+ *              created once it is bound to the driver.
  * @driver:	Driver model structure.
  * @dynids:	List of dynamically added device IDs.
  */
@@ -873,6 +875,7 @@ struct pci_driver {
 	int  (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */
 	const struct pci_error_handlers *err_handler;
 	const struct attribute_group **groups;
+	const struct attribute_group **dev_groups;
 	struct device_driver	driver;
 	struct pci_dynids	dynids;
 };
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 09/27] dmr/amdgpu: Move some sysfs attrs creation to default_attr

[Andrey Grodzovsky]

This allows to remove explicit creation and destruction
of those attrs and by this avoids warnings on device
finilizing post physical device extraction.

v5: Use newly added pci_driver.dev_groups directly

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c      | 13 ++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c  | 25 ++++++++------------
 drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++-------
 4 files changed, 37 insertions(+), 32 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
index 86add0f4ea4d..0346e124ab8c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
@@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev,
 static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version,
 		   NULL);
 
+static struct attribute *amdgpu_vbios_version_attrs[] = {
+	&dev_attr_vbios_version.attr,
+	NULL
+};
+
+const struct attribute_group amdgpu_vbios_version_attr_group = {
+	.attrs = amdgpu_vbios_version_attrs
+};
+
 /**
  * amdgpu_atombios_fini - free the driver info and callbacks for atombios
  *
@@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev)
 	adev->mode_info.atom_context = NULL;
 	kfree(adev->mode_info.atom_card_info);
 	adev->mode_info.atom_card_info = NULL;
-	device_remove_file(adev->dev, &dev_attr_vbios_version);
 }
 
 /**
@@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
 {
 	struct card_info *atom_card_info =
 	    kzalloc(sizeof(struct card_info), GFP_KERNEL);
-	int ret;
 
 	if (!atom_card_info)
 		return -ENOMEM;
@@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
 		amdgpu_atombios_allocate_fb_scratch(adev);
 	}
 
-	ret = device_create_file(adev->dev, &dev_attr_vbios_version);
-	if (ret) {
-		DRM_ERROR("Failed to create device file for VBIOS version\n");
-		return ret;
-	}
-
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 54cb5ee2f563..f799c40d7e72 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -1605,6 +1605,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = {
 	.resume		= amdgpu_pci_resume,
 };
 
+extern const struct attribute_group amdgpu_vram_mgr_attr_group;
+extern const struct attribute_group amdgpu_gtt_mgr_attr_group;
+extern const struct attribute_group amdgpu_vbios_version_attr_group;
+
+static const struct attribute_group *amdgpu_sysfs_groups[] = {
+	&amdgpu_vram_mgr_attr_group,
+	&amdgpu_gtt_mgr_attr_group,
+	&amdgpu_vbios_version_attr_group,
+	NULL,
+};
+
+
 static struct pci_driver amdgpu_kms_pci_driver = {
 	.name = DRIVER_NAME,
 	.id_table = pciidlist,
@@ -1613,6 +1625,7 @@ static struct pci_driver amdgpu_kms_pci_driver = {
 	.shutdown = amdgpu_pci_shutdown,
 	.driver.pm = &amdgpu_pm_ops,
 	.err_handler = &amdgpu_pci_err_handler,
+	.dev_groups = amdgpu_sysfs_groups,
 };
 
 static int __init amdgpu_init(void)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
index 8980329cded0..3b7150e1c5ed 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
@@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO,
 static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO,
 	           amdgpu_mem_info_gtt_used_show, NULL);
 
+static struct attribute *amdgpu_gtt_mgr_attributes[] = {
+	&dev_attr_mem_info_gtt_total.attr,
+	&dev_attr_mem_info_gtt_used.attr,
+	NULL
+};
+
+const struct attribute_group amdgpu_gtt_mgr_attr_group = {
+	.attrs = amdgpu_gtt_mgr_attributes
+};
+
 static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func;
 /**
  * amdgpu_gtt_mgr_init - init GTT manager and DRM MM
@@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
 	struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr;
 	struct ttm_resource_manager *man = &mgr->manager;
 	uint64_t start, size;
-	int ret;
 
 	man->use_tt = true;
 	man->func = &amdgpu_gtt_mgr_func;
@@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
 	spin_lock_init(&mgr->lock);
 	atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT);
 
-	ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total);
-	if (ret) {
-		DRM_ERROR("Failed to create device file mem_info_gtt_total\n");
-		return ret;
-	}
-	ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used);
-	if (ret) {
-		DRM_ERROR("Failed to create device file mem_info_gtt_used\n");
-		return ret;
-	}
-
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager);
 	ttm_resource_manager_set_used(man, true);
 	return 0;
@@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev)
 	drm_mm_takedown(&mgr->mm);
 	spin_unlock(&mgr->lock);
 
-	device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total);
-	device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used);
-
 	ttm_resource_manager_cleanup(man);
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL);
 }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
index c89b66bb70e2..68369b38aebb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
@@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO,
 static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO,
 		   amdgpu_mem_info_vram_vendor, NULL);
 
-static const struct attribute *amdgpu_vram_mgr_attributes[] = {
+static struct attribute *amdgpu_vram_mgr_attributes[] = {
 	&dev_attr_mem_info_vram_total.attr,
 	&dev_attr_mem_info_vis_vram_total.attr,
 	&dev_attr_mem_info_vram_used.attr,
@@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = {
 	NULL
 };
 
+const struct attribute_group amdgpu_vram_mgr_attr_group = {
+	.attrs = amdgpu_vram_mgr_attributes
+};
+
 static const struct ttm_resource_manager_func amdgpu_vram_mgr_func;
 
 /**
@@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
 {
 	struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr;
 	struct ttm_resource_manager *man = &mgr->manager;
-	int ret;
 
 	ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT);
 
@@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
 	INIT_LIST_HEAD(&mgr->reservations_pending);
 	INIT_LIST_HEAD(&mgr->reserved_pages);
 
-	/* Add the two VRAM-related sysfs files */
-	ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
-	if (ret)
-		DRM_ERROR("Failed to register sysfs\n");
-
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager);
 	ttm_resource_manager_set_used(man, true);
 	return 0;
@@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
 	drm_mm_takedown(&mgr->mm);
 	spin_unlock(&mgr->lock);
 
-	sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
-
 	ttm_resource_manager_cleanup(man);
 	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL);
 }
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 10/27] drm/amdgpu: Guard against write accesses after device removal

[Andrey Grodzovsky]

This should prevent writing to memory or IO ranges possibly
already allocated for other uses after our device is removed.

v5:
Protect more places wher memcopy_to/form_io takes place
Protect IB submissions

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    |  75 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c       |   9 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c        | 228 +++++++++---------
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       | 115 +++++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h       |   3 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c      |  70 ++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h      |  49 +---
 drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c       |  31 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c       |  11 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c       |  22 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c        |   7 +-
 drivers/gpu/drm/amd/amdgpu/psp_v11_0.c        |  44 ++--
 drivers/gpu/drm/amd/amdgpu/psp_v12_0.c        |   8 +-
 drivers/gpu/drm/amd/amdgpu/psp_v3_1.c         |   8 +-
 drivers/gpu/drm/amd/amdgpu/vce_v4_0.c         |  26 +-
 drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c         |  22 +-
 .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c |   2 +
 17 files changed, 425 insertions(+), 305 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 91594ddc2459..22b09c4db255 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -71,6 +71,8 @@
 #include <linux/pm_runtime.h>
 
 
+#include <drm/drm_drv.h>
+
 MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
 MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
 MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
@@ -279,48 +281,55 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
 	unsigned long flags;
 	uint32_t hi = ~0;
 	uint64_t last;
+	int idx;
 
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
 #ifdef CONFIG_64BIT
-	last = min(pos + size, adev->gmc.visible_vram_size);
-	if (last > pos) {
-		void __iomem *addr = adev->mman.aper_base_kaddr + pos;
-		size_t count = last - pos;
-
-		if (write) {
-			memcpy_toio(addr, buf, count);
-			mb();
-			amdgpu_asic_flush_hdp(adev, NULL);
-		} else {
-			amdgpu_asic_invalidate_hdp(adev, NULL);
-			mb();
-			memcpy_fromio(buf, addr, count);
-		}
+		last = min(pos + size, adev->gmc.visible_vram_size);
+		if (last > pos) {
+			void __iomem *addr = adev->mman.aper_base_kaddr + pos;
+			size_t count = last - pos;
+
+			if (write) {
+				memcpy_toio(addr, buf, count);
+				mb();
+				amdgpu_asic_flush_hdp(adev, NULL);
+			} else {
+				amdgpu_asic_invalidate_hdp(adev, NULL);
+				mb();
+				memcpy_fromio(buf, addr, count);
+			}
 
-		if (count == size)
-			return;
+			if (count == size) {
+				drm_dev_exit(idx);
+				return;
+			}
 
-		pos += count;
-		buf += count / 4;
-		size -= count;
-	}
+			pos += count;
+			buf += count / 4;
+			size -= count;
+		}
 #endif
 
-	spin_lock_irqsave(&adev->mmio_idx_lock, flags);
-	for (last = pos + size; pos < last; pos += 4) {
-		uint32_t tmp = pos >> 31;
+		spin_lock_irqsave(&adev->mmio_idx_lock, flags);
+		for (last = pos + size; pos < last; pos += 4) {
+			uint32_t tmp = pos >> 31;
 
-		WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000);
-		if (tmp != hi) {
-			WREG32_NO_KIQ(mmMM_INDEX_HI, tmp);
-			hi = tmp;
+			WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000);
+			if (tmp != hi) {
+				WREG32_NO_KIQ(mmMM_INDEX_HI, tmp);
+				hi = tmp;
+			}
+			if (write)
+				WREG32_NO_KIQ(mmMM_DATA, *buf++);
+			else
+				*buf++ = RREG32_NO_KIQ(mmMM_DATA);
 		}
-		if (write)
-			WREG32_NO_KIQ(mmMM_DATA, *buf++);
-		else
-			*buf++ = RREG32_NO_KIQ(mmMM_DATA);
+		spin_unlock_irqrestore(&adev->mmio_idx_lock, flags);
+
+		drm_dev_exit(idx);
 	}
-	spin_unlock_irqrestore(&adev->mmio_idx_lock, flags);
 }
 
 /*
@@ -402,6 +411,7 @@ uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset)
  */
 void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value)
 {
+
 	if (adev->in_pci_err_recovery)
 		return;
 
@@ -542,6 +552,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index)
  */
 void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v)
 {
+
 	if (adev->in_pci_err_recovery)
 		return;
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index fe1a39ffda72..1beb4e64b884 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -31,6 +31,8 @@
 #include "amdgpu_ras.h"
 #include "amdgpu_xgmi.h"
 
+#include <drm/drm_drv.h>
+
 /**
  * amdgpu_gmc_get_pde_for_bo - get the PDE for a BO
  *
@@ -98,6 +100,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr,
 {
 	void __iomem *ptr = (void *)cpu_pt_addr;
 	uint64_t value;
+	int idx;
+
+	if (!drm_dev_enter(&adev->ddev, &idx))
+		return 0;
 
 	/*
 	 * The following is for PTE only. GART does not have PDEs.
@@ -105,6 +111,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr,
 	value = addr & 0x0000FFFFFFFFF000ULL;
 	value |= flags;
 	writeq(value, ptr + (gpu_page_idx * 8));
+
+	drm_dev_exit(idx);
+
 	return 0;
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
index 7645223ea0ef..b3a1ff04c10f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
@@ -31,6 +31,7 @@
 
 #include <drm/amdgpu_drm.h>
 #include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "atom.h"
@@ -138,7 +139,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
 	bool secure;
 
 	unsigned i;
-	int r = 0;
+	int idx, r = 0;
 	bool need_pipe_sync = false;
 
 	if (num_ibs == 0)
@@ -170,142 +171,151 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
 		return -EINVAL;
 	}
 
-	alloc_size = ring->funcs->emit_frame_size + num_ibs *
-		ring->funcs->emit_ib_size;
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
-	r = amdgpu_ring_alloc(ring, alloc_size);
-	if (r) {
-		dev_err(adev->dev, "scheduling IB failed (%d).\n", r);
-		return r;
-	}
-
-	need_ctx_switch = ring->current_ctx != fence_ctx;
-	if (ring->funcs->emit_pipeline_sync && job &&
-	    ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
-	     (amdgpu_sriov_vf(adev) && need_ctx_switch) ||
-	     amdgpu_vm_need_pipeline_sync(ring, job))) {
-		need_pipe_sync = true;
-
-		if (tmp)
-			trace_amdgpu_ib_pipe_sync(job, tmp);
-
-		dma_fence_put(tmp);
-	}
+		alloc_size = ring->funcs->emit_frame_size + num_ibs *
+			ring->funcs->emit_ib_size;
 
-	if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync)
-		ring->funcs->emit_mem_sync(ring);
+		r = amdgpu_ring_alloc(ring, alloc_size);
+		if (r) {
+			dev_err(adev->dev, "scheduling IB failed (%d).\n", r);
+			goto exit;
+		}
 
-	if (ring->funcs->emit_wave_limit &&
-	    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
-		ring->funcs->emit_wave_limit(ring, true);
+		need_ctx_switch = ring->current_ctx != fence_ctx;
+		if (ring->funcs->emit_pipeline_sync && job &&
+		    ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
+		     (amdgpu_sriov_vf(adev) && need_ctx_switch) ||
+		     amdgpu_vm_need_pipeline_sync(ring, job))) {
+			need_pipe_sync = true;
 
-	if (ring->funcs->insert_start)
-		ring->funcs->insert_start(ring);
+			if (tmp)
+				trace_amdgpu_ib_pipe_sync(job, tmp);
 
-	if (job) {
-		r = amdgpu_vm_flush(ring, job, need_pipe_sync);
-		if (r) {
-			amdgpu_ring_undo(ring);
-			return r;
+			dma_fence_put(tmp);
 		}
-	}
 
-	if (job && ring->funcs->init_cond_exec)
-		patch_offset = amdgpu_ring_init_cond_exec(ring);
+		if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync)
+			ring->funcs->emit_mem_sync(ring);
 
-#ifdef CONFIG_X86_64
-	if (!(adev->flags & AMD_IS_APU))
-#endif
-	{
-		if (ring->funcs->emit_hdp_flush)
-			amdgpu_ring_emit_hdp_flush(ring);
-		else
-			amdgpu_asic_flush_hdp(adev, ring);
-	}
+		if (ring->funcs->emit_wave_limit &&
+		    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
+			ring->funcs->emit_wave_limit(ring, true);
 
-	if (need_ctx_switch)
-		status |= AMDGPU_HAVE_CTX_SWITCH;
+		if (ring->funcs->insert_start)
+			ring->funcs->insert_start(ring);
 
-	skip_preamble = ring->current_ctx == fence_ctx;
-	if (job && ring->funcs->emit_cntxcntl) {
-		status |= job->preamble_status;
-		status |= job->preemption_status;
-		amdgpu_ring_emit_cntxcntl(ring, status);
-	}
+		if (job) {
+			r = amdgpu_vm_flush(ring, job, need_pipe_sync);
+			if (r) {
+				amdgpu_ring_undo(ring);
+				goto exit;
+			}
+		}
 
-	/* Setup initial TMZiness and send it off.
-	 */
-	secure = false;
-	if (job && ring->funcs->emit_frame_cntl) {
-		secure = ib->flags & AMDGPU_IB_FLAGS_SECURE;
-		amdgpu_ring_emit_frame_cntl(ring, true, secure);
-	}
+		if (job && ring->funcs->init_cond_exec)
+			patch_offset = amdgpu_ring_init_cond_exec(ring);
+
+	#ifdef CONFIG_X86_64
+		if (!(adev->flags & AMD_IS_APU))
+	#endif
+		{
+			if (ring->funcs->emit_hdp_flush)
+				amdgpu_ring_emit_hdp_flush(ring);
+			else
+				amdgpu_asic_flush_hdp(adev, ring);
+		}
 
-	for (i = 0; i < num_ibs; ++i) {
-		ib = &ibs[i];
+		if (need_ctx_switch)
+			status |= AMDGPU_HAVE_CTX_SWITCH;
 
-		/* drop preamble IBs if we don't have a context switch */
-		if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) &&
-		    skip_preamble &&
-		    !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) &&
-		    !amdgpu_mcbp &&
-		    !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */
-			continue;
+		skip_preamble = ring->current_ctx == fence_ctx;
+		if (job && ring->funcs->emit_cntxcntl) {
+			status |= job->preamble_status;
+			status |= job->preemption_status;
+			amdgpu_ring_emit_cntxcntl(ring, status);
+		}
 
+		/* Setup initial TMZiness and send it off.
+		 */
+		secure = false;
 		if (job && ring->funcs->emit_frame_cntl) {
-			if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) {
-				amdgpu_ring_emit_frame_cntl(ring, false, secure);
-				secure = !secure;
-				amdgpu_ring_emit_frame_cntl(ring, true, secure);
+			secure = ib->flags & AMDGPU_IB_FLAGS_SECURE;
+			amdgpu_ring_emit_frame_cntl(ring, true, secure);
+		}
+
+		for (i = 0; i < num_ibs; ++i) {
+			ib = &ibs[i];
+
+			/* drop preamble IBs if we don't have a context switch */
+			if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) &&
+			    skip_preamble &&
+			    !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) &&
+			    !amdgpu_mcbp &&
+			    !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */
+				continue;
+
+			if (job && ring->funcs->emit_frame_cntl) {
+				if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) {
+					amdgpu_ring_emit_frame_cntl(ring, false, secure);
+					secure = !secure;
+					amdgpu_ring_emit_frame_cntl(ring, true, secure);
+				}
 			}
+
+			amdgpu_ring_emit_ib(ring, job, ib, status);
+			status &= ~AMDGPU_HAVE_CTX_SWITCH;
 		}
 
-		amdgpu_ring_emit_ib(ring, job, ib, status);
-		status &= ~AMDGPU_HAVE_CTX_SWITCH;
-	}
+		if (job && ring->funcs->emit_frame_cntl)
+			amdgpu_ring_emit_frame_cntl(ring, false, secure);
 
-	if (job && ring->funcs->emit_frame_cntl)
-		amdgpu_ring_emit_frame_cntl(ring, false, secure);
+	#ifdef CONFIG_X86_64
+		if (!(adev->flags & AMD_IS_APU))
+	#endif
+			amdgpu_asic_invalidate_hdp(adev, ring);
 
-#ifdef CONFIG_X86_64
-	if (!(adev->flags & AMD_IS_APU))
-#endif
-		amdgpu_asic_invalidate_hdp(adev, ring);
+		if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE)
+			fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY;
 
-	if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE)
-		fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY;
+		/* wrap the last IB with fence */
+		if (job && job->uf_addr) {
+			amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence,
+					       fence_flags | AMDGPU_FENCE_FLAG_64BIT);
+		}
 
-	/* wrap the last IB with fence */
-	if (job && job->uf_addr) {
-		amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence,
-				       fence_flags | AMDGPU_FENCE_FLAG_64BIT);
-	}
+		r = amdgpu_fence_emit(ring, f, fence_flags);
+		if (r) {
+			dev_err(adev->dev, "failed to emit fence (%d)\n", r);
+			if (job && job->vmid)
+				amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid);
+			amdgpu_ring_undo(ring);
+			goto exit;
+		}
 
-	r = amdgpu_fence_emit(ring, f, fence_flags);
-	if (r) {
-		dev_err(adev->dev, "failed to emit fence (%d)\n", r);
-		if (job && job->vmid)
-			amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid);
-		amdgpu_ring_undo(ring);
-		return r;
-	}
+		if (ring->funcs->insert_end)
+			ring->funcs->insert_end(ring);
 
-	if (ring->funcs->insert_end)
-		ring->funcs->insert_end(ring);
+		if (patch_offset != ~0 && ring->funcs->patch_cond_exec)
+			amdgpu_ring_patch_cond_exec(ring, patch_offset);
 
-	if (patch_offset != ~0 && ring->funcs->patch_cond_exec)
-		amdgpu_ring_patch_cond_exec(ring, patch_offset);
+		ring->current_ctx = fence_ctx;
+		if (vm && ring->funcs->emit_switch_buffer)
+			amdgpu_ring_emit_switch_buffer(ring);
 
-	ring->current_ctx = fence_ctx;
-	if (vm && ring->funcs->emit_switch_buffer)
-		amdgpu_ring_emit_switch_buffer(ring);
+		if (ring->funcs->emit_wave_limit &&
+		    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
+			ring->funcs->emit_wave_limit(ring, false);
 
-	if (ring->funcs->emit_wave_limit &&
-	    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
-		ring->funcs->emit_wave_limit(ring, false);
+		amdgpu_ring_commit(ring);
 
-	amdgpu_ring_commit(ring);
-	return 0;
+	} else {
+		return -ENODEV;
+	}
+
+exit:
+	drm_dev_exit(idx);
+	return r;
 }
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 839917eb7bc3..638b7fd1857c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -25,6 +25,7 @@
 
 #include <linux/firmware.h>
 #include <linux/dma-mapping.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_psp.h"
@@ -38,6 +39,8 @@
 #include "amdgpu_ras.h"
 #include "amdgpu_securedisplay.h"
 
+#include <drm/drm_drv.h>
+
 static int psp_sysfs_init(struct amdgpu_device *adev);
 static void psp_sysfs_fini(struct amdgpu_device *adev);
 
@@ -249,7 +252,7 @@ psp_cmd_submit_buf(struct psp_context *psp,
 		   struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr)
 {
 	int ret;
-	int index;
+	int index, idx;
 	int timeout = 20000;
 	bool ras_intr = false;
 	bool skip_unsupport = false;
@@ -257,6 +260,9 @@ psp_cmd_submit_buf(struct psp_context *psp,
 	if (psp->adev->in_pci_err_recovery)
 		return 0;
 
+	if (!drm_dev_enter(&psp->adev->ddev, &idx))
+		return 0;
+
 	mutex_lock(&psp->mutex);
 
 	memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE);
@@ -267,8 +273,7 @@ psp_cmd_submit_buf(struct psp_context *psp,
 	ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index);
 	if (ret) {
 		atomic_dec(&psp->fence_value);
-		mutex_unlock(&psp->mutex);
-		return ret;
+		goto exit;
 	}
 
 	amdgpu_asic_invalidate_hdp(psp->adev, NULL);
@@ -308,8 +313,8 @@ psp_cmd_submit_buf(struct psp_context *psp,
 			 psp->cmd_buf_mem->cmd_id,
 			 psp->cmd_buf_mem->resp.status);
 		if (!timeout) {
-			mutex_unlock(&psp->mutex);
-			return -EINVAL;
+			ret = -EINVAL;
+			goto exit;
 		}
 	}
 
@@ -317,8 +322,10 @@ psp_cmd_submit_buf(struct psp_context *psp,
 		ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo;
 		ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi;
 	}
-	mutex_unlock(&psp->mutex);
 
+exit:
+	mutex_unlock(&psp->mutex);
+	drm_dev_exit(idx);
 	return ret;
 }
 
@@ -355,8 +362,7 @@ static int psp_load_toc(struct psp_context *psp,
 	if (!cmd)
 		return -ENOMEM;
 	/* Copy toc to psp firmware private buffer */
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-	memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size);
+	psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size);
 
 	psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size);
 
@@ -571,8 +577,7 @@ static int psp_asd_load(struct psp_context *psp)
 	if (!cmd)
 		return -ENOMEM;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-	memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size);
+	psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size);
 
 	psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr,
 				  psp->asd_ucode_size);
@@ -727,8 +732,7 @@ static int psp_xgmi_load(struct psp_context *psp)
 	if (!cmd)
 		return -ENOMEM;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-	memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size);
+	psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size);
 
 	psp_prep_ta_load_cmd_buf(cmd,
 				 psp->fw_pri_mc_addr,
@@ -983,8 +987,7 @@ static int psp_ras_load(struct psp_context *psp)
 	if (!cmd)
 		return -ENOMEM;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-	memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size);
+	psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size);
 
 	psp_prep_ta_load_cmd_buf(cmd,
 				 psp->fw_pri_mc_addr,
@@ -1220,8 +1223,7 @@ static int psp_hdcp_load(struct psp_context *psp)
 	if (!cmd)
 		return -ENOMEM;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-	memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr,
+	psp_copy_fw(psp, psp->ta_hdcp_start_addr,
 	       psp->ta_hdcp_ucode_size);
 
 	psp_prep_ta_load_cmd_buf(cmd,
@@ -1372,8 +1374,7 @@ static int psp_dtm_load(struct psp_context *psp)
 	if (!cmd)
 		return -ENOMEM;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-	memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size);
+	psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size);
 
 	psp_prep_ta_load_cmd_buf(cmd,
 				 psp->fw_pri_mc_addr,
@@ -1518,8 +1519,7 @@ static int psp_rap_load(struct psp_context *psp)
 	if (!cmd)
 		return -ENOMEM;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-	memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size);
+	psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size);
 
 	psp_prep_ta_load_cmd_buf(cmd,
 				 psp->fw_pri_mc_addr,
@@ -2928,7 +2928,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev,
 	struct amdgpu_device *adev = drm_to_adev(ddev);
 	void *cpu_addr;
 	dma_addr_t dma_addr;
-	int ret;
+	int ret, idx;
 	char fw_name[100];
 	const struct firmware *usbc_pd_fw;
 
@@ -2937,47 +2937,66 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev,
 		return -EBUSY;
 	}
 
-	snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf);
-	ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev);
-	if (ret)
-		goto fail;
+	if (drm_dev_enter(ddev, &idx)) {
 
-	/* We need contiguous physical mem to place the FW  for psp to access */
-	cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL);
+		snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf);
+		ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev);
+		if (ret)
+			goto fail;
 
-	ret = dma_mapping_error(adev->dev, dma_addr);
-	if (ret)
-		goto rel_buf;
+		/* We need contiguous physical mem to place the FW  for psp to access */
+		cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL);
 
-	memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size);
+		ret = dma_mapping_error(adev->dev, dma_addr);
+		if (ret)
+			goto rel_buf;
 
-	/*
-	 * x86 specific workaround.
-	 * Without it the buffer is invisible in PSP.
-	 *
-	 * TODO Remove once PSP starts snooping CPU cache
-	 */
+		memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size);
+
+		/*
+		 * x86 specific workaround.
+		 * Without it the buffer is invisible in PSP.
+		 *
+		 * TODO Remove once PSP starts snooping CPU cache
+		 */
 #ifdef CONFIG_X86
-	clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1)));
+		clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1)));
 #endif
 
-	mutex_lock(&adev->psp.mutex);
-	ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr);
-	mutex_unlock(&adev->psp.mutex);
+		mutex_lock(&adev->psp.mutex);
+		ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr);
+		mutex_unlock(&adev->psp.mutex);
 
 rel_buf:
-	dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr);
-	release_firmware(usbc_pd_fw);
-
+		dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr);
+		release_firmware(usbc_pd_fw);
 fail:
-	if (ret) {
-		DRM_ERROR("Failed to load USBC PD FW, err = %d", ret);
-		return ret;
+		if (ret) {
+			DRM_ERROR("Failed to load USBC PD FW, err = %d", ret);
+			return ret;
+		}
+
+		drm_dev_exit(idx);
+		return count;
+	} else {
+		return -ENODEV;
 	}
+}
+
+void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size)
+{
+	int idx;
 
-	return count;
+	if (!drm_dev_enter(&psp->adev->ddev, &idx))
+		return;
+
+	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
+	memcpy(psp->fw_pri_buf, start_addr, bin_size);
+
+	drm_dev_exit(idx);
 }
 
+
 static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR,
 		   psp_usbc_pd_fw_sysfs_read,
 		   psp_usbc_pd_fw_sysfs_write);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
index cb50ba445f8c..0dbbedb84c84 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
@@ -417,4 +417,7 @@ int psp_init_ta_microcode(struct psp_context *psp,
 			  const char *chip_name);
 int psp_get_fw_attestation_records_addr(struct psp_context *psp,
 					uint64_t *output_ptr);
+
+void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size);
+
 #endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
index b644c78475fd..f9f71008f454 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
@@ -35,6 +35,8 @@
 #include "amdgpu.h"
 #include "atom.h"
 
+#include <drm/drm_drv.h>
+
 /*
  * Rings
  * Most engines on the GPU are fed via ring buffers.  Ring
@@ -459,3 +461,71 @@ int amdgpu_ring_test_helper(struct amdgpu_ring *ring)
 	ring->sched.ready = !r;
 	return r;
 }
+
+void amdgpu_ring_clear_ring(struct amdgpu_ring *ring)
+{
+	int idx;
+	int i = 0;
+
+	if (!drm_dev_enter(&ring->adev->ddev, &idx))
+		return;
+
+	while (i <= ring->buf_mask)
+		ring->ring[i++] = ring->funcs->nop;
+
+	drm_dev_exit(idx);
+
+}
+
+void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v)
+{
+	int idx;
+
+	if (!drm_dev_enter(&ring->adev->ddev, &idx))
+		return;
+
+	if (ring->count_dw <= 0)
+		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
+	ring->ring[ring->wptr++ & ring->buf_mask] = v;
+	ring->wptr &= ring->ptr_mask;
+	ring->count_dw--;
+
+	drm_dev_exit(idx);
+}
+
+void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
+					      void *src, int count_dw)
+{
+	unsigned occupied, chunk1, chunk2;
+	void *dst;
+	int idx;
+
+	if (!drm_dev_enter(&ring->adev->ddev, &idx))
+		return;
+
+	if (unlikely(ring->count_dw < count_dw))
+		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
+
+	occupied = ring->wptr & ring->buf_mask;
+	dst = (void *)&ring->ring[occupied];
+	chunk1 = ring->buf_mask + 1 - occupied;
+	chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1;
+	chunk2 = count_dw - chunk1;
+	chunk1 <<= 2;
+	chunk2 <<= 2;
+
+	if (chunk1)
+		memcpy(dst, src, chunk1);
+
+	if (chunk2) {
+		src += chunk1;
+		dst = (void *)ring->ring;
+		memcpy(dst, src, chunk2);
+	}
+
+	ring->wptr += count_dw;
+	ring->wptr &= ring->ptr_mask;
+	ring->count_dw -= count_dw;
+
+	drm_dev_exit(idx);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
index 0f195f7bf797..c2b83f48e6d6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
@@ -297,53 +297,12 @@ static inline void amdgpu_ring_set_preempt_cond_exec(struct amdgpu_ring *ring,
 	*ring->cond_exe_cpu_addr = cond_exec;
 }
 
-static inline void amdgpu_ring_clear_ring(struct amdgpu_ring *ring)
-{
-	int i = 0;
-	while (i <= ring->buf_mask)
-		ring->ring[i++] = ring->funcs->nop;
-
-}
-
-static inline void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v)
-{
-	if (ring->count_dw <= 0)
-		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
-	ring->ring[ring->wptr++ & ring->buf_mask] = v;
-	ring->wptr &= ring->ptr_mask;
-	ring->count_dw--;
-}
+void amdgpu_ring_clear_ring(struct amdgpu_ring *ring);
 
-static inline void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
-					      void *src, int count_dw)
-{
-	unsigned occupied, chunk1, chunk2;
-	void *dst;
-
-	if (unlikely(ring->count_dw < count_dw))
-		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
-
-	occupied = ring->wptr & ring->buf_mask;
-	dst = (void *)&ring->ring[occupied];
-	chunk1 = ring->buf_mask + 1 - occupied;
-	chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1;
-	chunk2 = count_dw - chunk1;
-	chunk1 <<= 2;
-	chunk2 <<= 2;
+void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v);
 
-	if (chunk1)
-		memcpy(dst, src, chunk1);
-
-	if (chunk2) {
-		src += chunk1;
-		dst = (void *)ring->ring;
-		memcpy(dst, src, chunk2);
-	}
-
-	ring->wptr += count_dw;
-	ring->wptr &= ring->ptr_mask;
-	ring->count_dw -= count_dw;
-}
+void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
+					      void *src, int count_dw);
 
 int amdgpu_ring_test_helper(struct amdgpu_ring *ring);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
index e2ed4689118a..df47f5ffa08f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
@@ -32,6 +32,7 @@
 #include <linux/module.h>
 
 #include <drm/drm.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_pm.h"
@@ -375,7 +376,7 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev)
 {
 	unsigned size;
 	void *ptr;
-	int i, j;
+	int i, j, idx;
 	bool in_ras_intr = amdgpu_ras_intr_triggered();
 
 	cancel_delayed_work_sync(&adev->uvd.idle_work);
@@ -403,11 +404,15 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev)
 		if (!adev->uvd.inst[j].saved_bo)
 			return -ENOMEM;
 
-		/* re-write 0 since err_event_athub will corrupt VCPU buffer */
-		if (in_ras_intr)
-			memset(adev->uvd.inst[j].saved_bo, 0, size);
-		else
-			memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size);
+		if (drm_dev_enter(&adev->ddev, &idx)) {
+			/* re-write 0 since err_event_athub will corrupt VCPU buffer */
+			if (in_ras_intr)
+				memset(adev->uvd.inst[j].saved_bo, 0, size);
+			else
+				memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size);
+
+			drm_dev_exit(idx);
+		}
 	}
 
 	if (in_ras_intr)
@@ -420,7 +425,7 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev)
 {
 	unsigned size;
 	void *ptr;
-	int i;
+	int i, idx;
 
 	for (i = 0; i < adev->uvd.num_uvd_inst; i++) {
 		if (adev->uvd.harvest_config & (1 << i))
@@ -432,7 +437,10 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev)
 		ptr = adev->uvd.inst[i].cpu_addr;
 
 		if (adev->uvd.inst[i].saved_bo != NULL) {
-			memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size);
+			if (drm_dev_enter(&adev->ddev, &idx)) {
+				memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size);
+				drm_dev_exit(idx);
+			}
 			kvfree(adev->uvd.inst[i].saved_bo);
 			adev->uvd.inst[i].saved_bo = NULL;
 		} else {
@@ -442,8 +450,11 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev)
 			hdr = (const struct common_firmware_header *)adev->uvd.fw->data;
 			if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) {
 				offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
-				memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset,
-					    le32_to_cpu(hdr->ucode_size_bytes));
+				if (drm_dev_enter(&adev->ddev, &idx)) {
+					memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset,
+						    le32_to_cpu(hdr->ucode_size_bytes));
+					drm_dev_exit(idx);
+				}
 				size -= le32_to_cpu(hdr->ucode_size_bytes);
 				ptr += le32_to_cpu(hdr->ucode_size_bytes);
 			}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
index ea6a62f67e38..833203401ef4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
@@ -29,6 +29,7 @@
 #include <linux/module.h>
 
 #include <drm/drm.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_pm.h"
@@ -293,7 +294,7 @@ int amdgpu_vce_resume(struct amdgpu_device *adev)
 	void *cpu_addr;
 	const struct common_firmware_header *hdr;
 	unsigned offset;
-	int r;
+	int r, idx;
 
 	if (adev->vce.vcpu_bo == NULL)
 		return -EINVAL;
@@ -313,8 +314,12 @@ int amdgpu_vce_resume(struct amdgpu_device *adev)
 
 	hdr = (const struct common_firmware_header *)adev->vce.fw->data;
 	offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
-	memcpy_toio(cpu_addr, adev->vce.fw->data + offset,
-		    adev->vce.fw->size - offset);
+
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+		memcpy_toio(cpu_addr, adev->vce.fw->data + offset,
+			    adev->vce.fw->size - offset);
+		drm_dev_exit(idx);
+	}
 
 	amdgpu_bo_kunmap(adev->vce.vcpu_bo);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
index 99b82f3c2617..b42db22761b8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
@@ -27,6 +27,7 @@
 #include <linux/firmware.h>
 #include <linux/module.h>
 #include <linux/pci.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_pm.h"
@@ -267,7 +268,7 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev)
 {
 	unsigned size;
 	void *ptr;
-	int i;
+	int i, idx;
 
 	cancel_delayed_work_sync(&adev->vcn.idle_work);
 
@@ -284,7 +285,10 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev)
 		if (!adev->vcn.inst[i].saved_bo)
 			return -ENOMEM;
 
-		memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size);
+		if (drm_dev_enter(&adev->ddev, &idx)) {
+			memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size);
+			drm_dev_exit(idx);
+		}
 	}
 	return 0;
 }
@@ -293,7 +297,7 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev)
 {
 	unsigned size;
 	void *ptr;
-	int i;
+	int i, idx;
 
 	for (i = 0; i < adev->vcn.num_vcn_inst; ++i) {
 		if (adev->vcn.harvest_config & (1 << i))
@@ -305,7 +309,10 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev)
 		ptr = adev->vcn.inst[i].cpu_addr;
 
 		if (adev->vcn.inst[i].saved_bo != NULL) {
-			memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size);
+			if (drm_dev_enter(&adev->ddev, &idx)) {
+				memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size);
+				drm_dev_exit(idx);
+			}
 			kvfree(adev->vcn.inst[i].saved_bo);
 			adev->vcn.inst[i].saved_bo = NULL;
 		} else {
@@ -315,8 +322,11 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev)
 			hdr = (const struct common_firmware_header *)adev->vcn.fw->data;
 			if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) {
 				offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
-				memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset,
-					    le32_to_cpu(hdr->ucode_size_bytes));
+				if (drm_dev_enter(&adev->ddev, &idx)) {
+					memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset,
+						    le32_to_cpu(hdr->ucode_size_bytes));
+					drm_dev_exit(idx);
+				}
 				size -= le32_to_cpu(hdr->ucode_size_bytes);
 				ptr += le32_to_cpu(hdr->ucode_size_bytes);
 			}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index ae18c0e32347..7b622056df58 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -31,6 +31,7 @@
 #include <linux/dma-buf.h>
 
 #include <drm/amdgpu_drm.h>
+#include <drm/drm_drv.h>
 #include "amdgpu.h"
 #include "amdgpu_trace.h"
 #include "amdgpu_amdkfd.h"
@@ -1604,7 +1605,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
 	struct amdgpu_vm_update_params params;
 	enum amdgpu_sync_mode sync_mode;
 	uint64_t pfn;
-	int r;
+	int r, idx;
+
+	if (!drm_dev_enter(&adev->ddev, &idx))
+		return -ENODEV;
 
 	memset(&params, 0, sizeof(params));
 	params.adev = adev;
@@ -1713,6 +1717,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
 
 error_unlock:
 	amdgpu_vm_eviction_unlock(vm);
+	drm_dev_exit(idx);
 	return r;
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
index c325d6f53a71..94cce172b98e 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
@@ -23,6 +23,7 @@
 #include <linux/firmware.h>
 #include <linux/module.h>
 #include <linux/vmalloc.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_psp.h"
@@ -269,10 +270,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy PSP KDB binary to memory */
-	memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size);
+	psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size);
 
 	/* Provide the PSP KDB to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -302,10 +301,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy PSP SPL binary to memory */
-	memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size);
+	psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size);
 
 	/* Provide the PSP SPL to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -335,10 +332,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy PSP System Driver binary to memory */
-	memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
+	psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
 
 	/* Provide the sys driver to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -371,10 +366,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy Secure OS binary to PSP memory */
-	memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
+	psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
 
 	/* Provide the PSP secure OS to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -608,7 +601,7 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops)
 	uint32_t p2c_header[4];
 	uint32_t sz;
 	void *buf;
-	int ret;
+	int ret, idx;
 
 	if (ctx->init == PSP_MEM_TRAIN_NOT_SUPPORT) {
 		DRM_DEBUG("Memory training is not supported.\n");
@@ -681,17 +674,24 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops)
 			return -ENOMEM;
 		}
 
-		memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz);
-		ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN);
-		if (ret) {
-			DRM_ERROR("Send long training msg failed.\n");
+		if (drm_dev_enter(&adev->ddev, &idx)) {
+			memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz);
+			ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN);
+			if (ret) {
+				DRM_ERROR("Send long training msg failed.\n");
+				vfree(buf);
+				drm_dev_exit(idx);
+				return ret;
+			}
+
+			memcpy_toio(adev->mman.aper_base_kaddr, buf, sz);
+			adev->hdp.funcs->flush_hdp(adev, NULL);
 			vfree(buf);
-			return ret;
+			drm_dev_exit(idx);
+		} else {
+			vfree(buf);
+			return -ENODEV;
 		}
-
-		memcpy_toio(adev->mman.aper_base_kaddr, buf, sz);
-		adev->hdp.funcs->flush_hdp(adev, NULL);
-		vfree(buf);
 	}
 
 	if (ops & PSP_MEM_TRAIN_SAVE) {
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
index c4828bd3264b..618e5b6b85d9 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
@@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy PSP System Driver binary to memory */
-	memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
+	psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
 
 	/* Provide the sys driver to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy Secure OS binary to PSP memory */
-	memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
+	psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
 
 	/* Provide the PSP secure OS to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
index f2e725f72d2f..d0a6cccd0897 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
@@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy PSP System Driver binary to memory */
-	memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
+	psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
 
 	/* Provide the sys driver to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
@@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp)
 	if (ret)
 		return ret;
 
-	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
-
 	/* Copy Secure OS binary to PSP memory */
-	memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
+	psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
 
 	/* Provide the PSP secure OS to bootloader */
 	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
index 37fa163393fd..fd859c778df0 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
@@ -25,6 +25,7 @@
  */
 
 #include <linux/firmware.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_vce.h"
@@ -555,16 +556,19 @@ static int vce_v4_0_hw_fini(void *handle)
 static int vce_v4_0_suspend(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-	int r;
+	int r, idx;
 
 	if (adev->vce.vcpu_bo == NULL)
 		return 0;
 
-	if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
-		unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
-		void *ptr = adev->vce.cpu_addr;
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+		if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
+			unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
+			void *ptr = adev->vce.cpu_addr;
 
-		memcpy_fromio(adev->vce.saved_bo, ptr, size);
+			memcpy_fromio(adev->vce.saved_bo, ptr, size);
+		}
+		drm_dev_exit(idx);
 	}
 
 	r = vce_v4_0_hw_fini(adev);
@@ -577,16 +581,20 @@ static int vce_v4_0_suspend(void *handle)
 static int vce_v4_0_resume(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-	int r;
+	int r, idx;
 
 	if (adev->vce.vcpu_bo == NULL)
 		return -EINVAL;
 
 	if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
-		unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
-		void *ptr = adev->vce.cpu_addr;
 
-		memcpy_toio(ptr, adev->vce.saved_bo, size);
+		if (drm_dev_enter(&adev->ddev, &idx)) {
+			unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
+			void *ptr = adev->vce.cpu_addr;
+
+			memcpy_toio(ptr, adev->vce.saved_bo, size);
+			drm_dev_exit(idx);
+		}
 	} else {
 		r = amdgpu_vce_resume(adev);
 		if (r)
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
index def583916294..5ac8ffe197aa 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
@@ -34,6 +34,8 @@
 #include "vcn/vcn_3_0_0_sh_mask.h"
 #include "ivsrcid/vcn/irqsrcs_vcn_2_0.h"
 
+#include <drm/drm_drv.h>
+
 #define mmUVD_CONTEXT_ID_INTERNAL_OFFSET			0x27
 #define mmUVD_GPCOM_VCPU_CMD_INTERNAL_OFFSET			0x0f
 #define mmUVD_GPCOM_VCPU_DATA0_INTERNAL_OFFSET			0x10
@@ -263,16 +265,20 @@ static int vcn_v3_0_sw_init(void *handle)
 static int vcn_v3_0_sw_fini(void *handle)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
-	int i, r;
+	int i, r, idx;
 
-	for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
-		volatile struct amdgpu_fw_shared *fw_shared;
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+		for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
+			volatile struct amdgpu_fw_shared *fw_shared;
 
-		if (adev->vcn.harvest_config & (1 << i))
-			continue;
-		fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr;
-		fw_shared->present_flag_0 = 0;
-		fw_shared->sw_ring.is_enabled = false;
+			if (adev->vcn.harvest_config & (1 << i))
+				continue;
+			fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr;
+			fw_shared->present_flag_0 = 0;
+			fw_shared->sw_ring.is_enabled = false;
+		}
+
+		drm_dev_exit(idx);
 	}
 
 	if (amdgpu_sriov_vf(adev))
diff --git a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c
index aae25243eb10..d628b91846c9 100644
--- a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c
+++ b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c
@@ -405,6 +405,8 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr)
 				UCODE_ID_MEC_STORAGE, &toc->entry[toc->num_entries++]),
 				"Failed to Get Firmware Entry.", r = -EINVAL; goto failed);
 	}
+
+	/* AG TODO Can't call drm_dev_enter/exit because access adev->ddev here ... */
 	memcpy_toio(smu_data->header_buffer.kaddr, smu_data->toc,
 		    sizeof(struct SMU_DRAMData_TOC));
 	smum_send_msg_to_smc_with_parameter(hwmgr,
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 11/27] drm/sched: Make timeout timer rearm conditional.

[Andrey Grodzovsky]

We don't want to rearm the timer if driver hook reports
that the device is gone.

v5: Update drm_gpu_sched_stat values in code.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/scheduler/sched_main.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index d82a7ebf6099..908b0b56032d 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -314,6 +314,7 @@ static void drm_sched_job_timedout(struct work_struct *work)
 {
 	struct drm_gpu_scheduler *sched;
 	struct drm_sched_job *job;
+	enum drm_gpu_sched_stat status = DRM_GPU_SCHED_STAT_NOMINAL;
 
 	sched = container_of(work, struct drm_gpu_scheduler, work_tdr.work);
 
@@ -331,7 +332,7 @@ static void drm_sched_job_timedout(struct work_struct *work)
 		list_del_init(&job->list);
 		spin_unlock(&sched->job_list_lock);
 
-		job->sched->ops->timedout_job(job);
+		status = job->sched->ops->timedout_job(job);
 
 		/*
 		 * Guilty job did complete and hence needs to be manually removed
@@ -345,9 +346,11 @@ static void drm_sched_job_timedout(struct work_struct *work)
 		spin_unlock(&sched->job_list_lock);
 	}
 
-	spin_lock(&sched->job_list_lock);
-	drm_sched_start_timeout(sched);
-	spin_unlock(&sched->job_list_lock);
+	if (status != DRM_GPU_SCHED_STAT_ENODEV) {
+		spin_lock(&sched->job_list_lock);
+		drm_sched_start_timeout(sched);
+		spin_unlock(&sched->job_list_lock);
+	}
 }
 
  /**
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 12/27] drm/amdgpu: Prevent any job recoveries after device is unplugged.

[Andrey Grodzovsky]

Return DRM_TASK_STATUS_ENODEV back to the scheduler when device
is not present so they timeout timer will not be rearmed.

v5: Update to match updated return values in enum drm_gpu_sched_stat

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index 759b34799221..d33e6d97cc89 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -25,6 +25,8 @@
 #include <linux/wait.h>
 #include <linux/sched.h>
 
+#include <drm/drm_drv.h>
+
 #include "amdgpu.h"
 #include "amdgpu_trace.h"
 
@@ -34,6 +36,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
 	struct amdgpu_job *job = to_amdgpu_job(s_job);
 	struct amdgpu_task_info ti;
 	struct amdgpu_device *adev = ring->adev;
+	int idx;
+
+	if (!drm_dev_enter(&adev->ddev, &idx)) {
+		DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s",
+			 __func__, s_job->sched->name);
+
+		/* Effectively the job is aborted as the device is gone */
+		return DRM_GPU_SCHED_STAT_ENODEV;
+	}
 
 	memset(&ti, 0, sizeof(struct amdgpu_task_info));
 
@@ -41,7 +52,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
 	    amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) {
 		DRM_ERROR("ring %s timeout, but soft recovered\n",
 			  s_job->sched->name);
-		return DRM_GPU_SCHED_STAT_NOMINAL;
+		goto exit;
 	}
 
 	amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti);
@@ -53,13 +64,15 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
 
 	if (amdgpu_device_should_recover_gpu(ring->adev)) {
 		amdgpu_device_gpu_recover(ring->adev, job);
-		return DRM_GPU_SCHED_STAT_NOMINAL;
 	} else {
 		drm_sched_suspend_timeout(&ring->sched);
 		if (amdgpu_sriov_vf(adev))
 			adev->virt.tdr_debug = true;
-		return DRM_GPU_SCHED_STAT_NOMINAL;
 	}
+
+exit:
+	drm_dev_exit(idx);
+	return DRM_GPU_SCHED_STAT_NOMINAL;
 }
 
 int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 13/27] drm/amdgpu: When filizing the fence driver. stop scheduler first.

[Andrey Grodzovsky]

No point calling amdgpu_fence_wait_empty before stopping the
SW scheduler otherwise there is always a chance another job sneaked
in after the wait.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index 34d51e962799..fd9282637549 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -533,6 +533,12 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 
 		if (!ring || !ring->fence_drv.initialized)
 			continue;
+
+		/* Stop any new job submissions from sched before flushing the ring */
+		/* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */
+		if (!ring->no_scheduler)
+			drm_sched_fini(&ring->sched);
+
 		r = amdgpu_fence_wait_empty(ring);
 		if (r) {
 			/* no need to trigger GPU reset as we are unloading */
@@ -541,8 +547,7 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 		if (ring->fence_drv.irq_src)
 			amdgpu_irq_put(adev, ring->fence_drv.irq_src,
 				       ring->fence_drv.irq_type);
-		if (!ring->no_scheduler)
-			drm_sched_fini(&ring->sched);
+
 		del_timer_sync(&ring->fence_drv.fallback_timer);
 	}
 }
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 14/27] drm/amdgpu: Fix hang on device removal.

[Andrey Grodzovsky]

If removing while commands in flight you cannot wait to flush the
HW fences on a ring since the device is gone.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index fd9282637549..2670201e78d3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -37,6 +37,7 @@
 #include <linux/pm_runtime.h>
 
 #include <drm/drm_debugfs.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_trace.h"
@@ -525,8 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
  */
 void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 {
-	unsigned i, j;
-	int r;
+	int i, r;
 
 	for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
 		struct amdgpu_ring *ring = adev->rings[i];
@@ -539,11 +539,15 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 		if (!ring->no_scheduler)
 			drm_sched_fini(&ring->sched);
 
-		r = amdgpu_fence_wait_empty(ring);
-		if (r) {
-			/* no need to trigger GPU reset as we are unloading */
+		/* You can't wait for HW to signal if it's gone */
+		if (!drm_dev_is_unplugged(&adev->ddev))
+			r = amdgpu_fence_wait_empty(ring);
+		else
+			r = -ENODEV;
+		/* no need to trigger GPU reset as we are unloading */
+		if (r)
 			amdgpu_fence_driver_force_completion(ring);
-		}
+
 		if (ring->fence_drv.irq_src)
 			amdgpu_irq_put(adev, ring->fence_drv.irq_src,
 				       ring->fence_drv.irq_type);
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Andrey Grodzovsky]

Problem: If scheduler is already stopped by the time sched_entity
is released and entity's job_queue not empty I encountred
a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle
never becomes false.

Fix: In drm_sched_fini detach all sched_entities from the
scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
Also wakeup all those processes stuck in sched_entity flushing
as the scheduler main thread which wakes them up is stopped by now.

v2:
Reverse order of drm_sched_rq_remove_entity and marking
s_entity as stopped to prevent reinserion back to rq due
to race.

v3:
Drop drm_sched_rq_remove_entity, only modify entity->stopped
and check for it in drm_sched_entity_is_idle

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
---
 drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
 drivers/gpu/drm/scheduler/sched_main.c   | 24 ++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index f0790e9471d1..cb58f692dad9 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity)
 	rmb(); /* for list_empty to work without lock */
 
 	if (list_empty(&entity->list) ||
-	    spsc_queue_count(&entity->job_queue) == 0)
+	    spsc_queue_count(&entity->job_queue) == 0 ||
+	    entity->stopped)
 		return true;
 
 	return false;
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 908b0b56032d..ba087354d0a8 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
  */
 void drm_sched_fini(struct drm_gpu_scheduler *sched)
 {
+	struct drm_sched_entity *s_entity;
+	int i;
+
 	if (sched->thread)
 		kthread_stop(sched->thread);
 
+	for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {
+		struct drm_sched_rq *rq = &sched->sched_rq[i];
+
+		if (!rq)
+			continue;
+
+		spin_lock(&rq->lock);
+		list_for_each_entry(s_entity, &rq->entities, list)
+			/*
+			 * Prevents reinsertion and marks job_queue as idle,
+			 * it will removed from rq in drm_sched_entity_fini
+			 * eventually
+			 */
+			s_entity->stopped = true;
+		spin_unlock(&rq->lock);
+
+	}
+
+	/* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */
+	wake_up_all(&sched->job_scheduled);
+
 	/* Confirm no work left behind accessing device structures */
 	cancel_delayed_work_sync(&sched->work_tdr);
 
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 16/27] drm/amdgpu: Unmap all MMIO mappings

[Andrey Grodzovsky]

Access to those must be prevented post pci_remove

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  5 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 38 ++++++++++++++++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 28 ++++++++++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  5 +++
 4 files changed, 71 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 30a24db5f4d1..3e4755fc10c8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1056,6 +1056,11 @@ struct amdgpu_device {
 	struct pci_saved_state          *pci_state;
 
 	struct list_head                device_bo_list;
+
+	/* List of all MMIO BOs */
+	struct list_head                mmio_list;
+	struct mutex                    mmio_list_lock;
+
 };
 
 static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 22b09c4db255..3ddad6cba62d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3320,6 +3320,9 @@ int amdgpu_device_init(struct amdgpu_device *adev,
 	INIT_LIST_HEAD(&adev->shadow_list);
 	mutex_init(&adev->shadow_list_lock);
 
+	INIT_LIST_HEAD(&adev->mmio_list);
+	mutex_init(&adev->mmio_list_lock);
+
 	INIT_DELAYED_WORK(&adev->delayed_init_work,
 			  amdgpu_device_delayed_init_work_handler);
 	INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work,
@@ -3636,6 +3639,36 @@ static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
 	spin_unlock(&adev->mman.bdev.lru_lock);
 }
 
+static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
+{
+	struct amdgpu_bo *bo;
+
+	/* Clear all CPU mappings pointing to this device */
+	unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1);
+
+	/* Unmap all MMIO mapped kernel BOs */
+	mutex_lock(&adev->mmio_list_lock);
+	list_for_each_entry(bo, &adev->mmio_list, mmio_list) {
+		amdgpu_bo_kunmap(bo);
+		if (*bo->kmap_ptr)
+			*bo->kmap_ptr = NULL;
+	}
+	mutex_unlock(&adev->mmio_list_lock);
+
+	/* Unmap all mapped bars - Doorbell, registers and VRAM */
+	amdgpu_device_doorbell_fini(adev);
+
+	iounmap(adev->rmmio);
+	adev->rmmio = NULL;
+	if (adev->mman.aper_base_kaddr)
+		iounmap(adev->mman.aper_base_kaddr);
+	adev->mman.aper_base_kaddr = NULL;
+
+	/* Memory manager related */
+	arch_phys_wc_del(adev->gmc.vram_mtrr);
+	arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size);
+}
+
 /**
  * amdgpu_device_fini - tear down the driver
  *
@@ -3683,6 +3716,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
 	amdgpu_clear_dma_mappings(adev);
 
 	amdgpu_gart_dummy_page_fini(adev);
+
+	amdgpu_device_unmap_mmio(adev);
 }
 
 void amdgpu_device_fini_sw(struct amdgpu_device *adev)
@@ -3713,9 +3748,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev)
 	if (adev->rio_mem)
 		pci_iounmap(adev->pdev, adev->rio_mem);
 	adev->rio_mem = NULL;
-	iounmap(adev->rmmio);
-	adev->rmmio = NULL;
-	amdgpu_device_doorbell_fini(adev);
 
 	if (IS_ENABLED(CONFIG_PERF_EVENTS))
 		amdgpu_pmu_fini(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 62d829f5e62c..9b05e3b96fa0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -531,6 +531,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
 		return -ENOMEM;
 	drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size);
 	INIT_LIST_HEAD(&bo->shadow_list);
+
+	INIT_LIST_HEAD(&bo->mmio_list);
+
 	bo->vm_bo = NULL;
 	bo->preferred_domains = bp->preferred_domain ? bp->preferred_domain :
 		bp->domain;
@@ -774,9 +777,21 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr)
 	if (r)
 		return r;
 
-	if (ptr)
+	if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {
+		struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+
+		mutex_lock(&adev->mmio_list_lock);
+		list_add_tail(&bo->mmio_list, &adev->mmio_list);
+		mutex_unlock(&adev->mmio_list_lock);
+	}
+
+	if (ptr) {
 		*ptr = amdgpu_bo_kptr(bo);
 
+		if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap)
+			bo->kmap_ptr = ptr;
+	}
+
 	return 0;
 }
 
@@ -804,8 +819,17 @@ void *amdgpu_bo_kptr(struct amdgpu_bo *bo)
  */
 void amdgpu_bo_kunmap(struct amdgpu_bo *bo)
 {
-	if (bo->kmap.bo)
+	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+
+	if (bo->kmap.bo) {
+		if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {
+			mutex_lock(&adev->mmio_list_lock);
+			list_del_init(&bo->mmio_list);
+			mutex_unlock(&adev->mmio_list_lock);
+		}
+
 		ttm_bo_kunmap(&bo->kmap);
+	}
 }
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index 5ae8555ef275..3129d9bbfa22 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -112,6 +112,11 @@ struct amdgpu_bo {
 	struct kgd_mem                  *kfd_bo;
 
 	struct list_head		bo;
+
+	struct list_head                mmio_list;
+	/* Address of kernel VA pointer to MMIO so they can be updated post remap */
+	void				**kmap_ptr;
+
 };
 
 static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 17/27] drm/amdgpu: Add rw_sem to pushing job into sched queue

[Andrey Grodzovsky]

Will be later used block further submissions once device is
removed. Also complete schedule fence if scheduling failed
due to submission blocking.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  3 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 13 ++++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_job.c    | 14 +++++++++++++-
 4 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 3e4755fc10c8..0db0ba4fba89 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1057,6 +1057,9 @@ struct amdgpu_device {
 
 	struct list_head                device_bo_list;
 
+	bool				stop_job_submissions;
+	struct rw_semaphore		sched_fence_completion_sem;
+
 	/* List of all MMIO BOs */
 	struct list_head                mmio_list;
 	struct mutex                    mmio_list_lock;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 3e240b952e79..ac092a5eb4e7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -1256,7 +1256,18 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p,
 
 	trace_amdgpu_cs_ioctl(job);
 	amdgpu_vm_bo_trace_cs(&fpriv->vm, &p->ticket);
-	drm_sched_entity_push_job(&job->base, entity);
+
+	down_read(&p->adev->sched_fence_completion_sem);
+	if (!p->adev->stop_job_submissions) {
+		drm_sched_entity_push_job(&job->base, entity);
+	} else {
+		dma_fence_set_error(&job->base.s_fence->scheduled, -ENODEV);
+		dma_fence_set_error(&job->base.s_fence->finished, -ENODEV);
+		dma_fence_signal(&job->base.s_fence->scheduled);
+		dma_fence_signal(&job->base.s_fence->finished);
+	}
+
+	up_read(&p->adev->sched_fence_completion_sem);
 
 	amdgpu_vm_move_to_lru_tail(p->adev, &fpriv->vm);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 3ddad6cba62d..33e8e9e1d1fe 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3302,6 +3302,7 @@ int amdgpu_device_init(struct amdgpu_device *adev,
 	init_rwsem(&adev->reset_sem);
 	mutex_init(&adev->psp.mutex);
 	mutex_init(&adev->notifier_lock);
+	init_rwsem(&adev->sched_fence_completion_sem);
 
 	r = amdgpu_device_check_arguments(adev);
 	if (r)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index d33e6d97cc89..26d8b79ea165 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -162,6 +162,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
 		      void *owner, struct dma_fence **f)
 {
 	int r;
+	struct amdgpu_ring *ring = to_amdgpu_ring(job->base.sched);
 
 	if (!f)
 		return -EINVAL;
@@ -172,7 +173,18 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity,
 
 	*f = dma_fence_get(&job->base.s_fence->finished);
 	amdgpu_job_free_resources(job);
-	drm_sched_entity_push_job(&job->base, entity);
+
+	down_read(&ring->adev->sched_fence_completion_sem);
+	if (!ring->adev->stop_job_submissions) {
+		drm_sched_entity_push_job(&job->base, entity);
+	} else {
+		dma_fence_set_error(&job->base.s_fence->scheduled, -ENODEV);
+		dma_fence_set_error(&job->base.s_fence->finished, -ENODEV);
+		dma_fence_signal(&job->base.s_fence->scheduled);
+		dma_fence_signal(&job->base.s_fence->finished);
+
+	}
+	up_read(&ring->adev->sched_fence_completion_sem);
 
 	return 0;
 }
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 18/27] drm/sched: Expose drm_sched_entity_kill_jobs

[Andrey Grodzovsky]

Will be used to complete all schedulte fences on device
remove

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/scheduler/sched_entity.c | 3 ++-
 include/drm/gpu_scheduler.h              | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
index cb58f692dad9..9ff4bfd8f548 100644
--- a/drivers/gpu/drm/scheduler/sched_entity.c
+++ b/drivers/gpu/drm/scheduler/sched_entity.c
@@ -219,7 +219,7 @@ static void drm_sched_entity_kill_jobs_cb(struct dma_fence *f,
  * Makes sure that all remaining jobs in an entity are killed before it is
  * destroyed.
  */
-static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
+void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
 {
 	struct drm_sched_job *job;
 	int r;
@@ -249,6 +249,7 @@ static void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity)
 			DRM_ERROR("fence add callback failed (%d)\n", r);
 	}
 }
+EXPORT_SYMBOL(drm_sched_entity_kill_jobs);
 
 /**
  * drm_sched_entity_cleanup - Destroy a context entity
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index f888b5e9583a..9601d5b966ba 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -339,6 +339,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,
 			  unsigned int num_sched_list,
 			  atomic_t *guilty);
 long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout);
+void drm_sched_entity_kill_jobs(struct drm_sched_entity *entity);
 void drm_sched_entity_fini(struct drm_sched_entity *entity);
 void drm_sched_entity_destroy(struct drm_sched_entity *entity);
 void drm_sched_entity_select_rq(struct drm_sched_entity *entity);
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 19/27] drm/amdgpu: Finilise device fences on device remove.

[Andrey Grodzovsky]

Make sure all fecens dependent on HW present are force signaled
when handling device removal. This helpes later to scope all HW
accesing code such as IOCTLs in drm_dev_enter/exit and use
drm_dev_unplug as synchronization point past which we know HW
will not be accessed anymore outside of pci remove driver callback.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 98 ++++++++++++++++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  6 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c  | 12 +--
 4 files changed, 103 insertions(+), 15 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 0db0ba4fba89..df6c5ed676b1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1374,6 +1374,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev);
 bool amdgpu_device_cache_pci_state(struct pci_dev *pdev);
 bool amdgpu_device_load_pci_state(struct pci_dev *pdev);
 
+void amdgpu_finilize_device_fences(struct drm_device *dev);
+
 #include "amdgpu_object.h"
 
 static inline bool amdgpu_is_tmz(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 33e8e9e1d1fe..55afc11c17e6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3692,15 +3692,12 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
 		amdgpu_virt_fini_data_exchange(adev);
 	}
 
-	/* disable all interrupts */
-	amdgpu_irq_disable_all(adev);
 	if (adev->mode_info.mode_config_initialized){
 		if (!amdgpu_device_has_dc_support(adev))
 			drm_helper_force_disable_all(adev_to_drm(adev));
 		else
 			drm_atomic_helper_shutdown(adev_to_drm(adev));
 	}
-	amdgpu_fence_driver_fini_hw(adev);
 
 	if (adev->pm_sysfs_en)
 		amdgpu_pm_sysfs_fini(adev);
@@ -4567,14 +4564,19 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev,
 	return true;
 }
 
-static void amdgpu_device_unlock_adev(struct amdgpu_device *adev)
+static void amdgpu_device_unlock_adev_imp(struct amdgpu_device *adev, bool skip_in_gpu_reset)
 {
 	amdgpu_vf_error_trans_all(adev);
 	adev->mp1_state = PP_MP1_STATE_NONE;
-	atomic_set(&adev->in_gpu_reset, 0);
+	!skip_in_gpu_reset ? atomic_set(&adev->in_gpu_reset, 0) : 0;
 	up_write(&adev->reset_sem);
 }
 
+static void amdgpu_device_unlock_adev(struct amdgpu_device *adev)
+{
+	amdgpu_device_unlock_adev_imp(adev, false);
+}
+
 /*
  * to lockup a list of amdgpu devices in a hive safely, if not a hive
  * with multiple nodes, it will be similar as amdgpu_device_lock_adev.
@@ -5321,3 +5323,89 @@ bool amdgpu_device_load_pci_state(struct pci_dev *pdev)
 }
 
 
+static void amdgpu_finilize_schedulded_fences(struct amdgpu_ctx_mgr *mgr)
+{
+	struct amdgpu_ctx *ctx;
+	struct idr *idp;
+	uint32_t id, i, j;
+
+	idp = &mgr->ctx_handles;
+
+	idr_for_each_entry(idp, ctx, id) {
+		for (i = 0; i < AMDGPU_HW_IP_NUM; ++i) {
+			for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) {
+				struct drm_sched_entity *entity;
+
+				if (!ctx->entities[i][j])
+					continue;
+
+				entity = &ctx->entities[i][j]->entity;
+				drm_sched_entity_kill_jobs(entity);
+			}
+		}
+	}
+}
+
+/**
+ * amdgpu_finilize_device_fences() - Finilize all device fences
+ * @pdev: pointer to PCI device
+ *
+ * Will disable and finilise ISRs and will signal all fences
+ * that might hang if HW is gone
+ */
+void amdgpu_finilize_device_fences(struct drm_device *dev)
+{
+	struct amdgpu_device *adev = drm_to_adev(dev);
+	struct drm_file *file;
+
+	/*
+	 *  Block TDRs from further execution by setting adev->in_gpu_reset
+	 *  instead of holding full reset lock in order to not deadlock
+	 *  further ahead against any thread locking the reset lock when we
+	 *  wait for it's completion
+	 */
+	while (!amdgpu_device_lock_adev(adev, NULL))
+		amdgpu_cancel_all_tdr(adev);
+	amdgpu_device_unlock_adev_imp(adev, true);
+
+
+	/* disable all HW interrupts */
+	amdgpu_irq_disable_all(adev);
+
+	/* stop and flush all in flight HW interrupts handlers */
+	disable_irq(pci_irq_vector(adev->pdev, 0));
+
+	/*
+	 * Stop SW GPU schedulers and force completion on all HW fences. Since
+	 * in the prev. step all ISRs were disabled and completed the
+	 * HW fence array is idle (no insertions or extractions) and so it's
+	 * safe to iterate it bellow.
+	 * After this step all HW fences in the system are signaled. As a result
+	 * also all the scheduler 'finished' fences are also signaled.
+	 */
+	amdgpu_fence_driver_fini_hw(adev);
+
+	/*
+	 * Reject any further jobs to any scheduler entity queue. After this
+	 * step no new insertions and because schedulers are stopped also no
+	 * new extractions.
+	 */
+	down_read(&adev->sched_fence_completion_sem);
+	adev->stop_job_submissions = true;
+	up_read(&adev->sched_fence_completion_sem);
+
+	/*
+	 * Complete all scheduler 'scheduled' fences currently pending.
+	 * It's OK if new contexts and sched entities are concurrently
+	 * still created as they will fail in pushing jobs to SW queues
+	 * and their schedule fences will be signaled with error
+	 */
+	mutex_lock(&adev->ddev.filelist_mutex);
+	list_for_each_entry(file, &adev->ddev.filelist, lhead) {
+		struct amdgpu_fpriv *fpriv = file->driver_priv;
+		amdgpu_finilize_schedulded_fences(&fpriv->ctx_mgr);
+	}
+	mutex_unlock(&adev->ddev.filelist_mutex);
+}
+
+
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index f799c40d7e72..8a19b8dd02ee 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -1249,6 +1249,12 @@ amdgpu_pci_remove(struct pci_dev *pdev)
 {
 	struct drm_device *dev = pci_get_drvdata(pdev);
 
+	/*
+	 * Force completion of all device related fences that might hang us when
+	 * synchronizing SRCU in the following step.
+	 */
+	amdgpu_finilize_device_fences(dev);
+
 	drm_dev_unplug(dev);
 	amdgpu_driver_unload_kms(dev);
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index 2670201e78d3..af592b28cd35 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -526,7 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
  */
 void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 {
-	int i, r;
+	int i;
 
 	for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
 		struct amdgpu_ring *ring = adev->rings[i];
@@ -535,18 +535,10 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 			continue;
 
 		/* Stop any new job submissions from sched before flushing the ring */
-		/* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */
 		if (!ring->no_scheduler)
 			drm_sched_fini(&ring->sched);
 
-		/* You can't wait for HW to signal if it's gone */
-		if (!drm_dev_is_unplugged(&adev->ddev))
-			r = amdgpu_fence_wait_empty(ring);
-		else
-			r = -ENODEV;
-		/* no need to trigger GPU reset as we are unloading */
-		if (r)
-			amdgpu_fence_driver_force_completion(ring);
+		amdgpu_fence_driver_force_completion(ring);
 
 		if (ring->fence_drv.irq_src)
 			amdgpu_irq_put(adev, ring->fence_drv.irq_src,
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

With this calling drm_dev_unplug will flush and block
all in flight IOCTLs

Also, add feature such that if device supports graceful unplug
we enclose entire IOCTL in SRCU critical section.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/drm_ioctl.c | 15 +++++++++++++--
 include/drm/drm_drv.h       |  6 ++++++
 2 files changed, 19 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
index d273d1a8603a..5882ef2183bb 100644
--- a/drivers/gpu/drm/drm_ioctl.c
+++ b/drivers/gpu/drm/drm_ioctl.c
@@ -815,7 +815,7 @@ long drm_ioctl(struct file *filp,
 	const struct drm_ioctl_desc *ioctl = NULL;
 	drm_ioctl_t *func;
 	unsigned int nr = DRM_IOCTL_NR(cmd);
-	int retcode = -EINVAL;
+	int idx, retcode = -EINVAL;
 	char stack_kdata[128];
 	char *kdata = NULL;
 	unsigned int in_size, out_size, drv_size, ksize;
@@ -884,7 +884,18 @@ long drm_ioctl(struct file *filp,
 	if (ksize > in_size)
 		memset(kdata + in_size, 0, ksize - in_size);
 
-	retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);
+	if (drm_core_check_feature(dev, DRIVER_HOTUNPLUG_SUPPORT)) {
+		if (drm_dev_enter(dev, &idx)) {
+			retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);
+			drm_dev_exit(idx);
+		} else {
+			retcode = -ENODEV;
+			goto err_i1;
+		}
+	} else {
+		retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);
+	}
+
 	if (copy_to_user((void __user *)arg, kdata, out_size) != 0)
 		retcode = -EFAULT;
 
diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
index b439ae1921b8..63e05cec46c1 100644
--- a/include/drm/drm_drv.h
+++ b/include/drm/drm_drv.h
@@ -94,6 +94,12 @@ enum drm_driver_feature {
 	 * synchronization of command submission.
 	 */
 	DRIVER_SYNCOBJ_TIMELINE         = BIT(6),
+	/**
+	 * @DRIVER_NO_HOTUNPLUG_SUPPORT:
+	 *
+	 * Driver support gracefull remove.
+	 */
+	DRIVER_HOTUNPLUG_SUPPORT         = BIT(7),
 
 	/* IMPORTANT: Below are all the legacy flags, add new ones above. */
 
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 21/27] drm/amdgpu: Add support for hot-unplug feature at DRM level.

[Andrey Grodzovsky]

To allow scoping DRM IOCTLs with drm_dev_enter/exit.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 8a19b8dd02ee..d0f34f230ef3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -1579,7 +1579,7 @@ static const struct drm_driver amdgpu_kms_driver = {
 	    DRIVER_ATOMIC |
 	    DRIVER_GEM |
 	    DRIVER_RENDER | DRIVER_MODESET | DRIVER_SYNCOBJ |
-	    DRIVER_SYNCOBJ_TIMELINE,
+	    DRIVER_SYNCOBJ_TIMELINE | DRIVER_HOTUNPLUG_SUPPORT,
 	.open = amdgpu_driver_open_kms,
 	.postclose = amdgpu_driver_postclose_kms,
 	.lastclose = amdgpu_driver_lastclose_kms,
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 22/27] drm/amd/display: Scope all DM queued work with drm_dev_enter/exit

[Andrey Grodzovsky]

To allow completion and further block of HW accesses post device PCI
remove.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 .../amd/display/amdgpu_dm/amdgpu_dm_hdcp.c    | 124 +++++++++++-------
 .../drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c |  24 +++-
 2 files changed, 98 insertions(+), 50 deletions(-)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
index 0cdbfcd475ec..81ea5a1ea46b 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
@@ -28,6 +28,7 @@
 #include "amdgpu_dm.h"
 #include "dm_helpers.h"
 #include <drm/drm_hdcp.h>
+#include <drm/drm_drv.h>
 #include "hdcp_psp.h"
 
 /*
@@ -260,20 +261,27 @@ void hdcp_handle_cpirq(struct hdcp_workqueue *hdcp_work, unsigned int link_index
 static void event_callback(struct work_struct *work)
 {
 	struct hdcp_workqueue *hdcp_work;
+	int idx;
 
 	hdcp_work = container_of(to_delayed_work(work), struct hdcp_workqueue,
 				      callback_dwork);
 
-	mutex_lock(&hdcp_work->mutex);
+	if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
 
-	cancel_delayed_work(&hdcp_work->callback_dwork);
+		mutex_lock(&hdcp_work->mutex);
 
-	mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CALLBACK,
-			       &hdcp_work->output);
+		cancel_delayed_work(&hdcp_work->callback_dwork);
+
+		mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CALLBACK,
+				       &hdcp_work->output);
 
-	process_output(hdcp_work);
+		process_output(hdcp_work);
 
-	mutex_unlock(&hdcp_work->mutex);
+		mutex_unlock(&hdcp_work->mutex);
+
+		drm_dev_exit(idx);
+
+	}
 
 
 }
@@ -284,34 +292,41 @@ static void event_property_update(struct work_struct *work)
 	struct amdgpu_dm_connector *aconnector = hdcp_work->aconnector;
 	struct drm_device *dev = hdcp_work->aconnector->base.dev;
 	long ret;
+	int idx;
+
+	if (drm_dev_enter(dev, &idx)) {
+
+		drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+		mutex_lock(&hdcp_work->mutex);
 
-	drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
-	mutex_lock(&hdcp_work->mutex);
 
+		if (aconnector->base.state->commit) {
+			ret = wait_for_completion_interruptible_timeout(&aconnector->base.state->commit->hw_done, 10 * HZ);
 
-	if (aconnector->base.state->commit) {
-		ret = wait_for_completion_interruptible_timeout(&aconnector->base.state->commit->hw_done, 10 * HZ);
+			if (ret == 0) {
+				DRM_ERROR("HDCP state unknown! Setting it to DESIRED");
+				hdcp_work->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF;
+			}
+		}
 
-		if (ret == 0) {
-			DRM_ERROR("HDCP state unknown! Setting it to DESIRED");
-			hdcp_work->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF;
+		if (hdcp_work->encryption_status != MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF) {
+			if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE0 &&
+			    hdcp_work->encryption_status <= MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE0_ON)
+				drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED);
+			else if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE1 &&
+				 hdcp_work->encryption_status == MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE1_ON)
+				drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED);
+		} else {
+			drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_DESIRED);
 		}
-	}
 
-	if (hdcp_work->encryption_status != MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF) {
-		if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE0 &&
-		    hdcp_work->encryption_status <= MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE0_ON)
-			drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED);
-		else if (aconnector->base.state->hdcp_content_type == DRM_MODE_HDCP_CONTENT_TYPE1 &&
-			 hdcp_work->encryption_status == MOD_HDCP_ENCRYPTION_STATUS_HDCP2_TYPE1_ON)
-			drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_ENABLED);
-	} else {
-		drm_hdcp_update_content_protection(&aconnector->base, DRM_MODE_CONTENT_PROTECTION_DESIRED);
-	}
 
+		mutex_unlock(&hdcp_work->mutex);
+		drm_modeset_unlock(&dev->mode_config.connection_mutex);
+
+		drm_dev_exit(idx);
 
-	mutex_unlock(&hdcp_work->mutex);
-	drm_modeset_unlock(&dev->mode_config.connection_mutex);
+	}
 }
 
 static void event_property_validate(struct work_struct *work)
@@ -320,58 +335,77 @@ static void event_property_validate(struct work_struct *work)
 		container_of(to_delayed_work(work), struct hdcp_workqueue, property_validate_dwork);
 	struct mod_hdcp_display_query query;
 	struct amdgpu_dm_connector *aconnector = hdcp_work->aconnector;
+	int idx;
 
 	if (!aconnector)
 		return;
 
-	mutex_lock(&hdcp_work->mutex);
+	if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
 
-	query.encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF;
-	mod_hdcp_query_display(&hdcp_work->hdcp, aconnector->base.index, &query);
+		mutex_lock(&hdcp_work->mutex);
 
-	if (query.encryption_status != hdcp_work->encryption_status) {
-		hdcp_work->encryption_status = query.encryption_status;
-		schedule_work(&hdcp_work->property_update_work);
-	}
+		query.encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF;
+		mod_hdcp_query_display(&hdcp_work->hdcp, aconnector->base.index, &query);
+
+		if (query.encryption_status != hdcp_work->encryption_status) {
+			hdcp_work->encryption_status = query.encryption_status;
+			schedule_work(&hdcp_work->property_update_work);
+		}
+
+		mutex_unlock(&hdcp_work->mutex);
 
-	mutex_unlock(&hdcp_work->mutex);
+		drm_dev_exit(idx);
+
+	}
 }
 
 static void event_watchdog_timer(struct work_struct *work)
 {
 	struct hdcp_workqueue *hdcp_work;
+	int idx;
 
 	hdcp_work = container_of(to_delayed_work(work),
 				      struct hdcp_workqueue,
 				      watchdog_timer_dwork);
 
-	mutex_lock(&hdcp_work->mutex);
+	if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
 
-	cancel_delayed_work(&hdcp_work->watchdog_timer_dwork);
+		mutex_lock(&hdcp_work->mutex);
 
-	mod_hdcp_process_event(&hdcp_work->hdcp,
-			       MOD_HDCP_EVENT_WATCHDOG_TIMEOUT,
-			       &hdcp_work->output);
+		cancel_delayed_work(&hdcp_work->watchdog_timer_dwork);
 
-	process_output(hdcp_work);
+		mod_hdcp_process_event(&hdcp_work->hdcp,
+				       MOD_HDCP_EVENT_WATCHDOG_TIMEOUT,
+				       &hdcp_work->output);
 
-	mutex_unlock(&hdcp_work->mutex);
+		process_output(hdcp_work);
+
+		mutex_unlock(&hdcp_work->mutex);
+
+		drm_dev_exit(idx);
+	}
 
 }
 
 static void event_cpirq(struct work_struct *work)
 {
 	struct hdcp_workqueue *hdcp_work;
+	int idx;
 
 	hdcp_work = container_of(work, struct hdcp_workqueue, cpirq_work);
 
-	mutex_lock(&hdcp_work->mutex);
+	if (drm_dev_enter(hdcp_work->aconnector->base.dev, &idx)) {
 
-	mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CPIRQ, &hdcp_work->output);
+		mutex_lock(&hdcp_work->mutex);
 
-	process_output(hdcp_work);
+		mod_hdcp_process_event(&hdcp_work->hdcp, MOD_HDCP_EVENT_CPIRQ, &hdcp_work->output);
 
-	mutex_unlock(&hdcp_work->mutex);
+		process_output(hdcp_work);
+
+		mutex_unlock(&hdcp_work->mutex);
+
+		drm_dev_exit(idx);
+	}
 
 }
 
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
index e0000c180ed1..d8ee552d373e 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
@@ -30,6 +30,8 @@
 #include "amdgpu_dm.h"
 #include "amdgpu_dm_irq.h"
 
+#include <drm/drm_drv.h>
+
 /**
  * DOC: overview
  *
@@ -115,15 +117,27 @@ static void dm_irq_work_func(struct work_struct *work)
 		container_of(work, struct irq_list_head, work);
 	struct list_head *handler_list = &irq_list_head->head;
 	struct amdgpu_dm_irq_handler_data *handler_data;
+	int idx;
+
+
+	handler_data = list_first_entry_or_null(handler_list, struct amdgpu_dm_irq_handler_data, list);
+
+	if (!handler_data)
+		return;
 
-	list_for_each_entry(handler_data, handler_list, list) {
-		DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n",
+	if (drm_dev_enter(handler_data->dm->ddev, &idx)) {
+
+		list_for_each_entry(handler_data, handler_list, list) {
+			DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n",
+					handler_data->irq_source);
+
+			DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n",
 				handler_data->irq_source);
 
-		DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n",
-			handler_data->irq_source);
+			handler_data->handler(handler_data->handler_arg);
+		}
 
-		handler_data->handler(handler_data->handler_arg);
+		drm_dev_exit(idx);
 	}
 
 	/* Call a DAL subcomponent which registered for interrupt notification
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 23/27] drm/amd/powerplay: Scope all PM queued work with drm_dev_enter/exit

[Andrey Grodzovsky]

To allow completion and further block of HW accesses post device PCI
remove.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/pm/amdgpu_dpm.c       | 44 +++++++++++++----------
 drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c | 26 +++++++++++---
 2 files changed, 47 insertions(+), 23 deletions(-)

diff --git a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
index 8fb12afe3c96..649e10d52d17 100644
--- a/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
+++ b/drivers/gpu/drm/amd/pm/amdgpu_dpm.c
@@ -31,6 +31,7 @@
 #include "amdgpu_display.h"
 #include "hwmgr.h"
 #include <linux/power_supply.h>
+#include <drm/drm_drv.h>
 
 #define WIDTH_4K 3840
 
@@ -1316,29 +1317,36 @@ void amdgpu_dpm_thermal_work_handler(struct work_struct *work)
 	/* switch to the thermal state */
 	enum amd_pm_state_type dpm_state = POWER_STATE_TYPE_INTERNAL_THERMAL;
 	int temp, size = sizeof(temp);
+	int idx;
 
 	if (!adev->pm.dpm_enabled)
 		return;
 
-	if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_TEMP,
-				    (void *)&temp, &size)) {
-		if (temp < adev->pm.dpm.thermal.min_temp)
-			/* switch back the user state */
-			dpm_state = adev->pm.dpm.user_state;
-	} else {
-		if (adev->pm.dpm.thermal.high_to_low)
-			/* switch back the user state */
-			dpm_state = adev->pm.dpm.user_state;
-	}
-	mutex_lock(&adev->pm.mutex);
-	if (dpm_state == POWER_STATE_TYPE_INTERNAL_THERMAL)
-		adev->pm.dpm.thermal_active = true;
-	else
-		adev->pm.dpm.thermal_active = false;
-	adev->pm.dpm.state = dpm_state;
-	mutex_unlock(&adev->pm.mutex);
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
-	amdgpu_pm_compute_clocks(adev);
+		if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_GPU_TEMP,
+					    (void *)&temp, &size)) {
+			if (temp < adev->pm.dpm.thermal.min_temp)
+				/* switch back the user state */
+				dpm_state = adev->pm.dpm.user_state;
+		} else {
+			if (adev->pm.dpm.thermal.high_to_low)
+				/* switch back the user state */
+				dpm_state = adev->pm.dpm.user_state;
+		}
+		mutex_lock(&adev->pm.mutex);
+		if (dpm_state == POWER_STATE_TYPE_INTERNAL_THERMAL)
+			adev->pm.dpm.thermal_active = true;
+		else
+			adev->pm.dpm.thermal_active = false;
+		adev->pm.dpm.state = dpm_state;
+		mutex_unlock(&adev->pm.mutex);
+
+		amdgpu_pm_compute_clocks(adev);
+
+		drm_dev_exit(idx);
+
+	}
 }
 
 static struct amdgpu_ps *amdgpu_dpm_pick_power_state(struct amdgpu_device *adev,
diff --git a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
index d143ef1b460b..f034c8a5eb44 100644
--- a/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
+++ b/drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c
@@ -25,6 +25,8 @@
 #include <linux/firmware.h>
 #include <linux/pci.h>
 
+#include <drm/drm_drv.h>
+
 #include "amdgpu.h"
 #include "amdgpu_smu.h"
 #include "smu_internal.h"
@@ -904,21 +906,35 @@ static void smu_throttling_logging_work_fn(struct work_struct *work)
 {
 	struct smu_context *smu = container_of(work, struct smu_context,
 					       throttling_logging_work);
+	int idx;
+
+
+	if (drm_dev_enter(&smu->adev->ddev, &idx)) {
+
+		smu_log_thermal_throttling(smu);
 
-	smu_log_thermal_throttling(smu);
+		drm_dev_exit(idx);
+	}
 }
 
 static void smu_interrupt_work_fn(struct work_struct *work)
 {
 	struct smu_context *smu = container_of(work, struct smu_context,
 					       interrupt_work);
+	int idx;
 
-	mutex_lock(&smu->mutex);
+	if (drm_dev_enter(&smu->adev->ddev, &idx)) {
 
-	if (smu->ppt_funcs && smu->ppt_funcs->interrupt_work)
-		smu->ppt_funcs->interrupt_work(smu);
+		mutex_lock(&smu->mutex);
 
-	mutex_unlock(&smu->mutex);
+		if (smu->ppt_funcs && smu->ppt_funcs->interrupt_work)
+			smu->ppt_funcs->interrupt_work(smu);
+
+		mutex_unlock(&smu->mutex);
+
+		drm_dev_exit(idx);
+
+	}
 }
 
 static int smu_sw_init(void *handle)
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 24/27] drm/amdkfd: Scope all KFD queued work with drm_dev_enter/exit

[Andrey Grodzovsky]

To allow completion and further block of HW accesses post device PCI
remove.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c
index bc47f6a44456..563f02ab5b95 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c
@@ -43,8 +43,10 @@
 #include <linux/slab.h>
 #include <linux/device.h>
 #include <linux/kfifo.h>
+#include <drm/drm_drv.h>
 #include "kfd_priv.h"
 
+
 #define KFD_IH_NUM_ENTRIES 8192
 
 static void interrupt_wq(struct work_struct *);
@@ -145,15 +147,21 @@ static void interrupt_wq(struct work_struct *work)
 	struct kfd_dev *dev = container_of(work, struct kfd_dev,
 						interrupt_work);
 	uint32_t ih_ring_entry[KFD_MAX_RING_ENTRY_SIZE];
+	int idx;
 
 	if (dev->device_info->ih_ring_entry_size > sizeof(ih_ring_entry)) {
 		dev_err_once(kfd_chardev(), "Ring entry too small\n");
 		return;
 	}
 
-	while (dequeue_ih_ring_entry(dev, ih_ring_entry))
-		dev->device_info->event_interrupt_class->interrupt_wq(dev,
-								ih_ring_entry);
+	if (drm_dev_enter(dev->ddev, &idx)) {
+
+		while (dequeue_ih_ring_entry(dev, ih_ring_entry))
+			dev->device_info->event_interrupt_class->interrupt_wq(dev,
+									ih_ring_entry);
+
+		drm_dev_exit(idx);
+	}
 }
 
 bool interrupt_is_wanted(struct kfd_dev *dev,
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 25/27] drm/amdgpu: Scope all amdgpu queued work with drm_dev_enter/exit

[Andrey Grodzovsky]

To allow completion and further block of HW accesses post device PCI
remove.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 11 +++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 29 ++++++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    | 26 +++++++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c   | 28 ++++++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c    | 55 ++++++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c    | 43 ++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c    | 30 ++++++-----
 drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c    | 61 ++++++++++++----------
 drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c      | 10 +++-
 9 files changed, 189 insertions(+), 104 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
index 9edb35ba181b..f942496c2b35 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -30,6 +30,7 @@
 #include <linux/dma-buf.h>
 #include "amdgpu_xgmi.h"
 #include <uapi/linux/kfd_ioctl.h>
+#include <drm/drm_drv.h>
 
 /* Total memory size in system memory and all GPU VRAM. Used to
  * estimate worst case amount of memory to reserve for page tables
@@ -223,9 +224,15 @@ int amdgpu_amdkfd_post_reset(struct amdgpu_device *adev)
 void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd)
 {
 	struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
+	int idx;
 
-	if (amdgpu_device_should_recover_gpu(adev))
-		amdgpu_device_gpu_recover(adev, NULL);
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+
+		if (amdgpu_device_should_recover_gpu(adev))
+			amdgpu_device_gpu_recover(adev, NULL);
+
+		drm_dev_exit(idx);
+	}
 }
 
 int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 55afc11c17e6..c30e0b0596a5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -2674,24 +2674,35 @@ static void amdgpu_device_delayed_init_work_handler(struct work_struct *work)
 {
 	struct amdgpu_device *adev =
 		container_of(work, struct amdgpu_device, delayed_init_work.work);
-	int r;
+	int r, idx;
 
-	r = amdgpu_ib_ring_tests(adev);
-	if (r)
-		DRM_ERROR("ib ring test failed (%d).\n", r);
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+		r = amdgpu_ib_ring_tests(adev);
+		if (r)
+			DRM_ERROR("ib ring test failed (%d).\n", r);
+
+		drm_dev_exit(idx);
+	}
 }
 
 static void amdgpu_device_delay_enable_gfx_off(struct work_struct *work)
 {
 	struct amdgpu_device *adev =
 		container_of(work, struct amdgpu_device, gfx.gfx_off_delay_work.work);
+	int idx;
+
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+
+		mutex_lock(&adev->gfx.gfx_off_mutex);
+		if (!adev->gfx.gfx_off_state && !adev->gfx.gfx_off_req_count) {
+			if (!amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_GFX, true))
+				adev->gfx.gfx_off_state = true;
+		}
+		mutex_unlock(&adev->gfx.gfx_off_mutex);
+
+		drm_dev_exit(idx);
 
-	mutex_lock(&adev->gfx.gfx_off_mutex);
-	if (!adev->gfx.gfx_off_state && !adev->gfx.gfx_off_req_count) {
-		if (!amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_GFX, true))
-			adev->gfx.gfx_off_state = true;
 	}
-	mutex_unlock(&adev->gfx.gfx_off_mutex);
 }
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index a922154953a7..5eda0d0fc974 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -188,8 +188,15 @@ static void amdgpu_irq_handle_ih1(struct work_struct *work)
 {
 	struct amdgpu_device *adev = container_of(work, struct amdgpu_device,
 						  irq.ih1_work);
+	int idx;
 
-	amdgpu_ih_process(adev, &adev->irq.ih1);
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+
+		amdgpu_ih_process(adev, &adev->irq.ih1);
+
+		drm_dev_exit(idx);
+
+	}
 }
 
 /**
@@ -203,8 +210,14 @@ static void amdgpu_irq_handle_ih2(struct work_struct *work)
 {
 	struct amdgpu_device *adev = container_of(work, struct amdgpu_device,
 						  irq.ih2_work);
+	int idx;
+
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+
+		amdgpu_ih_process(adev, &adev->irq.ih2);
 
-	amdgpu_ih_process(adev, &adev->irq.ih2);
+		drm_dev_exit(idx);
+	}
 }
 
 /**
@@ -218,8 +231,15 @@ static void amdgpu_irq_handle_ih_soft(struct work_struct *work)
 {
 	struct amdgpu_device *adev = container_of(work, struct amdgpu_device,
 						  irq.ih_soft_work);
+	int idx;
+
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+
+		amdgpu_ih_process(adev, &adev->irq.ih_soft);
 
-	amdgpu_ih_process(adev, &adev->irq.ih_soft);
+		drm_dev_exit(idx);
+
+	}
 }
 
 /**
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c
index 8996cb4ed57a..1e8fd66c1e43 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c
@@ -29,6 +29,7 @@
 #include "amdgpu_pm.h"
 #include "soc15d.h"
 #include "soc15_common.h"
+#include <drm/drm_drv.h>
 
 #define JPEG_IDLE_TIMEOUT	msecs_to_jiffies(1000)
 
@@ -78,20 +79,25 @@ static void amdgpu_jpeg_idle_work_handler(struct work_struct *work)
 	struct amdgpu_device *adev =
 		container_of(work, struct amdgpu_device, jpeg.idle_work.work);
 	unsigned int fences = 0;
-	unsigned int i;
+	unsigned int i, idx;
 
-	for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) {
-		if (adev->jpeg.harvest_config & (1 << i))
-			continue;
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
-		fences += amdgpu_fence_count_emitted(&adev->jpeg.inst[i].ring_dec);
-	}
+		for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) {
+			if (adev->jpeg.harvest_config & (1 << i))
+				continue;
 
-	if (!fences && !atomic_read(&adev->jpeg.total_submission_cnt))
-		amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_JPEG,
-						       AMD_PG_STATE_GATE);
-	else
-		schedule_delayed_work(&adev->jpeg.idle_work, JPEG_IDLE_TIMEOUT);
+			fences += amdgpu_fence_count_emitted(&adev->jpeg.inst[i].ring_dec);
+		}
+
+		if (!fences && !atomic_read(&adev->jpeg.total_submission_cnt))
+			amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_JPEG,
+							       AMD_PG_STATE_GATE);
+		else
+			schedule_delayed_work(&adev->jpeg.idle_work, JPEG_IDLE_TIMEOUT);
+
+		drm_dev_exit(idx);
+	}
 }
 
 void amdgpu_jpeg_ring_begin_use(struct amdgpu_ring *ring)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index c0a16eac4923..97a6c028ac74 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -27,6 +27,7 @@
 #include <linux/uaccess.h>
 #include <linux/reboot.h>
 #include <linux/syscalls.h>
+#include <drm/drm_drv.h>
 
 #include "amdgpu.h"
 #include "amdgpu_ras.h"
@@ -1334,7 +1335,15 @@ static void amdgpu_ras_interrupt_process_handler(struct work_struct *work)
 	struct ras_manager *obj =
 		container_of(data, struct ras_manager, ih_data);
 
-	amdgpu_ras_interrupt_handler(obj);
+	int idx;
+
+	if (drm_dev_enter(&obj->adev->ddev, &idx)) {
+
+		amdgpu_ras_interrupt_handler(obj);
+
+		drm_dev_exit(idx);
+
+	}
 }
 
 int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev,
@@ -1565,31 +1574,37 @@ static void amdgpu_ras_do_recovery(struct work_struct *work)
 	struct amdgpu_device *remote_adev = NULL;
 	struct amdgpu_device *adev = ras->adev;
 	struct list_head device_list, *device_list_handle =  NULL;
+	int idx;
 
-	if (!ras->disable_ras_err_cnt_harvest) {
-		struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev);
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
-		/* Build list of devices to query RAS related errors */
-		if  (hive && adev->gmc.xgmi.num_physical_nodes > 1) {
-			device_list_handle = &hive->device_list;
-		} else {
-			INIT_LIST_HEAD(&device_list);
-			list_add_tail(&adev->gmc.xgmi.head, &device_list);
-			device_list_handle = &device_list;
-		}
+		if (!ras->disable_ras_err_cnt_harvest) {
+			struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev);
 
-		list_for_each_entry(remote_adev,
-				device_list_handle, gmc.xgmi.head) {
-			amdgpu_ras_query_err_status(remote_adev);
-			amdgpu_ras_log_on_err_counter(remote_adev);
+			/* Build list of devices to query RAS related errors */
+			if  (hive && adev->gmc.xgmi.num_physical_nodes > 1) {
+				device_list_handle = &hive->device_list;
+			} else {
+				INIT_LIST_HEAD(&device_list);
+				list_add_tail(&adev->gmc.xgmi.head, &device_list);
+				device_list_handle = &device_list;
+			}
+
+			list_for_each_entry(remote_adev,
+					device_list_handle, gmc.xgmi.head) {
+				amdgpu_ras_query_err_status(remote_adev);
+				amdgpu_ras_log_on_err_counter(remote_adev);
+			}
+
+			amdgpu_put_xgmi_hive(hive);
 		}
 
-		amdgpu_put_xgmi_hive(hive);
-	}
+		if (amdgpu_device_should_recover_gpu(ras->adev))
+			amdgpu_device_gpu_recover(ras->adev, NULL);
+		atomic_set(&ras->in_recovery, 0);
 
-	if (amdgpu_device_should_recover_gpu(ras->adev))
-		amdgpu_device_gpu_recover(ras->adev, NULL);
-	atomic_set(&ras->in_recovery, 0);
+		drm_dev_exit(idx);
+	}
 }
 
 /* alloc/realloc bps array */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
index df47f5ffa08f..19790afd2893 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
@@ -1234,29 +1234,36 @@ static void amdgpu_uvd_idle_work_handler(struct work_struct *work)
 	struct amdgpu_device *adev =
 		container_of(work, struct amdgpu_device, uvd.idle_work.work);
 	unsigned fences = 0, i, j;
+	int idx;
 
-	for (i = 0; i < adev->uvd.num_uvd_inst; ++i) {
-		if (adev->uvd.harvest_config & (1 << i))
-			continue;
-		fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring);
-		for (j = 0; j < adev->uvd.num_enc_rings; ++j) {
-			fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring_enc[j]);
+	if (drm_dev_enter(&adev->ddev, &idx)) {
+
+		for (i = 0; i < adev->uvd.num_uvd_inst; ++i) {
+			if (adev->uvd.harvest_config & (1 << i))
+				continue;
+			fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring);
+			for (j = 0; j < adev->uvd.num_enc_rings; ++j) {
+				fences += amdgpu_fence_count_emitted(&adev->uvd.inst[i].ring_enc[j]);
+			}
 		}
-	}
 
-	if (fences == 0) {
-		if (adev->pm.dpm_enabled) {
-			amdgpu_dpm_enable_uvd(adev, false);
+		if (fences == 0) {
+			if (adev->pm.dpm_enabled) {
+				amdgpu_dpm_enable_uvd(adev, false);
+			} else {
+				amdgpu_asic_set_uvd_clocks(adev, 0, 0);
+				/* shutdown the UVD block */
+				amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_UVD,
+								       AMD_PG_STATE_GATE);
+				amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_UVD,
+								       AMD_CG_STATE_GATE);
+			}
 		} else {
-			amdgpu_asic_set_uvd_clocks(adev, 0, 0);
-			/* shutdown the UVD block */
-			amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_UVD,
-							       AMD_PG_STATE_GATE);
-			amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_UVD,
-							       AMD_CG_STATE_GATE);
+			schedule_delayed_work(&adev->uvd.idle_work, UVD_IDLE_TIMEOUT);
 		}
-	} else {
-		schedule_delayed_work(&adev->uvd.idle_work, UVD_IDLE_TIMEOUT);
+
+		drm_dev_exit(idx);
+
 	}
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
index 833203401ef4..81ad937936bb 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
@@ -340,22 +340,28 @@ static void amdgpu_vce_idle_work_handler(struct work_struct *work)
 	struct amdgpu_device *adev =
 		container_of(work, struct amdgpu_device, vce.idle_work.work);
 	unsigned i, count = 0;
+	int idx;
 
-	for (i = 0; i < adev->vce.num_rings; i++)
-		count += amdgpu_fence_count_emitted(&adev->vce.ring[i]);
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
-	if (count == 0) {
-		if (adev->pm.dpm_enabled) {
-			amdgpu_dpm_enable_vce(adev, false);
+		for (i = 0; i < adev->vce.num_rings; i++)
+			count += amdgpu_fence_count_emitted(&adev->vce.ring[i]);
+
+		if (count == 0) {
+			if (adev->pm.dpm_enabled) {
+				amdgpu_dpm_enable_vce(adev, false);
+			} else {
+				amdgpu_asic_set_vce_clocks(adev, 0, 0);
+				amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCE,
+								       AMD_PG_STATE_GATE);
+				amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_VCE,
+								       AMD_CG_STATE_GATE);
+			}
 		} else {
-			amdgpu_asic_set_vce_clocks(adev, 0, 0);
-			amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCE,
-							       AMD_PG_STATE_GATE);
-			amdgpu_device_ip_set_clockgating_state(adev, AMD_IP_BLOCK_TYPE_VCE,
-							       AMD_CG_STATE_GATE);
+			schedule_delayed_work(&adev->vce.idle_work, VCE_IDLE_TIMEOUT);
 		}
-	} else {
-		schedule_delayed_work(&adev->vce.idle_work, VCE_IDLE_TIMEOUT);
+
+		drm_dev_exit(idx);
 	}
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
index b42db22761b8..0e7404653ac5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
@@ -342,42 +342,47 @@ static void amdgpu_vcn_idle_work_handler(struct work_struct *work)
 		container_of(work, struct amdgpu_device, vcn.idle_work.work);
 	unsigned int fences = 0, fence[AMDGPU_MAX_VCN_INSTANCES] = {0};
 	unsigned int i, j;
-	int r = 0;
+	int idx, r = 0;
 
-	for (j = 0; j < adev->vcn.num_vcn_inst; ++j) {
-		if (adev->vcn.harvest_config & (1 << j))
-			continue;
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
-		for (i = 0; i < adev->vcn.num_enc_rings; ++i) {
-			fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_enc[i]);
-		}
+		for (j = 0; j < adev->vcn.num_vcn_inst; ++j) {
+			if (adev->vcn.harvest_config & (1 << j))
+				continue;
 
-		if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)	{
-			struct dpg_pause_state new_state;
+			for (i = 0; i < adev->vcn.num_enc_rings; ++i)
+				fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_enc[i]);
 
-			if (fence[j] ||
-				unlikely(atomic_read(&adev->vcn.inst[j].dpg_enc_submission_cnt)))
-				new_state.fw_based = VCN_DPG_STATE__PAUSE;
-			else
-				new_state.fw_based = VCN_DPG_STATE__UNPAUSE;
+			if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)	{
+				struct dpg_pause_state new_state;
+
+				if (fence[j] ||
+					unlikely(atomic_read(&adev->vcn.inst[j].dpg_enc_submission_cnt)))
+					new_state.fw_based = VCN_DPG_STATE__PAUSE;
+				else
+					new_state.fw_based = VCN_DPG_STATE__UNPAUSE;
 
-			adev->vcn.pause_dpg_mode(adev, j, &new_state);
+				adev->vcn.pause_dpg_mode(adev, j, &new_state);
+			}
+
+			fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_dec);
+			fences += fence[j];
 		}
 
-		fence[j] += amdgpu_fence_count_emitted(&adev->vcn.inst[j].ring_dec);
-		fences += fence[j];
-	}
+		if (!fences && !atomic_read(&adev->vcn.total_submission_cnt)) {
+			amdgpu_gfx_off_ctrl(adev, true);
+			amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCN,
+			       AMD_PG_STATE_GATE);
+			r = amdgpu_dpm_switch_power_profile(adev, PP_SMC_POWER_PROFILE_VIDEO,
+					false);
+			if (r)
+				dev_warn(adev->dev, "(%d) failed to disable video power profile mode\n", r);
+		} else {
+			schedule_delayed_work(&adev->vcn.idle_work, VCN_IDLE_TIMEOUT);
+		}
+
+		drm_dev_exit(idx);
 
-	if (!fences && !atomic_read(&adev->vcn.total_submission_cnt)) {
-		amdgpu_gfx_off_ctrl(adev, true);
-		amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCN,
-		       AMD_PG_STATE_GATE);
-		r = amdgpu_dpm_switch_power_profile(adev, PP_SMC_POWER_PROFILE_VIDEO,
-				false);
-		if (r)
-			dev_warn(adev->dev, "(%d) failed to disable video power profile mode\n", r);
-	} else {
-		schedule_delayed_work(&adev->vcn.idle_work, VCN_IDLE_TIMEOUT);
 	}
 }
 
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
index 84d2eaa38101..4799290e5625 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
@@ -57,6 +57,8 @@
 
 #include "ivsrcid/ivsrcid_vislands30.h"
 
+#include <drm/drm_drv.h>
+
 #define GFX8_NUM_GFX_RINGS     1
 #define GFX8_MEC_HPD_SIZE 4096
 
@@ -6793,8 +6795,14 @@ static void gfx_v8_0_sq_irq_work_func(struct work_struct *work)
 
 	struct amdgpu_device *adev = container_of(work, struct amdgpu_device, gfx.sq_work.work);
 	struct sq_work *sq_work = container_of(work, struct sq_work, work);
+	int idx;
+
+	if (drm_dev_enter(&adev->ddev, &idx)) {
 
-	gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data);
+		gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data);
+
+		drm_dev_exit(idx);
+	}
 }
 
 static int gfx_v8_0_sq_irq(struct amdgpu_device *adev,
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 26/27] drm/amd/display: Remove superflous drm_mode_config_cleanup

[Andrey Grodzovsky]

It's already being released by DRM core through devm

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 1 -
 1 file changed, 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index c0b9abb773a4..b9aa15f22cfc 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -3590,7 +3590,6 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev)
 
 static void amdgpu_dm_destroy_drm_device(struct amdgpu_display_manager *dm)
 {
-	drm_mode_config_cleanup(dm->ddev);
 	drm_atomic_private_obj_fini(&dm->atomic_obj);
 	return;
 }
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v5 27/27] drm/amdgpu: Verify DMA opearations from device are done

[Andrey Grodzovsky]

In case device remove is just simualted by sysfs then verify
device doesn't keep doing DMA to the released memory after
pci_remove is done.

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index d0f34f230ef3..f3e8fbde62a0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -1258,7 +1258,13 @@ amdgpu_pci_remove(struct pci_dev *pdev)
 	drm_dev_unplug(dev);
 	amdgpu_driver_unload_kms(dev);
 
+	/*
+	 * Flush any in flight DMA operations from device.
+	 * Clear the Bus Master Enable bit and then wait on the PCIe Device
+	 * StatusTransactions Pending bit.
+	 */
 	pci_disable_device(pdev);
+	pci_wait_for_pending_transaction(pdev);
 }
 
 static void
-- 
2.25.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 19/27] drm/amdgpu: Finilise device fences on device remove.

[Andrey Grodzovsky]

On 2021-04-28 11:11 a.m., Andrey Grodzovsky wrote:
> Make sure all fecens dependent on HW present are force signaled
> when handling device removal. This helpes later to scope all HW
> accesing code such as IOCTLs in drm_dev_enter/exit and use
> drm_dev_unplug as synchronization point past which we know HW
> will not be accessed anymore outside of pci remove driver callback.
> 
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 98 ++++++++++++++++++++--
>   drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  6 ++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c  | 12 +--
>   4 files changed, 103 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 0db0ba4fba89..df6c5ed676b1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1374,6 +1374,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev);
>   bool amdgpu_device_cache_pci_state(struct pci_dev *pdev);
>   bool amdgpu_device_load_pci_state(struct pci_dev *pdev);
>   
> +void amdgpu_finilize_device_fences(struct drm_device *dev);
> +
>   #include "amdgpu_object.h"
>   
>   static inline bool amdgpu_is_tmz(struct amdgpu_device *adev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 33e8e9e1d1fe..55afc11c17e6 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -3692,15 +3692,12 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>   		amdgpu_virt_fini_data_exchange(adev);
>   	}
>   
> -	/* disable all interrupts */
> -	amdgpu_irq_disable_all(adev);
>   	if (adev->mode_info.mode_config_initialized){
>   		if (!amdgpu_device_has_dc_support(adev))
>   			drm_helper_force_disable_all(adev_to_drm(adev));
>   		else
>   			drm_atomic_helper_shutdown(adev_to_drm(adev));
>   	}
> -	amdgpu_fence_driver_fini_hw(adev);
>   
>   	if (adev->pm_sysfs_en)
>   		amdgpu_pm_sysfs_fini(adev);
> @@ -4567,14 +4564,19 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev,
>   	return true;
>   }
>   
> -static void amdgpu_device_unlock_adev(struct amdgpu_device *adev)
> +static void amdgpu_device_unlock_adev_imp(struct amdgpu_device *adev, bool skip_in_gpu_reset)
>   {
>   	amdgpu_vf_error_trans_all(adev);
>   	adev->mp1_state = PP_MP1_STATE_NONE;
> -	atomic_set(&adev->in_gpu_reset, 0);
> +	!skip_in_gpu_reset ? atomic_set(&adev->in_gpu_reset, 0) : 0;
>   	up_write(&adev->reset_sem);
>   }
>   
> +static void amdgpu_device_unlock_adev(struct amdgpu_device *adev)
> +{
> +	amdgpu_device_unlock_adev_imp(adev, false);
> +}
> +
>   /*
>    * to lockup a list of amdgpu devices in a hive safely, if not a hive
>    * with multiple nodes, it will be similar as amdgpu_device_lock_adev.
> @@ -5321,3 +5323,89 @@ bool amdgpu_device_load_pci_state(struct pci_dev *pdev)
>   }
>   
>   
> +static void amdgpu_finilize_schedulded_fences(struct amdgpu_ctx_mgr *mgr)
> +{
> +	struct amdgpu_ctx *ctx;
> +	struct idr *idp;
> +	uint32_t id, i, j;
> +
> +	idp = &mgr->ctx_handles;
> +
> +	idr_for_each_entry(idp, ctx, id) {
> +		for (i = 0; i < AMDGPU_HW_IP_NUM; ++i) {
> +			for (j = 0; j < amdgpu_ctx_num_entities[i]; ++j) {
> +				struct drm_sched_entity *entity;
> +
> +				if (!ctx->entities[i][j])
> +					continue;
> +
> +				entity = &ctx->entities[i][j]->entity;
> +				drm_sched_entity_kill_jobs(entity);
> +			}
> +		}
> +	}
> +}
> +
> +/**
> + * amdgpu_finilize_device_fences() - Finilize all device fences
> + * @pdev: pointer to PCI device
> + *
> + * Will disable and finilise ISRs and will signal all fences
> + * that might hang if HW is gone
> + */
> +void amdgpu_finilize_device_fences(struct drm_device *dev)
> +{
> +	struct amdgpu_device *adev = drm_to_adev(dev);
> +	struct drm_file *file;
> +
> +	/*
> +	 *  Block TDRs from further execution by setting adev->in_gpu_reset
> +	 *  instead of holding full reset lock in order to not deadlock
> +	 *  further ahead against any thread locking the reset lock when we
> +	 *  wait for it's completion
> +	 */
> +	while (!amdgpu_device_lock_adev(adev, NULL))
> +		amdgpu_cancel_all_tdr(adev);
> +	amdgpu_device_unlock_adev_imp(adev, true);
> +
> +
> +	/* disable all HW interrupts */
> +	amdgpu_irq_disable_all(adev);
> +
> +	/* stop and flush all in flight HW interrupts handlers */
> +	disable_irq(pci_irq_vector(adev->pdev, 0));
> +
> +	/*
> +	 * Stop SW GPU schedulers and force completion on all HW fences. Since
> +	 * in the prev. step all ISRs were disabled and completed the
> +	 * HW fence array is idle (no insertions or extractions) and so it's
> +	 * safe to iterate it bellow.
> +	 * After this step all HW fences in the system are signaled. As a result
> +	 * also all the scheduler 'finished' fences are also signaled.
> +	 */
> +	amdgpu_fence_driver_fini_hw(adev);
> +
> +	/*
> +	 * Reject any further jobs to any scheduler entity queue. After this
> +	 * step no new insertions and because schedulers are stopped also no
> +	 * new extractions.
> +	 */
> +	down_read(&adev->sched_fence_completion_sem);
> +	adev->stop_job_submissions = true;
> +	up_read(&adev->sched_fence_completion_sem);

FYI: Typo here - down_write and up_write obviously.

Andrey

> +
> +	/*
> +	 * Complete all scheduler 'scheduled' fences currently pending.
> +	 * It's OK if new contexts and sched entities are concurrently
> +	 * still created as they will fail in pushing jobs to SW queues
> +	 * and their schedule fences will be signaled with error
> +	 */
> +	mutex_lock(&adev->ddev.filelist_mutex);
> +	list_for_each_entry(file, &adev->ddev.filelist, lhead) {
> +		struct amdgpu_fpriv *fpriv = file->driver_priv;
> +		amdgpu_finilize_schedulded_fences(&fpriv->ctx_mgr);
> +	}
> +	mutex_unlock(&adev->ddev.filelist_mutex);
> +}
> +
> +
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> index f799c40d7e72..8a19b8dd02ee 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> @@ -1249,6 +1249,12 @@ amdgpu_pci_remove(struct pci_dev *pdev)
>   {
>   	struct drm_device *dev = pci_get_drvdata(pdev);
>   
> +	/*
> +	 * Force completion of all device related fences that might hang us when
> +	 * synchronizing SRCU in the following step.
> +	 */
> +	amdgpu_finilize_device_fences(dev);
> +
>   	drm_dev_unplug(dev);
>   	amdgpu_driver_unload_kms(dev);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> index 2670201e78d3..af592b28cd35 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> @@ -526,7 +526,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
>    */
>   void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
>   {
> -	int i, r;
> +	int i;
>   
>   	for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
>   		struct amdgpu_ring *ring = adev->rings[i];
> @@ -535,18 +535,10 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
>   			continue;
>   
>   		/* Stop any new job submissions from sched before flushing the ring */
> -		/* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */
>   		if (!ring->no_scheduler)
>   			drm_sched_fini(&ring->sched);
>   
> -		/* You can't wait for HW to signal if it's gone */
> -		if (!drm_dev_is_unplugged(&adev->ddev))
> -			r = amdgpu_fence_wait_empty(ring);
> -		else
> -			r = -ENODEV;
> -		/* no need to trigger GPU reset as we are unloading */
> -		if (r)
> -			amdgpu_fence_driver_force_completion(ring);
> +		amdgpu_fence_driver_force_completion(ring);
>   
>   		if (ring->fence_drv.irq_src)
>   			amdgpu_irq_put(adev, ring->fence_drv.irq_src,
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 08/27] PCI: add support for dev_groups to struct pci_device_driver

[Bjorn Helgaas]

In subject:

s/PCI: add support/PCI: Add support/

to match convention ("git log --oneline drivers/pci/pci-driver.c" to
learn this).

On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
> This is exact copy of 'USB: add support for dev_groups to
> struct usb_device_driver' patch by Greg but just for
> the PCI case.

Ideally this would be an imperative sentence telling us what the patch
*does*, not just that it's a copy of something else.

I'd also like a brief comment about why this is useful, i.e., why you
need this and what you're going to use it for.

The usual commit citation format is 7d9c1d2f7aca ("USB: add support
for dev_groups to struct usb_device_driver") so it's easier to locate
the commit.

I see there is also b71b283e3d6d ("USB: add support for dev_groups to
struct usb_driver").  I don't know enough about USB to know whether
7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing
here, but I do see that struct usb_driver is far more common than
struct usb_device_driver.

PCI has struct pci_driver, but doesn't have the concept of a struct
pci_device_driver.

> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> ---
>  drivers/pci/pci-driver.c | 1 +
>  include/linux/pci.h      | 3 +++
>  2 files changed, 4 insertions(+)
> 
> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
> index ec44a79e951a..3a72352aa5cf 100644
> --- a/drivers/pci/pci-driver.c
> +++ b/drivers/pci/pci-driver.c
> @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner,
>  	drv->driver.owner = owner;
>  	drv->driver.mod_name = mod_name;
>  	drv->driver.groups = drv->groups;
> +	drv->driver.dev_groups = drv->dev_groups;
>  
>  	spin_lock_init(&drv->dynids.lock);
>  	INIT_LIST_HEAD(&drv->dynids.list);
> diff --git a/include/linux/pci.h b/include/linux/pci.h
> index 86c799c97b77..b57755b03009 100644
> --- a/include/linux/pci.h
> +++ b/include/linux/pci.h
> @@ -858,6 +858,8 @@ struct module;
>   *		number of VFs to enable via sysfs "sriov_numvfs" file.
>   * @err_handler: See Documentation/PCI/pci-error-recovery.rst
>   * @groups:	Sysfs attribute groups.
> + * @dev_groups: Attributes attached to the device that will be
> + *              created once it is bound to the driver.
>   * @driver:	Driver model structure.
>   * @dynids:	List of dynamically added device IDs.
>   */
> @@ -873,6 +875,7 @@ struct pci_driver {
>  	int  (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */
>  	const struct pci_error_handlers *err_handler;
>  	const struct attribute_group **groups;
> +	const struct attribute_group **dev_groups;
>  	struct device_driver	driver;
>  	struct pci_dynids	dynids;
>  };
> -- 
> 2.25.1
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 00/27] RFC Support hot device unplug in amdgpu

[Bjorn Helgaas]

On Wed, Apr 28, 2021 at 11:11:40AM -0400, Andrey Grodzovsky wrote:
> Until now extracting a card either by physical extraction (e.g. eGPU with 
> thunderbolt connection or by emulation through  syfs -> /sys/bus/pci/devices/device_id/remove) 
> would cause random crashes in user apps. The random crashes in apps were 
> mostly due to the app having mapped a device backed BO into its address 
> space was still trying to access the BO while the backing device was gone.
> To answer this first problem Christian suggested to fix the handling of mapped 
> memory in the clients when the device goes away by forcibly unmap all buffers the 
> user processes has by clearing their respective VMAs mapping the device BOs. 
> Then when the VMAs try to fill in the page tables again we check in the fault 
> handlerif the device is removed and if so, return an error. This will generate a 
> SIGBUS to the application which can then cleanly terminate.This indeed was done 
> but this in turn created a problem of kernel OOPs were the OOPSes were due to the 
> fact that while the app was terminating because of the SIGBUSit would trigger use 
> after free in the driver by calling to accesses device structures that were already 
> released from the pci remove sequence.This was handled by introducing a 'flush' 
> sequence during device removal were we wait for drm file reference to drop to 0 
> meaning all user clients directly using this device terminated.

If DRM includes cover letters in merges, maybe fix the below.  If they
also include the v2, v3, etc below, also consider picking a line
width and sticking to it.  It seems to be creeping wider every rev.

BO?
s/syfs/sysfs/
s/forcibly unmap/forcibly unmapping/
s/handlerif/handler if/
s/processes has/processes have/
s/terminate.This/terminate. This/
s/were the/where the/
s/SIGBUSit/SIGBUS it/
s/to accesses/to access/
s/sequence.This/sequence. This/
s/were we/where we/

> v2:
> Based on discussions in the mailing list with Daniel and Pekka [1] and based on the document 
> produced by Pekka from those discussions [2] the whole approach with returning SIGBUS and 
> waiting for all user clients having CPU mapping of device BOs to die was dropped. 
> Instead as per the document suggestion the device structures are kept alive until 
> the last reference to the device is dropped by user client and in the meanwhile all existing and new CPU mappings of the BOs 
> belonging to the device directly or by dma-buf import are rerouted to per user 
> process dummy rw page.Also, I skipped the 'Requirements for KMS UAPI' section of [2] 
> since i am trying to get the minimal set of requirements that still give useful solution 
> to work and this is the'Requirements for Render and Cross-Device UAPI' section and so my 
> test case is removing a secondary device, which is render only and is not involved 
> in KMS.
> 
> v3:
> More updates following comments from v2 such as removing loop to find DRM file when rerouting 
> page faults to dummy page,getting rid of unnecessary sysfs handling refactoring and moving 
> prevention of GPU recovery post device unplug from amdgpu to scheduler layer. 
> On top of that added unplug support for the IOMMU enabled system.
> 
> v4:
> Drop last sysfs hack and use sysfs default attribute.
> Guard against write accesses after device removal to avoid modifying released memory.
> Update dummy pages handling to on demand allocation and release through drm managed framework.
> Add return value to scheduler job TO handler (by Luben Tuikov) and use this in amdgpu for prevention 
> of GPU recovery post device unplug
> Also rebase on top of drm-misc-mext instead of amd-staging-drm-next
> 
> v5:
> The most significant in this series is the improved protection from kernel driver accessing MMIO ranges that were allocated
> for the device once the device is gone. To do this, first a patch 'drm/amdgpu: Unmap all MMIO mappings' is introduced.
> This patch unamps all MMIO mapped into the kernel address space in the form of BARs and kernel BOs with CPU visible VRAM mappings.
> This way it helped to discover multiple such access points because a page fault would be immediately generated on access. Most of them
> were solved by moving HW fini code into pci_remove stage (patch drm/amdgpu: Add early fini callback) and for some who 
> were harder to unwind drm_dev_enter/exit scoping was used. In addition all the IOCTLs and all background work and timers 
> are now protected with drm_dev_enter/exit at their root in an attempt that after drm_dev_unplug is finished none of them 
> run anymore and the pci_remove thread is the only thread executing which might touch the HW. To prevent deadlocks in such 
> case against threads stuck on various HW or SW fences patches 'drm/amdgpu: Finalise device fences on device remove'  
> and drm/amdgpu: Add rw_sem to pushing job into sched queue' take care of force signaling all such existing fences 
> and rejecting any newly added ones.
> 
> With these patches I am able to gracefully remove the secondary card using sysfs remove hook while glxgears is running off of secondary 
> card (DRI_PRIME=1) without kernel oopses or hangs and keep working with the primary card or soft reset the device without hangs or oopses.
> Also as per Daniel's comment I added 3 tests to IGT [4] to core_hotunplug test suite - remove device while commands are submitted, 
> exported BO and exported fence (not pushed yet).
> Also now it's possible to plug back the device after unplug 
> Also some users now can successfully use those patches with eGPU boxes[3].
> 
> 
> 
> 
> TODOs for followup work:
> Convert AMDGPU code to use devm (for hw stuff) and drmm (for sw stuff and allocations) (Daniel)
> Add support for 'Requirements for KMS UAPI' section of [2] - unplugging primary, display connected card.
> 
> [1] - Discussions during v4 of the patchset https://lists.freedesktop.org/archives/amd-gfx/2021-January/058595.html
> [2] - drm/doc: device hot-unplug for userspace https://www.spinics.net/lists/dri-devel/msg259755.html
> [3] - Related gitlab ticket https://gitlab.freedesktop.org/drm/amd/-/issues/1081
> [4] - https://gitlab.freedesktop.org/agrodzov/igt-gpu-tools/-/commits/master
> 
> Andrey Grodzovsky (27):
>   drm/ttm: Remap all page faults to per process dummy page.
>   drm/ttm: Expose ttm_tt_unpopulate for driver use
>   drm/amdgpu: Split amdgpu_device_fini into early and late
>   drm/amdkfd: Split kfd suspend from devie exit
>   drm/amdgpu: Add early fini callback
>   drm/amdgpu: Handle IOMMU enabled case.
>   drm/amdgpu: Remap all page faults to per process dummy page.
>   PCI: add support for dev_groups to struct pci_device_driver
>   dmr/amdgpu: Move some sysfs attrs creation to default_attr
>   drm/amdgpu: Guard against write accesses after device removal
>   drm/sched: Make timeout timer rearm conditional.
>   drm/amdgpu: Prevent any job recoveries after device is unplugged.
>   drm/amdgpu: When filizing the fence driver. stop scheduler first.
>   drm/amdgpu: Fix hang on device removal.
>   drm/scheduler: Fix hang when sched_entity released
>   drm/amdgpu: Unmap all MMIO mappings
>   drm/amdgpu: Add rw_sem to pushing job into sched queue
>   drm/sched: Expose drm_sched_entity_kill_jobs
>   drm/amdgpu: Finilise device fences on device remove.
>   drm: Scope all DRM IOCTLs  with drm_dev_enter/exit
>   drm/amdgpu: Add support for hot-unplug feature at DRM level.
>   drm/amd/display: Scope all DM queued work with drm_dev_enter/exit
>   drm/amd/powerplay: Scope all PM queued work with drm_dev_enter/exit
>   drm/amdkfd: Scope all KFD queued work with drm_dev_enter/exit
>   drm/amdgpu: Scope all amdgpu queued work with drm_dev_enter/exit
>   drm/amd/display: Remove superflous drm_mode_config_cleanup
>   drm/amdgpu: Verify DMA opearations from device are done
> 
>  drivers/gpu/drm/amd/amdgpu/amdgpu.h           |  18 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c    |  13 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h    |   2 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c  |  17 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c        |  13 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    | 353 ++++++++++++++----
>  drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c       |  34 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c     |  34 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c      |   3 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h      |   1 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c       |   9 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c   |  25 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c        | 228 +++++------
>  drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c       |  61 ++-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h       |   3 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c       |  33 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.c      |  28 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c       |  12 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |  41 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_object.h    |   7 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       | 115 +++---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h       |   3 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c       |  56 ++-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c      |  70 ++++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h      |  52 +--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c       |  21 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c       |  74 ++--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c       |  45 ++-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c       |  83 ++--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c        |   7 +-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c  |  14 +-
>  drivers/gpu/drm/amd/amdgpu/cik_ih.c           |   3 +-
>  drivers/gpu/drm/amd/amdgpu/cz_ih.c            |   3 +-
>  drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c         |  10 +-
>  drivers/gpu/drm/amd/amdgpu/iceland_ih.c       |   3 +-
>  drivers/gpu/drm/amd/amdgpu/navi10_ih.c        |   5 +-
>  drivers/gpu/drm/amd/amdgpu/psp_v11_0.c        |  44 +--
>  drivers/gpu/drm/amd/amdgpu/psp_v12_0.c        |   8 +-
>  drivers/gpu/drm/amd/amdgpu/psp_v3_1.c         |   8 +-
>  drivers/gpu/drm/amd/amdgpu/si_ih.c            |   3 +-
>  drivers/gpu/drm/amd/amdgpu/tonga_ih.c         |   3 +-
>  drivers/gpu/drm/amd/amdgpu/vce_v4_0.c         |  26 +-
>  drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c         |  22 +-
>  drivers/gpu/drm/amd/amdgpu/vega10_ih.c        |   5 +-
>  drivers/gpu/drm/amd/amdgpu/vega20_ih.c        |   2 +-
>  drivers/gpu/drm/amd/amdkfd/kfd_device.c       |   3 +-
>  drivers/gpu/drm/amd/amdkfd/kfd_interrupt.c    |  14 +-
>  .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c |  13 +-
>  .../amd/display/amdgpu_dm/amdgpu_dm_hdcp.c    | 124 +++---
>  .../drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c |  24 +-
>  drivers/gpu/drm/amd/include/amd_shared.h      |   2 +
>  drivers/gpu/drm/amd/pm/amdgpu_dpm.c           |  44 ++-
>  .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c |   2 +
>  drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c     |  26 +-
>  drivers/gpu/drm/drm_ioctl.c                   |  15 +-
>  drivers/gpu/drm/scheduler/sched_entity.c      |   6 +-
>  drivers/gpu/drm/scheduler/sched_main.c        |  35 +-
>  drivers/gpu/drm/ttm/ttm_bo_vm.c               |  79 +++-
>  drivers/gpu/drm/ttm/ttm_tt.c                  |   1 +
>  drivers/pci/pci-driver.c                      |   1 +
>  include/drm/drm_drv.h                         |   6 +
>  include/drm/gpu_scheduler.h                   |   1 +
>  include/drm/ttm/ttm_bo_api.h                  |   2 +
>  include/linux/pci.h                           |   3 +
>  64 files changed, 1388 insertions(+), 633 deletions(-)
> 
> -- 
> 2.25.1
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 09/27] dmr/amdgpu: Move some sysfs attrs creation to default_attr

[Bjorn Helgaas]

In subject,

s/dmr/drm/
s/Move some/Move/ ("some" consumes space without adding meaning)

Or maybe something like: 

  drm/amdgpu: Convert driver sysfs attributes to static attributes

On Wed, Apr 28, 2021 at 11:11:49AM -0400, Andrey Grodzovsky wrote:
> This allows to remove explicit creation and destruction
> of those attrs and by this avoids warnings on device
> finilizing post physical device extraction.

s/finilizing/finalizing/

> v5: Use newly added pci_driver.dev_groups directly

I don't know the DRM convention, but IMO, change notes like "v5: Use
..." can go after "---" so they don't go in the git log.  To me,
they're useful during review, but not after being merged.

I love the patch!  Much cleaner than creating/removing all these
attributes explicitly.

> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++-------
>  drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c      | 13 ++++++++++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c  | 25 ++++++++------------
>  drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++-------
>  4 files changed, 37 insertions(+), 32 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> index 86add0f4ea4d..0346e124ab8c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> @@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev,
>  static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version,
>  		   NULL);
>  
> +static struct attribute *amdgpu_vbios_version_attrs[] = {
> +	&dev_attr_vbios_version.attr,
> +	NULL
> +};
> +
> +const struct attribute_group amdgpu_vbios_version_attr_group = {
> +	.attrs = amdgpu_vbios_version_attrs
> +};
> +
>  /**
>   * amdgpu_atombios_fini - free the driver info and callbacks for atombios
>   *
> @@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev)
>  	adev->mode_info.atom_context = NULL;
>  	kfree(adev->mode_info.atom_card_info);
>  	adev->mode_info.atom_card_info = NULL;
> -	device_remove_file(adev->dev, &dev_attr_vbios_version);
>  }
>  
>  /**
> @@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
>  {
>  	struct card_info *atom_card_info =
>  	    kzalloc(sizeof(struct card_info), GFP_KERNEL);
> -	int ret;
>  
>  	if (!atom_card_info)
>  		return -ENOMEM;
> @@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
>  		amdgpu_atombios_allocate_fb_scratch(adev);
>  	}
>  
> -	ret = device_create_file(adev->dev, &dev_attr_vbios_version);
> -	if (ret) {
> -		DRM_ERROR("Failed to create device file for VBIOS version\n");
> -		return ret;
> -	}
> -
>  	return 0;
>  }
>  
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> index 54cb5ee2f563..f799c40d7e72 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> @@ -1605,6 +1605,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = {
>  	.resume		= amdgpu_pci_resume,
>  };
>  
> +extern const struct attribute_group amdgpu_vram_mgr_attr_group;
> +extern const struct attribute_group amdgpu_gtt_mgr_attr_group;
> +extern const struct attribute_group amdgpu_vbios_version_attr_group;
> +
> +static const struct attribute_group *amdgpu_sysfs_groups[] = {
> +	&amdgpu_vram_mgr_attr_group,
> +	&amdgpu_gtt_mgr_attr_group,
> +	&amdgpu_vbios_version_attr_group,
> +	NULL,
> +};
> +
> +
>  static struct pci_driver amdgpu_kms_pci_driver = {
>  	.name = DRIVER_NAME,
>  	.id_table = pciidlist,
> @@ -1613,6 +1625,7 @@ static struct pci_driver amdgpu_kms_pci_driver = {
>  	.shutdown = amdgpu_pci_shutdown,
>  	.driver.pm = &amdgpu_pm_ops,
>  	.err_handler = &amdgpu_pci_err_handler,
> +	.dev_groups = amdgpu_sysfs_groups,
>  };
>  
>  static int __init amdgpu_init(void)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
> index 8980329cded0..3b7150e1c5ed 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
> @@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO,
>  static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO,
>  	           amdgpu_mem_info_gtt_used_show, NULL);
>  
> +static struct attribute *amdgpu_gtt_mgr_attributes[] = {
> +	&dev_attr_mem_info_gtt_total.attr,
> +	&dev_attr_mem_info_gtt_used.attr,
> +	NULL
> +};
> +
> +const struct attribute_group amdgpu_gtt_mgr_attr_group = {
> +	.attrs = amdgpu_gtt_mgr_attributes
> +};
> +
>  static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func;
>  /**
>   * amdgpu_gtt_mgr_init - init GTT manager and DRM MM
> @@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
>  	struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr;
>  	struct ttm_resource_manager *man = &mgr->manager;
>  	uint64_t start, size;
> -	int ret;
>  
>  	man->use_tt = true;
>  	man->func = &amdgpu_gtt_mgr_func;
> @@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
>  	spin_lock_init(&mgr->lock);
>  	atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT);
>  
> -	ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total);
> -	if (ret) {
> -		DRM_ERROR("Failed to create device file mem_info_gtt_total\n");
> -		return ret;
> -	}
> -	ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used);
> -	if (ret) {
> -		DRM_ERROR("Failed to create device file mem_info_gtt_used\n");
> -		return ret;
> -	}
> -
>  	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager);
>  	ttm_resource_manager_set_used(man, true);
>  	return 0;
> @@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev)
>  	drm_mm_takedown(&mgr->mm);
>  	spin_unlock(&mgr->lock);
>  
> -	device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total);
> -	device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used);
> -
>  	ttm_resource_manager_cleanup(man);
>  	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL);
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> index c89b66bb70e2..68369b38aebb 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> @@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO,
>  static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO,
>  		   amdgpu_mem_info_vram_vendor, NULL);
>  
> -static const struct attribute *amdgpu_vram_mgr_attributes[] = {
> +static struct attribute *amdgpu_vram_mgr_attributes[] = {
>  	&dev_attr_mem_info_vram_total.attr,
>  	&dev_attr_mem_info_vis_vram_total.attr,
>  	&dev_attr_mem_info_vram_used.attr,
> @@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = {
>  	NULL
>  };
>  
> +const struct attribute_group amdgpu_vram_mgr_attr_group = {
> +	.attrs = amdgpu_vram_mgr_attributes
> +};
> +
>  static const struct ttm_resource_manager_func amdgpu_vram_mgr_func;
>  
>  /**
> @@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
>  {
>  	struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr;
>  	struct ttm_resource_manager *man = &mgr->manager;
> -	int ret;
>  
>  	ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT);
>  
> @@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
>  	INIT_LIST_HEAD(&mgr->reservations_pending);
>  	INIT_LIST_HEAD(&mgr->reserved_pages);
>  
> -	/* Add the two VRAM-related sysfs files */
> -	ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
> -	if (ret)
> -		DRM_ERROR("Failed to register sysfs\n");
> -
>  	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager);
>  	ttm_resource_manager_set_used(man, true);
>  	return 0;
> @@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
>  	drm_mm_takedown(&mgr->mm);
>  	spin_unlock(&mgr->lock);
>  
> -	sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
> -
>  	ttm_resource_manager_cleanup(man);
>  	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL);
>  }
> -- 
> 2.25.1
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late

[Christian König]

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> Some of the stuff in amdgpu_device_fini such as HW interrupts
> disable and pending fences finilization must be done right away on
> pci_remove while most of the stuff which relates to finilizing and
> releasing driver data structures can be kept until
> drm_driver.release hook is called, i.e. when the last device
> reference is dropped.
>
> v4: Change functions prefix early->hw and late->sw
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>

Acked-by: Christian König <christian.koenig@amd.com>

But Alex should acknowledge this as well since it is general driver design.

Christian.

> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 ++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  7 ++----
>   drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c  | 15 ++++++++++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    | 26 +++++++++++++---------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h    |  3 ++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 12 +++++++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c    |  1 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h   |  3 ++-
>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  2 +-
>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  2 +-
>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  2 +-
>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  2 +-
>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  2 +-
>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  2 +-
>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  2 +-
>   drivers/gpu/drm/amd/amdgpu/vega20_ih.c     |  2 +-
>   17 files changed, 79 insertions(+), 36 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 1af2fa1591fd..fddb82897e5d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
>   
>   int amdgpu_device_init(struct amdgpu_device *adev,
>   		       uint32_t flags);
> -void amdgpu_device_fini(struct amdgpu_device *adev);
> +void amdgpu_device_fini_hw(struct amdgpu_device *adev);
> +void amdgpu_device_fini_sw(struct amdgpu_device *adev);
> +
>   int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
>   
>   void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
> @@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev);
>   int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
>   void amdgpu_driver_postclose_kms(struct drm_device *dev,
>   				 struct drm_file *file_priv);
> +void amdgpu_driver_release_kms(struct drm_device *dev);
> +
>   int amdgpu_device_ip_suspend(struct amdgpu_device *adev);
>   int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);
>   int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 6447cd6ca5a8..8d22b79fc1cd 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>    * Tear down the driver info (all asics).
>    * Called at driver shutdown.
>    */
> -void amdgpu_device_fini(struct amdgpu_device *adev)
> +void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>   {
>   	dev_info(adev->dev, "amdgpu: finishing device.\n");
>   	flush_delayed_work(&adev->delayed_init_work);
>   	adev->shutdown = true;
>   
> -	kfree(adev->pci_state);
> -
>   	/* make sure IB test finished before entering exclusive mode
>   	 * to avoid preemption on IB test
>   	 * */
> @@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
>   		else
>   			drm_atomic_helper_shutdown(adev_to_drm(adev));
>   	}
> -	amdgpu_fence_driver_fini(adev);
> +	amdgpu_fence_driver_fini_hw(adev);
> +
>   	if (adev->pm_sysfs_en)
>   		amdgpu_pm_sysfs_fini(adev);
> +	if (adev->ucode_sysfs_en)
> +		amdgpu_ucode_sysfs_fini(adev);
> +	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
> +
> +
>   	amdgpu_fbdev_fini(adev);
> +
> +	amdgpu_irq_fini_hw(adev);
> +}
> +
> +void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> +{
>   	amdgpu_device_ip_fini(adev);
> +	amdgpu_fence_driver_fini_sw(adev);
>   	release_firmware(adev->firmware.gpu_info_fw);
>   	adev->firmware.gpu_info_fw = NULL;
>   	adev->accel_working = false;
> @@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
>   	adev->rmmio = NULL;
>   	amdgpu_device_doorbell_fini(adev);
>   
> -	if (adev->ucode_sysfs_en)
> -		amdgpu_ucode_sysfs_fini(adev);
> -
> -	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>   	if (IS_ENABLED(CONFIG_PERF_EVENTS))
>   		amdgpu_pmu_fini(adev);
>   	if (adev->mman.discovery_bin)
>   		amdgpu_discovery_fini(adev);
> +
> +	kfree(adev->pci_state);
> +
>   }
>   
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> index 671ec1002230..54cb5ee2f563 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev)
>   {
>   	struct drm_device *dev = pci_get_drvdata(pdev);
>   
> -#ifdef MODULE
> -	if (THIS_MODULE->state != MODULE_STATE_GOING)
> -#endif
> -		DRM_ERROR("Hotplug removal is not supported\n");
>   	drm_dev_unplug(dev);
>   	amdgpu_driver_unload_kms(dev);
> +
>   	pci_disable_device(pdev);
> -	pci_set_drvdata(pdev, NULL);
>   }
>   
>   static void
> @@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = {
>   	.dumb_create = amdgpu_mode_dumb_create,
>   	.dumb_map_offset = amdgpu_mode_dumb_mmap,
>   	.fops = &amdgpu_driver_kms_fops,
> +	.release = &amdgpu_driver_release_kms,
>   
>   	.prime_handle_to_fd = drm_gem_prime_handle_to_fd,
>   	.prime_fd_to_handle = drm_gem_prime_fd_to_handle,
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> index 8e0a5650d383..34d51e962799 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
>    *
>    * Tear down the fence driver for all possible rings (all asics).
>    */
> -void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
> +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
>   {
>   	unsigned i, j;
>   	int r;
> @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
>   		if (!ring->no_scheduler)
>   			drm_sched_fini(&ring->sched);
>   		del_timer_sync(&ring->fence_drv.fallback_timer);
> +	}
> +}
> +
> +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev)
> +{
> +	unsigned int i, j;
> +
> +	for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
> +		struct amdgpu_ring *ring = adev->rings[i];
> +
> +		if (!ring || !ring->fence_drv.initialized)
> +			continue;
> +
>   		for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j)
>   			dma_fence_put(ring->fence_drv.fences[j]);
>   		kfree(ring->fence_drv.fences);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> index afbbec82a289..63e815c27585 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> @@ -49,6 +49,7 @@
>   #include <drm/drm_irq.h>
>   #include <drm/drm_vblank.h>
>   #include <drm/amdgpu_drm.h>
> +#include <drm/drm_drv.h>
>   #include "amdgpu.h"
>   #include "amdgpu_ih.h"
>   #include "atom.h"
> @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
>   	return 0;
>   }
>   
> +
> +void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
> +{
> +	if (adev->irq.installed) {
> +		drm_irq_uninstall(&adev->ddev);
> +		adev->irq.installed = false;
> +		if (adev->irq.msi_enabled)
> +			pci_free_irq_vectors(adev->pdev);
> +
> +		if (!amdgpu_device_has_dc_support(adev))
> +			flush_work(&adev->hotplug_work);
> +	}
> +}
> +
>   /**
>    * amdgpu_irq_fini - shut down interrupt handling
>    *
> @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
>    * functionality, shuts down vblank, hotplug and reset interrupt handling,
>    * turns off interrupts from all sources (all ASICs).
>    */
> -void amdgpu_irq_fini(struct amdgpu_device *adev)
> +void amdgpu_irq_fini_sw(struct amdgpu_device *adev)
>   {
>   	unsigned i, j;
>   
> -	if (adev->irq.installed) {
> -		drm_irq_uninstall(adev_to_drm(adev));
> -		adev->irq.installed = false;
> -		if (adev->irq.msi_enabled)
> -			pci_free_irq_vectors(adev->pdev);
> -		if (!amdgpu_device_has_dc_support(adev))
> -			flush_work(&adev->hotplug_work);
> -	}
> -
>   	for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) {
>   		if (!adev->irq.client[i].sources)
>   			continue;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> index ac527e5deae6..392a7324e2b1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev);
>   irqreturn_t amdgpu_irq_handler(int irq, void *arg);
>   
>   int amdgpu_irq_init(struct amdgpu_device *adev);
> -void amdgpu_irq_fini(struct amdgpu_device *adev);
> +void amdgpu_irq_fini_sw(struct amdgpu_device *adev);
> +void amdgpu_irq_fini_hw(struct amdgpu_device *adev);
>   int amdgpu_irq_add_id(struct amdgpu_device *adev,
>   		      unsigned client_id, unsigned src_id,
>   		      struct amdgpu_irq_src *source);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> index 64beb3399604..1af3fba7bfd4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> @@ -29,6 +29,7 @@
>   #include "amdgpu.h"
>   #include <drm/drm_debugfs.h>
>   #include <drm/amdgpu_drm.h>
> +#include <drm/drm_drv.h>
>   #include "amdgpu_uvd.h"
>   #include "amdgpu_vce.h"
>   #include "atom.h"
> @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev)
>   	}
>   
>   	amdgpu_acpi_fini(adev);
> -	amdgpu_device_fini(adev);
> +	amdgpu_device_fini_hw(adev);
>   }
>   
>   void amdgpu_register_gpu_instance(struct amdgpu_device *adev)
> @@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
>   	pm_runtime_put_autosuspend(dev->dev);
>   }
>   
> +
> +void amdgpu_driver_release_kms(struct drm_device *dev)
> +{
> +	struct amdgpu_device *adev = drm_to_adev(dev);
> +
> +	amdgpu_device_fini_sw(adev);
> +	pci_set_drvdata(adev->pdev, NULL);
> +}
> +
>   /*
>    * VBlank related functions.
>    */
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> index 1fb2a91ad30a..c0a16eac4923 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev)
>   	if (!con)
>   		return 0;
>   
> +
>   	/* Need disable ras on all IPs here before ip [hw/sw]fini */
>   	amdgpu_ras_disable_all_features(adev, 0);
>   	amdgpu_ras_recovery_fini(adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> index 56acec1075ac..0f195f7bf797 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> @@ -107,7 +107,8 @@ struct amdgpu_fence_driver {
>   };
>   
>   int amdgpu_fence_driver_init(struct amdgpu_device *adev);
> -void amdgpu_fence_driver_fini(struct amdgpu_device *adev);
> +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev);
> +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev);
>   void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
>   
>   int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> index d3745711d55f..183d44a6583c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> index 307c01301c87..d32743949003 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> index cc957471f31e..da96c6013477 100644
> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> index f4e4040bbd25..5eea4550b856 100644
> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> index 51880f6ef634..751307f3252c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   
>   	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> index 249fcbee7871..973d80ec7f6c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> index 88626d83e07b..2d0094c276ca 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> index 5a3c867d5881..9059b21b079f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
> -	amdgpu_irq_fini(adev);
> +	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Christian König]

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> Handle all DMA IOMMU gropup related dependencies before the
> group is removed.
>
> v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate

Maybe split that up into more patches.

>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++--
>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>   14 files changed, 56 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index fddb82897e5d..30a24db5f4d1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>   
>   	bool                            in_pci_err_recovery;
>   	struct pci_saved_state          *pci_state;
> +
> +	struct list_head                device_bo_list;
>   };
>   
>   static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 46d646c40338..91594ddc2459 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -70,6 +70,7 @@
>   #include <drm/task_barrier.h>
>   #include <linux/pm_runtime.h>
>   
> +
>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
> @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = {
>   	NULL
>   };
>   
> -
>   /**
>    * amdgpu_device_init - initialize the driver
>    *
> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>   
>   	INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>   
> +	INIT_LIST_HEAD(&adev->device_bo_list);
> +
>   	adev->gfx.gfx_off_req_count = 1;
>   	adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>   
> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>   	return r;
>   }
>   
> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
> +{
> +	struct amdgpu_bo *bo = NULL;
> +
> +	/*
> +	 * Unmaps all DMA mappings before device will be removed from it's
> +	 * IOMMU group otherwise in case of IOMMU enabled system a crash
> +	 * will happen.
> +	 */
> +
> +	spin_lock(&adev->mman.bdev.lru_lock);
> +	while (!list_empty(&adev->device_bo_list)) {
> +		bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);
> +		list_del_init(&bo->bo);
> +		spin_unlock(&adev->mman.bdev.lru_lock);
> +		if (bo->tbo.ttm)
> +			ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
> +		spin_lock(&adev->mman.bdev.lru_lock);
> +	}
> +	spin_unlock(&adev->mman.bdev.lru_lock);

Can you try to use the same approach as amdgpu_gtt_mgr_recover() instead 
of adding something to the BO?

Christian.

> +}
> +
>   /**
>    * amdgpu_device_fini - tear down the driver
>    *
> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>   		amdgpu_ucode_sysfs_fini(adev);
>   	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>   
> -
>   	amdgpu_fbdev_fini(adev);
>   
>   	amdgpu_irq_fini_hw(adev);
>   
>   	amdgpu_device_ip_fini_early(adev);
> +
> +	amdgpu_clear_dma_mappings(adev);
> +
> +	amdgpu_gart_dummy_page_fini(adev);
>   }
>   
>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> index fde2d899b2c4..49cdcaf8512d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
>    *
>    * Frees the dummy page used by the driver (all asics).
>    */
> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>   {
>   	if (!adev->dummy_page_addr)
>   		return;
> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>   	vfree(adev->gart.pages);
>   	adev->gart.pages = NULL;
>   #endif
> -	amdgpu_gart_dummy_page_fini(adev);
>   }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> index afa2e2877d87..5678d9c105ab 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev);
>   void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>   int amdgpu_gart_init(struct amdgpu_device *adev);
>   void amdgpu_gart_fini(struct amdgpu_device *adev);
> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>   int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>   		       int pages);
>   int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> index 63e815c27585..a922154953a7 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
>   		if (!amdgpu_device_has_dc_support(adev))
>   			flush_work(&adev->hotplug_work);
>   	}
> +
> +	if (adev->irq.ih_soft.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> +	if (adev->irq.ih.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> +	if (adev->irq.ih1.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> +	if (adev->irq.ih2.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>   }
>   
>   /**
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> index 485f249d063a..62d829f5e62c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
>   		list_del_init(&bo->shadow_list);
>   		mutex_unlock(&adev->shadow_list_lock);
>   	}
> -	amdgpu_bo_unref(&bo->parent);
>   
> +
> +	spin_lock(&adev->mman.bdev.lru_lock);
> +	list_del(&bo->bo);
> +	spin_unlock(&adev->mman.bdev.lru_lock);
> +
> +	amdgpu_bo_unref(&bo->parent);
>   	kfree(bo->metadata);
>   	kfree(bo);
>   }
> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
>   	if (bp->type == ttm_bo_type_device)
>   		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>   
> +	INIT_LIST_HEAD(&bo->bo);
> +
> +	spin_lock(&adev->mman.bdev.lru_lock);
> +	list_add_tail(&bo->bo, &adev->device_bo_list);
> +	spin_unlock(&adev->mman.bdev.lru_lock);
> +
>   	return 0;
>   
>   fail_unreserve:
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> index 9ac37569823f..5ae8555ef275 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>   	struct list_head		shadow_list;
>   
>   	struct kgd_mem                  *kfd_bo;
> +
> +	struct list_head		bo;
>   };
>   
>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> index 183d44a6583c..df385ffc9768 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
>   	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
>   	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> index d32743949003..b8c47e0cf37a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
>   	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
>   	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> index da96c6013477..ddfe4eaeea05 100644
> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
>   	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
>   	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> index 5eea4550b856..e171a9e78544 100644
> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>   
>   	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   
>   	return 0;
>   }
> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> index 751307f3252c..9a24f17a5750 100644
> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
>   	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   
>   	return 0;
>   }
> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> index 973d80ec7f6c..b08905d1c00f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>   
>   	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   	amdgpu_irq_remove_domain(adev);
>   
>   	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> index 2d0094c276ca..8c8abc00f710 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>   
>   	amdgpu_irq_fini_sw(adev);
>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>   
>   	return 0;
>   }

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 07/27] drm/amdgpu: Remap all page faults to per process dummy page.

[Christian König]

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> On device removal reroute all CPU mappings to dummy page
> per drm_file instance or imported GEM object.
>
> v4:
> Update for modified ttm_bo_vm_dummy_page
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++-----
>   1 file changed, 16 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> index a785acc09f20..93163b220e46 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
> @@ -49,6 +49,7 @@
>   
>   #include <drm/drm_debugfs.h>
>   #include <drm/amdgpu_drm.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "amdgpu_object.h"
> @@ -1982,18 +1983,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable)
>   static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf)
>   {
>   	struct ttm_buffer_object *bo = vmf->vma->vm_private_data;
> +	struct drm_device *ddev = bo->base.dev;
>   	vm_fault_t ret;
> +	int idx;
>   
>   	ret = ttm_bo_vm_reserve(bo, vmf);
>   	if (ret)
>   		return ret;
>   
> -	ret = amdgpu_bo_fault_reserve_notify(bo);
> -	if (ret)
> -		goto unlock;
> +	if (drm_dev_enter(ddev, &idx)) {
> +		ret = amdgpu_bo_fault_reserve_notify(bo);
> +		if (ret) {
> +			drm_dev_exit(idx);
> +			goto unlock;
> +		}
>   
> -	ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot,
> -				       TTM_BO_VM_NUM_PREFAULT, 1);
> +		 ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot,
> +						TTM_BO_VM_NUM_PREFAULT, 1);
> +
> +		 drm_dev_exit(idx);
> +	} else {
> +		ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot);
> +	}
>   	if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT))
>   		return ret;
>   

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 09/27] dmr/amdgpu: Move some sysfs attrs creation to default_attr

[Christian König]

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> This allows to remove explicit creation and destruction
> of those attrs and by this avoids warnings on device
> finilizing post physical device extraction.
>
> v5: Use newly added pci_driver.dev_groups directly
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>

Looks like a nice cleanup to me, but this is beyond my understand of sysfs.

Acked-by: Christian König <christian.koenig@amd.com>

Christian.

> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 ++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c      | 13 ++++++++++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c  | 25 ++++++++------------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 ++++-------
>   4 files changed, 37 insertions(+), 32 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> index 86add0f4ea4d..0346e124ab8c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
> @@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev,
>   static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version,
>   		   NULL);
>   
> +static struct attribute *amdgpu_vbios_version_attrs[] = {
> +	&dev_attr_vbios_version.attr,
> +	NULL
> +};
> +
> +const struct attribute_group amdgpu_vbios_version_attr_group = {
> +	.attrs = amdgpu_vbios_version_attrs
> +};
> +
>   /**
>    * amdgpu_atombios_fini - free the driver info and callbacks for atombios
>    *
> @@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev)
>   	adev->mode_info.atom_context = NULL;
>   	kfree(adev->mode_info.atom_card_info);
>   	adev->mode_info.atom_card_info = NULL;
> -	device_remove_file(adev->dev, &dev_attr_vbios_version);
>   }
>   
>   /**
> @@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
>   {
>   	struct card_info *atom_card_info =
>   	    kzalloc(sizeof(struct card_info), GFP_KERNEL);
> -	int ret;
>   
>   	if (!atom_card_info)
>   		return -ENOMEM;
> @@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
>   		amdgpu_atombios_allocate_fb_scratch(adev);
>   	}
>   
> -	ret = device_create_file(adev->dev, &dev_attr_vbios_version);
> -	if (ret) {
> -		DRM_ERROR("Failed to create device file for VBIOS version\n");
> -		return ret;
> -	}
> -
>   	return 0;
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> index 54cb5ee2f563..f799c40d7e72 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> @@ -1605,6 +1605,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = {
>   	.resume		= amdgpu_pci_resume,
>   };
>   
> +extern const struct attribute_group amdgpu_vram_mgr_attr_group;
> +extern const struct attribute_group amdgpu_gtt_mgr_attr_group;
> +extern const struct attribute_group amdgpu_vbios_version_attr_group;
> +
> +static const struct attribute_group *amdgpu_sysfs_groups[] = {
> +	&amdgpu_vram_mgr_attr_group,
> +	&amdgpu_gtt_mgr_attr_group,
> +	&amdgpu_vbios_version_attr_group,
> +	NULL,
> +};
> +
> +
>   static struct pci_driver amdgpu_kms_pci_driver = {
>   	.name = DRIVER_NAME,
>   	.id_table = pciidlist,
> @@ -1613,6 +1625,7 @@ static struct pci_driver amdgpu_kms_pci_driver = {
>   	.shutdown = amdgpu_pci_shutdown,
>   	.driver.pm = &amdgpu_pm_ops,
>   	.err_handler = &amdgpu_pci_err_handler,
> +	.dev_groups = amdgpu_sysfs_groups,
>   };
>   
>   static int __init amdgpu_init(void)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
> index 8980329cded0..3b7150e1c5ed 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
> @@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO,
>   static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO,
>   	           amdgpu_mem_info_gtt_used_show, NULL);
>   
> +static struct attribute *amdgpu_gtt_mgr_attributes[] = {
> +	&dev_attr_mem_info_gtt_total.attr,
> +	&dev_attr_mem_info_gtt_used.attr,
> +	NULL
> +};
> +
> +const struct attribute_group amdgpu_gtt_mgr_attr_group = {
> +	.attrs = amdgpu_gtt_mgr_attributes
> +};
> +
>   static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func;
>   /**
>    * amdgpu_gtt_mgr_init - init GTT manager and DRM MM
> @@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
>   	struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr;
>   	struct ttm_resource_manager *man = &mgr->manager;
>   	uint64_t start, size;
> -	int ret;
>   
>   	man->use_tt = true;
>   	man->func = &amdgpu_gtt_mgr_func;
> @@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size)
>   	spin_lock_init(&mgr->lock);
>   	atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT);
>   
> -	ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total);
> -	if (ret) {
> -		DRM_ERROR("Failed to create device file mem_info_gtt_total\n");
> -		return ret;
> -	}
> -	ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used);
> -	if (ret) {
> -		DRM_ERROR("Failed to create device file mem_info_gtt_used\n");
> -		return ret;
> -	}
> -
>   	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager);
>   	ttm_resource_manager_set_used(man, true);
>   	return 0;
> @@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev)
>   	drm_mm_takedown(&mgr->mm);
>   	spin_unlock(&mgr->lock);
>   
> -	device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total);
> -	device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used);
> -
>   	ttm_resource_manager_cleanup(man);
>   	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL);
>   }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> index c89b66bb70e2..68369b38aebb 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
> @@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO,
>   static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO,
>   		   amdgpu_mem_info_vram_vendor, NULL);
>   
> -static const struct attribute *amdgpu_vram_mgr_attributes[] = {
> +static struct attribute *amdgpu_vram_mgr_attributes[] = {
>   	&dev_attr_mem_info_vram_total.attr,
>   	&dev_attr_mem_info_vis_vram_total.attr,
>   	&dev_attr_mem_info_vram_used.attr,
> @@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = {
>   	NULL
>   };
>   
> +const struct attribute_group amdgpu_vram_mgr_attr_group = {
> +	.attrs = amdgpu_vram_mgr_attributes
> +};
> +
>   static const struct ttm_resource_manager_func amdgpu_vram_mgr_func;
>   
>   /**
> @@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
>   {
>   	struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr;
>   	struct ttm_resource_manager *man = &mgr->manager;
> -	int ret;
>   
>   	ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT);
>   
> @@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev)
>   	INIT_LIST_HEAD(&mgr->reservations_pending);
>   	INIT_LIST_HEAD(&mgr->reserved_pages);
>   
> -	/* Add the two VRAM-related sysfs files */
> -	ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
> -	if (ret)
> -		DRM_ERROR("Failed to register sysfs\n");
> -
>   	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager);
>   	ttm_resource_manager_set_used(man, true);
>   	return 0;
> @@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev)
>   	drm_mm_takedown(&mgr->mm);
>   	spin_unlock(&mgr->lock);
>   
> -	sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes);
> -
>   	ttm_resource_manager_cleanup(man);
>   	ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL);
>   }

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 10/27] drm/amdgpu: Guard against write accesses after device removal

[Christian König]

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> This should prevent writing to memory or IO ranges possibly
> already allocated for other uses after our device is removed.
>
> v5:
> Protect more places wher memcopy_to/form_io takes place
> Protect IB submissions
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    |  75 +++---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c       |   9 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c        | 228 +++++++++---------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       | 115 +++++----
>   drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h       |   3 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c      |  70 ++++++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h      |  49 +---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c       |  31 ++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c       |  11 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c       |  22 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c        |   7 +-
>   drivers/gpu/drm/amd/amdgpu/psp_v11_0.c        |  44 ++--
>   drivers/gpu/drm/amd/amdgpu/psp_v12_0.c        |   8 +-
>   drivers/gpu/drm/amd/amdgpu/psp_v3_1.c         |   8 +-
>   drivers/gpu/drm/amd/amdgpu/vce_v4_0.c         |  26 +-
>   drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c         |  22 +-
>   .../drm/amd/pm/powerplay/smumgr/smu7_smumgr.c |   2 +
>   17 files changed, 425 insertions(+), 305 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 91594ddc2459..22b09c4db255 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -71,6 +71,8 @@
>   #include <linux/pm_runtime.h>
>   
>   
> +#include <drm/drm_drv.h>
> +
>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
> @@ -279,48 +281,55 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
>   	unsigned long flags;
>   	uint32_t hi = ~0;
>   	uint64_t last;
> +	int idx;
>   
> +	if (drm_dev_enter(&adev->ddev, &idx)) {

In general I think it would be better to do stuff like this as

if (!drm_dev_enter(...))
     return; /* or goto error_nodev; */

...

drm_dev_exit();

This is essentially error handling and we avoid modifying the whole 
function for adding the prerequisite.

Regards,
Christian.


>   
>   #ifdef CONFIG_64BIT
> -	last = min(pos + size, adev->gmc.visible_vram_size);
> -	if (last > pos) {
> -		void __iomem *addr = adev->mman.aper_base_kaddr + pos;
> -		size_t count = last - pos;
> -
> -		if (write) {
> -			memcpy_toio(addr, buf, count);
> -			mb();
> -			amdgpu_asic_flush_hdp(adev, NULL);
> -		} else {
> -			amdgpu_asic_invalidate_hdp(adev, NULL);
> -			mb();
> -			memcpy_fromio(buf, addr, count);
> -		}
> +		last = min(pos + size, adev->gmc.visible_vram_size);
> +		if (last > pos) {
> +			void __iomem *addr = adev->mman.aper_base_kaddr + pos;
> +			size_t count = last - pos;
> +
> +			if (write) {
> +				memcpy_toio(addr, buf, count);
> +				mb();
> +				amdgpu_asic_flush_hdp(adev, NULL);
> +			} else {
> +				amdgpu_asic_invalidate_hdp(adev, NULL);
> +				mb();
> +				memcpy_fromio(buf, addr, count);
> +			}
>   
> -		if (count == size)
> -			return;
> +			if (count == size) {
> +				drm_dev_exit(idx);
> +				return;
> +			}
>   
> -		pos += count;
> -		buf += count / 4;
> -		size -= count;
> -	}
> +			pos += count;
> +			buf += count / 4;
> +			size -= count;
> +		}
>   #endif
>   
> -	spin_lock_irqsave(&adev->mmio_idx_lock, flags);
> -	for (last = pos + size; pos < last; pos += 4) {
> -		uint32_t tmp = pos >> 31;
> +		spin_lock_irqsave(&adev->mmio_idx_lock, flags);
> +		for (last = pos + size; pos < last; pos += 4) {
> +			uint32_t tmp = pos >> 31;
>   
> -		WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000);
> -		if (tmp != hi) {
> -			WREG32_NO_KIQ(mmMM_INDEX_HI, tmp);
> -			hi = tmp;
> +			WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000);
> +			if (tmp != hi) {
> +				WREG32_NO_KIQ(mmMM_INDEX_HI, tmp);
> +				hi = tmp;
> +			}
> +			if (write)
> +				WREG32_NO_KIQ(mmMM_DATA, *buf++);
> +			else
> +				*buf++ = RREG32_NO_KIQ(mmMM_DATA);
>   		}
> -		if (write)
> -			WREG32_NO_KIQ(mmMM_DATA, *buf++);
> -		else
> -			*buf++ = RREG32_NO_KIQ(mmMM_DATA);
> +		spin_unlock_irqrestore(&adev->mmio_idx_lock, flags);
> +
> +		drm_dev_exit(idx);
>   	}
> -	spin_unlock_irqrestore(&adev->mmio_idx_lock, flags);
>   }
>   
>   /*
> @@ -402,6 +411,7 @@ uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset)
>    */
>   void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value)
>   {
> +
>   	if (adev->in_pci_err_recovery)
>   		return;
>   
> @@ -542,6 +552,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index)
>    */
>   void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v)
>   {
> +
>   	if (adev->in_pci_err_recovery)
>   		return;
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
> index fe1a39ffda72..1beb4e64b884 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
> @@ -31,6 +31,8 @@
>   #include "amdgpu_ras.h"
>   #include "amdgpu_xgmi.h"
>   
> +#include <drm/drm_drv.h>
> +
>   /**
>    * amdgpu_gmc_get_pde_for_bo - get the PDE for a BO
>    *
> @@ -98,6 +100,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr,
>   {
>   	void __iomem *ptr = (void *)cpu_pt_addr;
>   	uint64_t value;
> +	int idx;
> +
> +	if (!drm_dev_enter(&adev->ddev, &idx))
> +		return 0;
>   
>   	/*
>   	 * The following is for PTE only. GART does not have PDEs.
> @@ -105,6 +111,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr,
>   	value = addr & 0x0000FFFFFFFFF000ULL;
>   	value |= flags;
>   	writeq(value, ptr + (gpu_page_idx * 8));
> +
> +	drm_dev_exit(idx);
> +
>   	return 0;
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
> index 7645223ea0ef..b3a1ff04c10f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
> @@ -31,6 +31,7 @@
>   
>   #include <drm/amdgpu_drm.h>
>   #include <drm/drm_debugfs.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "atom.h"
> @@ -138,7 +139,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
>   	bool secure;
>   
>   	unsigned i;
> -	int r = 0;
> +	int idx, r = 0;
>   	bool need_pipe_sync = false;
>   
>   	if (num_ibs == 0)
> @@ -170,142 +171,151 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
>   		return -EINVAL;
>   	}
>   
> -	alloc_size = ring->funcs->emit_frame_size + num_ibs *
> -		ring->funcs->emit_ib_size;
> +	if (drm_dev_enter(&adev->ddev, &idx)) {
>   
> -	r = amdgpu_ring_alloc(ring, alloc_size);
> -	if (r) {
> -		dev_err(adev->dev, "scheduling IB failed (%d).\n", r);
> -		return r;
> -	}
> -
> -	need_ctx_switch = ring->current_ctx != fence_ctx;
> -	if (ring->funcs->emit_pipeline_sync && job &&
> -	    ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
> -	     (amdgpu_sriov_vf(adev) && need_ctx_switch) ||
> -	     amdgpu_vm_need_pipeline_sync(ring, job))) {
> -		need_pipe_sync = true;
> -
> -		if (tmp)
> -			trace_amdgpu_ib_pipe_sync(job, tmp);
> -
> -		dma_fence_put(tmp);
> -	}
> +		alloc_size = ring->funcs->emit_frame_size + num_ibs *
> +			ring->funcs->emit_ib_size;
>   
> -	if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync)
> -		ring->funcs->emit_mem_sync(ring);
> +		r = amdgpu_ring_alloc(ring, alloc_size);
> +		if (r) {
> +			dev_err(adev->dev, "scheduling IB failed (%d).\n", r);
> +			goto exit;
> +		}
>   
> -	if (ring->funcs->emit_wave_limit &&
> -	    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
> -		ring->funcs->emit_wave_limit(ring, true);
> +		need_ctx_switch = ring->current_ctx != fence_ctx;
> +		if (ring->funcs->emit_pipeline_sync && job &&
> +		    ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
> +		     (amdgpu_sriov_vf(adev) && need_ctx_switch) ||
> +		     amdgpu_vm_need_pipeline_sync(ring, job))) {
> +			need_pipe_sync = true;
>   
> -	if (ring->funcs->insert_start)
> -		ring->funcs->insert_start(ring);
> +			if (tmp)
> +				trace_amdgpu_ib_pipe_sync(job, tmp);
>   
> -	if (job) {
> -		r = amdgpu_vm_flush(ring, job, need_pipe_sync);
> -		if (r) {
> -			amdgpu_ring_undo(ring);
> -			return r;
> +			dma_fence_put(tmp);
>   		}
> -	}
>   
> -	if (job && ring->funcs->init_cond_exec)
> -		patch_offset = amdgpu_ring_init_cond_exec(ring);
> +		if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync)
> +			ring->funcs->emit_mem_sync(ring);
>   
> -#ifdef CONFIG_X86_64
> -	if (!(adev->flags & AMD_IS_APU))
> -#endif
> -	{
> -		if (ring->funcs->emit_hdp_flush)
> -			amdgpu_ring_emit_hdp_flush(ring);
> -		else
> -			amdgpu_asic_flush_hdp(adev, ring);
> -	}
> +		if (ring->funcs->emit_wave_limit &&
> +		    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
> +			ring->funcs->emit_wave_limit(ring, true);
>   
> -	if (need_ctx_switch)
> -		status |= AMDGPU_HAVE_CTX_SWITCH;
> +		if (ring->funcs->insert_start)
> +			ring->funcs->insert_start(ring);
>   
> -	skip_preamble = ring->current_ctx == fence_ctx;
> -	if (job && ring->funcs->emit_cntxcntl) {
> -		status |= job->preamble_status;
> -		status |= job->preemption_status;
> -		amdgpu_ring_emit_cntxcntl(ring, status);
> -	}
> +		if (job) {
> +			r = amdgpu_vm_flush(ring, job, need_pipe_sync);
> +			if (r) {
> +				amdgpu_ring_undo(ring);
> +				goto exit;
> +			}
> +		}
>   
> -	/* Setup initial TMZiness and send it off.
> -	 */
> -	secure = false;
> -	if (job && ring->funcs->emit_frame_cntl) {
> -		secure = ib->flags & AMDGPU_IB_FLAGS_SECURE;
> -		amdgpu_ring_emit_frame_cntl(ring, true, secure);
> -	}
> +		if (job && ring->funcs->init_cond_exec)
> +			patch_offset = amdgpu_ring_init_cond_exec(ring);
> +
> +	#ifdef CONFIG_X86_64
> +		if (!(adev->flags & AMD_IS_APU))
> +	#endif
> +		{
> +			if (ring->funcs->emit_hdp_flush)
> +				amdgpu_ring_emit_hdp_flush(ring);
> +			else
> +				amdgpu_asic_flush_hdp(adev, ring);
> +		}
>   
> -	for (i = 0; i < num_ibs; ++i) {
> -		ib = &ibs[i];
> +		if (need_ctx_switch)
> +			status |= AMDGPU_HAVE_CTX_SWITCH;
>   
> -		/* drop preamble IBs if we don't have a context switch */
> -		if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) &&
> -		    skip_preamble &&
> -		    !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) &&
> -		    !amdgpu_mcbp &&
> -		    !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */
> -			continue;
> +		skip_preamble = ring->current_ctx == fence_ctx;
> +		if (job && ring->funcs->emit_cntxcntl) {
> +			status |= job->preamble_status;
> +			status |= job->preemption_status;
> +			amdgpu_ring_emit_cntxcntl(ring, status);
> +		}
>   
> +		/* Setup initial TMZiness and send it off.
> +		 */
> +		secure = false;
>   		if (job && ring->funcs->emit_frame_cntl) {
> -			if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) {
> -				amdgpu_ring_emit_frame_cntl(ring, false, secure);
> -				secure = !secure;
> -				amdgpu_ring_emit_frame_cntl(ring, true, secure);
> +			secure = ib->flags & AMDGPU_IB_FLAGS_SECURE;
> +			amdgpu_ring_emit_frame_cntl(ring, true, secure);
> +		}
> +
> +		for (i = 0; i < num_ibs; ++i) {
> +			ib = &ibs[i];
> +
> +			/* drop preamble IBs if we don't have a context switch */
> +			if ((ib->flags & AMDGPU_IB_FLAG_PREAMBLE) &&
> +			    skip_preamble &&
> +			    !(status & AMDGPU_PREAMBLE_IB_PRESENT_FIRST) &&
> +			    !amdgpu_mcbp &&
> +			    !amdgpu_sriov_vf(adev)) /* for SRIOV preemption, Preamble CE ib must be inserted anyway */
> +				continue;
> +
> +			if (job && ring->funcs->emit_frame_cntl) {
> +				if (secure != !!(ib->flags & AMDGPU_IB_FLAGS_SECURE)) {
> +					amdgpu_ring_emit_frame_cntl(ring, false, secure);
> +					secure = !secure;
> +					amdgpu_ring_emit_frame_cntl(ring, true, secure);
> +				}
>   			}
> +
> +			amdgpu_ring_emit_ib(ring, job, ib, status);
> +			status &= ~AMDGPU_HAVE_CTX_SWITCH;
>   		}
>   
> -		amdgpu_ring_emit_ib(ring, job, ib, status);
> -		status &= ~AMDGPU_HAVE_CTX_SWITCH;
> -	}
> +		if (job && ring->funcs->emit_frame_cntl)
> +			amdgpu_ring_emit_frame_cntl(ring, false, secure);
>   
> -	if (job && ring->funcs->emit_frame_cntl)
> -		amdgpu_ring_emit_frame_cntl(ring, false, secure);
> +	#ifdef CONFIG_X86_64
> +		if (!(adev->flags & AMD_IS_APU))
> +	#endif
> +			amdgpu_asic_invalidate_hdp(adev, ring);
>   
> -#ifdef CONFIG_X86_64
> -	if (!(adev->flags & AMD_IS_APU))
> -#endif
> -		amdgpu_asic_invalidate_hdp(adev, ring);
> +		if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE)
> +			fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY;
>   
> -	if (ib->flags & AMDGPU_IB_FLAG_TC_WB_NOT_INVALIDATE)
> -		fence_flags |= AMDGPU_FENCE_FLAG_TC_WB_ONLY;
> +		/* wrap the last IB with fence */
> +		if (job && job->uf_addr) {
> +			amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence,
> +					       fence_flags | AMDGPU_FENCE_FLAG_64BIT);
> +		}
>   
> -	/* wrap the last IB with fence */
> -	if (job && job->uf_addr) {
> -		amdgpu_ring_emit_fence(ring, job->uf_addr, job->uf_sequence,
> -				       fence_flags | AMDGPU_FENCE_FLAG_64BIT);
> -	}
> +		r = amdgpu_fence_emit(ring, f, fence_flags);
> +		if (r) {
> +			dev_err(adev->dev, "failed to emit fence (%d)\n", r);
> +			if (job && job->vmid)
> +				amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid);
> +			amdgpu_ring_undo(ring);
> +			goto exit;
> +		}
>   
> -	r = amdgpu_fence_emit(ring, f, fence_flags);
> -	if (r) {
> -		dev_err(adev->dev, "failed to emit fence (%d)\n", r);
> -		if (job && job->vmid)
> -			amdgpu_vmid_reset(adev, ring->funcs->vmhub, job->vmid);
> -		amdgpu_ring_undo(ring);
> -		return r;
> -	}
> +		if (ring->funcs->insert_end)
> +			ring->funcs->insert_end(ring);
>   
> -	if (ring->funcs->insert_end)
> -		ring->funcs->insert_end(ring);
> +		if (patch_offset != ~0 && ring->funcs->patch_cond_exec)
> +			amdgpu_ring_patch_cond_exec(ring, patch_offset);
>   
> -	if (patch_offset != ~0 && ring->funcs->patch_cond_exec)
> -		amdgpu_ring_patch_cond_exec(ring, patch_offset);
> +		ring->current_ctx = fence_ctx;
> +		if (vm && ring->funcs->emit_switch_buffer)
> +			amdgpu_ring_emit_switch_buffer(ring);
>   
> -	ring->current_ctx = fence_ctx;
> -	if (vm && ring->funcs->emit_switch_buffer)
> -		amdgpu_ring_emit_switch_buffer(ring);
> +		if (ring->funcs->emit_wave_limit &&
> +		    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
> +			ring->funcs->emit_wave_limit(ring, false);
>   
> -	if (ring->funcs->emit_wave_limit &&
> -	    ring->hw_prio == AMDGPU_GFX_PIPE_PRIO_HIGH)
> -		ring->funcs->emit_wave_limit(ring, false);
> +		amdgpu_ring_commit(ring);
>   
> -	amdgpu_ring_commit(ring);
> -	return 0;
> +	} else {
> +		return -ENODEV;
> +	}
> +
> +exit:
> +	drm_dev_exit(idx);
> +	return r;
>   }
>   
>   /**
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
> index 839917eb7bc3..638b7fd1857c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
> @@ -25,6 +25,7 @@
>   
>   #include <linux/firmware.h>
>   #include <linux/dma-mapping.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "amdgpu_psp.h"
> @@ -38,6 +39,8 @@
>   #include "amdgpu_ras.h"
>   #include "amdgpu_securedisplay.h"
>   
> +#include <drm/drm_drv.h>
> +
>   static int psp_sysfs_init(struct amdgpu_device *adev);
>   static void psp_sysfs_fini(struct amdgpu_device *adev);
>   
> @@ -249,7 +252,7 @@ psp_cmd_submit_buf(struct psp_context *psp,
>   		   struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr)
>   {
>   	int ret;
> -	int index;
> +	int index, idx;
>   	int timeout = 20000;
>   	bool ras_intr = false;
>   	bool skip_unsupport = false;
> @@ -257,6 +260,9 @@ psp_cmd_submit_buf(struct psp_context *psp,
>   	if (psp->adev->in_pci_err_recovery)
>   		return 0;
>   
> +	if (!drm_dev_enter(&psp->adev->ddev, &idx))
> +		return 0;
> +
>   	mutex_lock(&psp->mutex);
>   
>   	memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE);
> @@ -267,8 +273,7 @@ psp_cmd_submit_buf(struct psp_context *psp,
>   	ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index);
>   	if (ret) {
>   		atomic_dec(&psp->fence_value);
> -		mutex_unlock(&psp->mutex);
> -		return ret;
> +		goto exit;
>   	}
>   
>   	amdgpu_asic_invalidate_hdp(psp->adev, NULL);
> @@ -308,8 +313,8 @@ psp_cmd_submit_buf(struct psp_context *psp,
>   			 psp->cmd_buf_mem->cmd_id,
>   			 psp->cmd_buf_mem->resp.status);
>   		if (!timeout) {
> -			mutex_unlock(&psp->mutex);
> -			return -EINVAL;
> +			ret = -EINVAL;
> +			goto exit;
>   		}
>   	}
>   
> @@ -317,8 +322,10 @@ psp_cmd_submit_buf(struct psp_context *psp,
>   		ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo;
>   		ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi;
>   	}
> -	mutex_unlock(&psp->mutex);
>   
> +exit:
> +	mutex_unlock(&psp->mutex);
> +	drm_dev_exit(idx);
>   	return ret;
>   }
>   
> @@ -355,8 +362,7 @@ static int psp_load_toc(struct psp_context *psp,
>   	if (!cmd)
>   		return -ENOMEM;
>   	/* Copy toc to psp firmware private buffer */
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -	memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size);
> +	psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size);
>   
>   	psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size);
>   
> @@ -571,8 +577,7 @@ static int psp_asd_load(struct psp_context *psp)
>   	if (!cmd)
>   		return -ENOMEM;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -	memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size);
> +	psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size);
>   
>   	psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr,
>   				  psp->asd_ucode_size);
> @@ -727,8 +732,7 @@ static int psp_xgmi_load(struct psp_context *psp)
>   	if (!cmd)
>   		return -ENOMEM;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -	memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size);
> +	psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size);
>   
>   	psp_prep_ta_load_cmd_buf(cmd,
>   				 psp->fw_pri_mc_addr,
> @@ -983,8 +987,7 @@ static int psp_ras_load(struct psp_context *psp)
>   	if (!cmd)
>   		return -ENOMEM;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -	memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size);
> +	psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size);
>   
>   	psp_prep_ta_load_cmd_buf(cmd,
>   				 psp->fw_pri_mc_addr,
> @@ -1220,8 +1223,7 @@ static int psp_hdcp_load(struct psp_context *psp)
>   	if (!cmd)
>   		return -ENOMEM;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -	memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr,
> +	psp_copy_fw(psp, psp->ta_hdcp_start_addr,
>   	       psp->ta_hdcp_ucode_size);
>   
>   	psp_prep_ta_load_cmd_buf(cmd,
> @@ -1372,8 +1374,7 @@ static int psp_dtm_load(struct psp_context *psp)
>   	if (!cmd)
>   		return -ENOMEM;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -	memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size);
> +	psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size);
>   
>   	psp_prep_ta_load_cmd_buf(cmd,
>   				 psp->fw_pri_mc_addr,
> @@ -1518,8 +1519,7 @@ static int psp_rap_load(struct psp_context *psp)
>   	if (!cmd)
>   		return -ENOMEM;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -	memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size);
> +	psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size);
>   
>   	psp_prep_ta_load_cmd_buf(cmd,
>   				 psp->fw_pri_mc_addr,
> @@ -2928,7 +2928,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev,
>   	struct amdgpu_device *adev = drm_to_adev(ddev);
>   	void *cpu_addr;
>   	dma_addr_t dma_addr;
> -	int ret;
> +	int ret, idx;
>   	char fw_name[100];
>   	const struct firmware *usbc_pd_fw;
>   
> @@ -2937,47 +2937,66 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev,
>   		return -EBUSY;
>   	}
>   
> -	snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf);
> -	ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev);
> -	if (ret)
> -		goto fail;
> +	if (drm_dev_enter(ddev, &idx)) {
>   
> -	/* We need contiguous physical mem to place the FW  for psp to access */
> -	cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL);
> +		snprintf(fw_name, sizeof(fw_name), "amdgpu/%s", buf);
> +		ret = request_firmware(&usbc_pd_fw, fw_name, adev->dev);
> +		if (ret)
> +			goto fail;
>   
> -	ret = dma_mapping_error(adev->dev, dma_addr);
> -	if (ret)
> -		goto rel_buf;
> +		/* We need contiguous physical mem to place the FW  for psp to access */
> +		cpu_addr = dma_alloc_coherent(adev->dev, usbc_pd_fw->size, &dma_addr, GFP_KERNEL);
>   
> -	memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size);
> +		ret = dma_mapping_error(adev->dev, dma_addr);
> +		if (ret)
> +			goto rel_buf;
>   
> -	/*
> -	 * x86 specific workaround.
> -	 * Without it the buffer is invisible in PSP.
> -	 *
> -	 * TODO Remove once PSP starts snooping CPU cache
> -	 */
> +		memcpy_toio(cpu_addr, usbc_pd_fw->data, usbc_pd_fw->size);
> +
> +		/*
> +		 * x86 specific workaround.
> +		 * Without it the buffer is invisible in PSP.
> +		 *
> +		 * TODO Remove once PSP starts snooping CPU cache
> +		 */
>   #ifdef CONFIG_X86
> -	clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1)));
> +		clflush_cache_range(cpu_addr, (usbc_pd_fw->size & ~(L1_CACHE_BYTES - 1)));
>   #endif
>   
> -	mutex_lock(&adev->psp.mutex);
> -	ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr);
> -	mutex_unlock(&adev->psp.mutex);
> +		mutex_lock(&adev->psp.mutex);
> +		ret = psp_load_usbc_pd_fw(&adev->psp, dma_addr);
> +		mutex_unlock(&adev->psp.mutex);
>   
>   rel_buf:
> -	dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr);
> -	release_firmware(usbc_pd_fw);
> -
> +		dma_free_coherent(adev->dev, usbc_pd_fw->size, cpu_addr, dma_addr);
> +		release_firmware(usbc_pd_fw);
>   fail:
> -	if (ret) {
> -		DRM_ERROR("Failed to load USBC PD FW, err = %d", ret);
> -		return ret;
> +		if (ret) {
> +			DRM_ERROR("Failed to load USBC PD FW, err = %d", ret);
> +			return ret;
> +		}
> +
> +		drm_dev_exit(idx);
> +		return count;
> +	} else {
> +		return -ENODEV;
>   	}
> +}
> +
> +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size)
> +{
> +	int idx;
>   
> -	return count;
> +	if (!drm_dev_enter(&psp->adev->ddev, &idx))
> +		return;
> +
> +	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> +	memcpy(psp->fw_pri_buf, start_addr, bin_size);
> +
> +	drm_dev_exit(idx);
>   }
>   
> +
>   static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR,
>   		   psp_usbc_pd_fw_sysfs_read,
>   		   psp_usbc_pd_fw_sysfs_write);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
> index cb50ba445f8c..0dbbedb84c84 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
> @@ -417,4 +417,7 @@ int psp_init_ta_microcode(struct psp_context *psp,
>   			  const char *chip_name);
>   int psp_get_fw_attestation_records_addr(struct psp_context *psp,
>   					uint64_t *output_ptr);
> +
> +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size);
> +
>   #endif
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
> index b644c78475fd..f9f71008f454 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
> @@ -35,6 +35,8 @@
>   #include "amdgpu.h"
>   #include "atom.h"
>   
> +#include <drm/drm_drv.h>
> +
>   /*
>    * Rings
>    * Most engines on the GPU are fed via ring buffers.  Ring
> @@ -459,3 +461,71 @@ int amdgpu_ring_test_helper(struct amdgpu_ring *ring)
>   	ring->sched.ready = !r;
>   	return r;
>   }
> +
> +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring)
> +{
> +	int idx;
> +	int i = 0;
> +
> +	if (!drm_dev_enter(&ring->adev->ddev, &idx))
> +		return;
> +
> +	while (i <= ring->buf_mask)
> +		ring->ring[i++] = ring->funcs->nop;
> +
> +	drm_dev_exit(idx);
> +
> +}
> +
> +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v)
> +{
> +	int idx;
> +
> +	if (!drm_dev_enter(&ring->adev->ddev, &idx))
> +		return;
> +
> +	if (ring->count_dw <= 0)
> +		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
> +	ring->ring[ring->wptr++ & ring->buf_mask] = v;
> +	ring->wptr &= ring->ptr_mask;
> +	ring->count_dw--;
> +
> +	drm_dev_exit(idx);
> +}
> +
> +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
> +					      void *src, int count_dw)
> +{
> +	unsigned occupied, chunk1, chunk2;
> +	void *dst;
> +	int idx;
> +
> +	if (!drm_dev_enter(&ring->adev->ddev, &idx))
> +		return;
> +
> +	if (unlikely(ring->count_dw < count_dw))
> +		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
> +
> +	occupied = ring->wptr & ring->buf_mask;
> +	dst = (void *)&ring->ring[occupied];
> +	chunk1 = ring->buf_mask + 1 - occupied;
> +	chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1;
> +	chunk2 = count_dw - chunk1;
> +	chunk1 <<= 2;
> +	chunk2 <<= 2;
> +
> +	if (chunk1)
> +		memcpy(dst, src, chunk1);
> +
> +	if (chunk2) {
> +		src += chunk1;
> +		dst = (void *)ring->ring;
> +		memcpy(dst, src, chunk2);
> +	}
> +
> +	ring->wptr += count_dw;
> +	ring->wptr &= ring->ptr_mask;
> +	ring->count_dw -= count_dw;
> +
> +	drm_dev_exit(idx);
> +}
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> index 0f195f7bf797..c2b83f48e6d6 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> @@ -297,53 +297,12 @@ static inline void amdgpu_ring_set_preempt_cond_exec(struct amdgpu_ring *ring,
>   	*ring->cond_exe_cpu_addr = cond_exec;
>   }
>   
> -static inline void amdgpu_ring_clear_ring(struct amdgpu_ring *ring)
> -{
> -	int i = 0;
> -	while (i <= ring->buf_mask)
> -		ring->ring[i++] = ring->funcs->nop;
> -
> -}
> -
> -static inline void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v)
> -{
> -	if (ring->count_dw <= 0)
> -		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
> -	ring->ring[ring->wptr++ & ring->buf_mask] = v;
> -	ring->wptr &= ring->ptr_mask;
> -	ring->count_dw--;
> -}
> +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring);
>   
> -static inline void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
> -					      void *src, int count_dw)
> -{
> -	unsigned occupied, chunk1, chunk2;
> -	void *dst;
> -
> -	if (unlikely(ring->count_dw < count_dw))
> -		DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n");
> -
> -	occupied = ring->wptr & ring->buf_mask;
> -	dst = (void *)&ring->ring[occupied];
> -	chunk1 = ring->buf_mask + 1 - occupied;
> -	chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1;
> -	chunk2 = count_dw - chunk1;
> -	chunk1 <<= 2;
> -	chunk2 <<= 2;
> +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v);
>   
> -	if (chunk1)
> -		memcpy(dst, src, chunk1);
> -
> -	if (chunk2) {
> -		src += chunk1;
> -		dst = (void *)ring->ring;
> -		memcpy(dst, src, chunk2);
> -	}
> -
> -	ring->wptr += count_dw;
> -	ring->wptr &= ring->ptr_mask;
> -	ring->count_dw -= count_dw;
> -}
> +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring,
> +					      void *src, int count_dw);
>   
>   int amdgpu_ring_test_helper(struct amdgpu_ring *ring);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
> index e2ed4689118a..df47f5ffa08f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
> @@ -32,6 +32,7 @@
>   #include <linux/module.h>
>   
>   #include <drm/drm.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "amdgpu_pm.h"
> @@ -375,7 +376,7 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev)
>   {
>   	unsigned size;
>   	void *ptr;
> -	int i, j;
> +	int i, j, idx;
>   	bool in_ras_intr = amdgpu_ras_intr_triggered();
>   
>   	cancel_delayed_work_sync(&adev->uvd.idle_work);
> @@ -403,11 +404,15 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev)
>   		if (!adev->uvd.inst[j].saved_bo)
>   			return -ENOMEM;
>   
> -		/* re-write 0 since err_event_athub will corrupt VCPU buffer */
> -		if (in_ras_intr)
> -			memset(adev->uvd.inst[j].saved_bo, 0, size);
> -		else
> -			memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size);
> +		if (drm_dev_enter(&adev->ddev, &idx)) {
> +			/* re-write 0 since err_event_athub will corrupt VCPU buffer */
> +			if (in_ras_intr)
> +				memset(adev->uvd.inst[j].saved_bo, 0, size);
> +			else
> +				memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size);
> +
> +			drm_dev_exit(idx);
> +		}
>   	}
>   
>   	if (in_ras_intr)
> @@ -420,7 +425,7 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev)
>   {
>   	unsigned size;
>   	void *ptr;
> -	int i;
> +	int i, idx;
>   
>   	for (i = 0; i < adev->uvd.num_uvd_inst; i++) {
>   		if (adev->uvd.harvest_config & (1 << i))
> @@ -432,7 +437,10 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev)
>   		ptr = adev->uvd.inst[i].cpu_addr;
>   
>   		if (adev->uvd.inst[i].saved_bo != NULL) {
> -			memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size);
> +			if (drm_dev_enter(&adev->ddev, &idx)) {
> +				memcpy_toio(ptr, adev->uvd.inst[i].saved_bo, size);
> +				drm_dev_exit(idx);
> +			}
>   			kvfree(adev->uvd.inst[i].saved_bo);
>   			adev->uvd.inst[i].saved_bo = NULL;
>   		} else {
> @@ -442,8 +450,11 @@ int amdgpu_uvd_resume(struct amdgpu_device *adev)
>   			hdr = (const struct common_firmware_header *)adev->uvd.fw->data;
>   			if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) {
>   				offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
> -				memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset,
> -					    le32_to_cpu(hdr->ucode_size_bytes));
> +				if (drm_dev_enter(&adev->ddev, &idx)) {
> +					memcpy_toio(adev->uvd.inst[i].cpu_addr, adev->uvd.fw->data + offset,
> +						    le32_to_cpu(hdr->ucode_size_bytes));
> +					drm_dev_exit(idx);
> +				}
>   				size -= le32_to_cpu(hdr->ucode_size_bytes);
>   				ptr += le32_to_cpu(hdr->ucode_size_bytes);
>   			}
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
> index ea6a62f67e38..833203401ef4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c
> @@ -29,6 +29,7 @@
>   #include <linux/module.h>
>   
>   #include <drm/drm.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "amdgpu_pm.h"
> @@ -293,7 +294,7 @@ int amdgpu_vce_resume(struct amdgpu_device *adev)
>   	void *cpu_addr;
>   	const struct common_firmware_header *hdr;
>   	unsigned offset;
> -	int r;
> +	int r, idx;
>   
>   	if (adev->vce.vcpu_bo == NULL)
>   		return -EINVAL;
> @@ -313,8 +314,12 @@ int amdgpu_vce_resume(struct amdgpu_device *adev)
>   
>   	hdr = (const struct common_firmware_header *)adev->vce.fw->data;
>   	offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
> -	memcpy_toio(cpu_addr, adev->vce.fw->data + offset,
> -		    adev->vce.fw->size - offset);
> +
> +	if (drm_dev_enter(&adev->ddev, &idx)) {
> +		memcpy_toio(cpu_addr, adev->vce.fw->data + offset,
> +			    adev->vce.fw->size - offset);
> +		drm_dev_exit(idx);
> +	}
>   
>   	amdgpu_bo_kunmap(adev->vce.vcpu_bo);
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
> index 99b82f3c2617..b42db22761b8 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
> @@ -27,6 +27,7 @@
>   #include <linux/firmware.h>
>   #include <linux/module.h>
>   #include <linux/pci.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "amdgpu_pm.h"
> @@ -267,7 +268,7 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev)
>   {
>   	unsigned size;
>   	void *ptr;
> -	int i;
> +	int i, idx;
>   
>   	cancel_delayed_work_sync(&adev->vcn.idle_work);
>   
> @@ -284,7 +285,10 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev)
>   		if (!adev->vcn.inst[i].saved_bo)
>   			return -ENOMEM;
>   
> -		memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size);
> +		if (drm_dev_enter(&adev->ddev, &idx)) {
> +			memcpy_fromio(adev->vcn.inst[i].saved_bo, ptr, size);
> +			drm_dev_exit(idx);
> +		}
>   	}
>   	return 0;
>   }
> @@ -293,7 +297,7 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev)
>   {
>   	unsigned size;
>   	void *ptr;
> -	int i;
> +	int i, idx;
>   
>   	for (i = 0; i < adev->vcn.num_vcn_inst; ++i) {
>   		if (adev->vcn.harvest_config & (1 << i))
> @@ -305,7 +309,10 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev)
>   		ptr = adev->vcn.inst[i].cpu_addr;
>   
>   		if (adev->vcn.inst[i].saved_bo != NULL) {
> -			memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size);
> +			if (drm_dev_enter(&adev->ddev, &idx)) {
> +				memcpy_toio(ptr, adev->vcn.inst[i].saved_bo, size);
> +				drm_dev_exit(idx);
> +			}
>   			kvfree(adev->vcn.inst[i].saved_bo);
>   			adev->vcn.inst[i].saved_bo = NULL;
>   		} else {
> @@ -315,8 +322,11 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev)
>   			hdr = (const struct common_firmware_header *)adev->vcn.fw->data;
>   			if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) {
>   				offset = le32_to_cpu(hdr->ucode_array_offset_bytes);
> -				memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset,
> -					    le32_to_cpu(hdr->ucode_size_bytes));
> +				if (drm_dev_enter(&adev->ddev, &idx)) {
> +					memcpy_toio(adev->vcn.inst[i].cpu_addr, adev->vcn.fw->data + offset,
> +						    le32_to_cpu(hdr->ucode_size_bytes));
> +					drm_dev_exit(idx);
> +				}
>   				size -= le32_to_cpu(hdr->ucode_size_bytes);
>   				ptr += le32_to_cpu(hdr->ucode_size_bytes);
>   			}
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index ae18c0e32347..7b622056df58 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -31,6 +31,7 @@
>   #include <linux/dma-buf.h>
>   
>   #include <drm/amdgpu_drm.h>
> +#include <drm/drm_drv.h>
>   #include "amdgpu.h"
>   #include "amdgpu_trace.h"
>   #include "amdgpu_amdkfd.h"
> @@ -1604,7 +1605,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
>   	struct amdgpu_vm_update_params params;
>   	enum amdgpu_sync_mode sync_mode;
>   	uint64_t pfn;
> -	int r;
> +	int r, idx;
> +
> +	if (!drm_dev_enter(&adev->ddev, &idx))
> +		return -ENODEV;
>   
>   	memset(&params, 0, sizeof(params));
>   	params.adev = adev;
> @@ -1713,6 +1717,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
>   
>   error_unlock:
>   	amdgpu_vm_eviction_unlock(vm);
> +	drm_dev_exit(idx);
>   	return r;
>   }
>   
> diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
> index c325d6f53a71..94cce172b98e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
> @@ -23,6 +23,7 @@
>   #include <linux/firmware.h>
>   #include <linux/module.h>
>   #include <linux/vmalloc.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "amdgpu_psp.h"
> @@ -269,10 +270,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy PSP KDB binary to memory */
> -	memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size);
> +	psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size);
>   
>   	/* Provide the PSP KDB to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> @@ -302,10 +301,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy PSP SPL binary to memory */
> -	memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size);
> +	psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size);
>   
>   	/* Provide the PSP SPL to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> @@ -335,10 +332,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy PSP System Driver binary to memory */
> -	memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
> +	psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
>   
>   	/* Provide the sys driver to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> @@ -371,10 +366,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy Secure OS binary to PSP memory */
> -	memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
> +	psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
>   
>   	/* Provide the PSP secure OS to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> @@ -608,7 +601,7 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops)
>   	uint32_t p2c_header[4];
>   	uint32_t sz;
>   	void *buf;
> -	int ret;
> +	int ret, idx;
>   
>   	if (ctx->init == PSP_MEM_TRAIN_NOT_SUPPORT) {
>   		DRM_DEBUG("Memory training is not supported.\n");
> @@ -681,17 +674,24 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops)
>   			return -ENOMEM;
>   		}
>   
> -		memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz);
> -		ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN);
> -		if (ret) {
> -			DRM_ERROR("Send long training msg failed.\n");
> +		if (drm_dev_enter(&adev->ddev, &idx)) {
> +			memcpy_fromio(buf, adev->mman.aper_base_kaddr, sz);
> +			ret = psp_v11_0_memory_training_send_msg(psp, PSP_BL__DRAM_LONG_TRAIN);
> +			if (ret) {
> +				DRM_ERROR("Send long training msg failed.\n");
> +				vfree(buf);
> +				drm_dev_exit(idx);
> +				return ret;
> +			}
> +
> +			memcpy_toio(adev->mman.aper_base_kaddr, buf, sz);
> +			adev->hdp.funcs->flush_hdp(adev, NULL);
>   			vfree(buf);
> -			return ret;
> +			drm_dev_exit(idx);
> +		} else {
> +			vfree(buf);
> +			return -ENODEV;
>   		}
> -
> -		memcpy_toio(adev->mman.aper_base_kaddr, buf, sz);
> -		adev->hdp.funcs->flush_hdp(adev, NULL);
> -		vfree(buf);
>   	}
>   
>   	if (ops & PSP_MEM_TRAIN_SAVE) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
> index c4828bd3264b..618e5b6b85d9 100644
> --- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
> @@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy PSP System Driver binary to memory */
> -	memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
> +	psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
>   
>   	/* Provide the sys driver to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> @@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy Secure OS binary to PSP memory */
> -	memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
> +	psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
>   
>   	/* Provide the PSP secure OS to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
> index f2e725f72d2f..d0a6cccd0897 100644
> --- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c
> @@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy PSP System Driver binary to memory */
> -	memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
> +	psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size);
>   
>   	/* Provide the sys driver to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> @@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp)
>   	if (ret)
>   		return ret;
>   
> -	memset(psp->fw_pri_buf, 0, PSP_1_MEG);
> -
>   	/* Copy Secure OS binary to PSP memory */
> -	memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
> +	psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size);
>   
>   	/* Provide the PSP secure OS to bootloader */
>   	WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36,
> diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
> index 37fa163393fd..fd859c778df0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
> @@ -25,6 +25,7 @@
>    */
>   
>   #include <linux/firmware.h>
> +#include <drm/drm_drv.h>
>   
>   #include "amdgpu.h"
>   #include "amdgpu_vce.h"
> @@ -555,16 +556,19 @@ static int vce_v4_0_hw_fini(void *handle)
>   static int vce_v4_0_suspend(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> -	int r;
> +	int r, idx;
>   
>   	if (adev->vce.vcpu_bo == NULL)
>   		return 0;
>   
> -	if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
> -		unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
> -		void *ptr = adev->vce.cpu_addr;
> +	if (drm_dev_enter(&adev->ddev, &idx)) {
> +		if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
> +			unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
> +			void *ptr = adev->vce.cpu_addr;
>   
> -		memcpy_fromio(adev->vce.saved_bo, ptr, size);
> +			memcpy_fromio(adev->vce.saved_bo, ptr, size);
> +		}
> +		drm_dev_exit(idx);
>   	}
>   
>   	r = vce_v4_0_hw_fini(adev);
> @@ -577,16 +581,20 @@ static int vce_v4_0_suspend(void *handle)
>   static int vce_v4_0_resume(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> -	int r;
> +	int r, idx;
>   
>   	if (adev->vce.vcpu_bo == NULL)
>   		return -EINVAL;
>   
>   	if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
> -		unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
> -		void *ptr = adev->vce.cpu_addr;
>   
> -		memcpy_toio(ptr, adev->vce.saved_bo, size);
> +		if (drm_dev_enter(&adev->ddev, &idx)) {
> +			unsigned size = amdgpu_bo_size(adev->vce.vcpu_bo);
> +			void *ptr = adev->vce.cpu_addr;
> +
> +			memcpy_toio(ptr, adev->vce.saved_bo, size);
> +			drm_dev_exit(idx);
> +		}
>   	} else {
>   		r = amdgpu_vce_resume(adev);
>   		if (r)
> diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
> index def583916294..5ac8ffe197aa 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
> @@ -34,6 +34,8 @@
>   #include "vcn/vcn_3_0_0_sh_mask.h"
>   #include "ivsrcid/vcn/irqsrcs_vcn_2_0.h"
>   
> +#include <drm/drm_drv.h>
> +
>   #define mmUVD_CONTEXT_ID_INTERNAL_OFFSET			0x27
>   #define mmUVD_GPCOM_VCPU_CMD_INTERNAL_OFFSET			0x0f
>   #define mmUVD_GPCOM_VCPU_DATA0_INTERNAL_OFFSET			0x10
> @@ -263,16 +265,20 @@ static int vcn_v3_0_sw_init(void *handle)
>   static int vcn_v3_0_sw_fini(void *handle)
>   {
>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> -	int i, r;
> +	int i, r, idx;
>   
> -	for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
> -		volatile struct amdgpu_fw_shared *fw_shared;
> +	if (drm_dev_enter(&adev->ddev, &idx)) {
> +		for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
> +			volatile struct amdgpu_fw_shared *fw_shared;
>   
> -		if (adev->vcn.harvest_config & (1 << i))
> -			continue;
> -		fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr;
> -		fw_shared->present_flag_0 = 0;
> -		fw_shared->sw_ring.is_enabled = false;
> +			if (adev->vcn.harvest_config & (1 << i))
> +				continue;
> +			fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr;
> +			fw_shared->present_flag_0 = 0;
> +			fw_shared->sw_ring.is_enabled = false;
> +		}
> +
> +		drm_dev_exit(idx);
>   	}
>   
>   	if (amdgpu_sriov_vf(adev))
> diff --git a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c
> index aae25243eb10..d628b91846c9 100644
> --- a/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c
> +++ b/drivers/gpu/drm/amd/pm/powerplay/smumgr/smu7_smumgr.c
> @@ -405,6 +405,8 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr)
>   				UCODE_ID_MEC_STORAGE, &toc->entry[toc->num_entries++]),
>   				"Failed to Get Firmware Entry.", r = -EINVAL; goto failed);
>   	}
> +
> +	/* AG TODO Can't call drm_dev_enter/exit because access adev->ddev here ... */
>   	memcpy_toio(smu_data->header_buffer.kaddr, smu_data->toc,
>   		    sizeof(struct SMU_DRAMData_TOC));
>   	smum_send_msg_to_smc_with_parameter(hwmgr,

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 13/27] drm/amdgpu: When filizing the fence driver. stop scheduler first.

[Christian König]

Filizing the fences? You mean finishing the fences, don't you? :)

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> No point calling amdgpu_fence_wait_empty before stopping the
> SW scheduler otherwise there is always a chance another job sneaked
> in after the wait.
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 9 +++++++--
>   1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> index 34d51e962799..fd9282637549 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> @@ -533,6 +533,12 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
>   
>   		if (!ring || !ring->fence_drv.initialized)
>   			continue;
> +
> +		/* Stop any new job submissions from sched before flushing the ring */
> +		/* TODO Handle amdgpu_job_submit_direct and amdgpu_amdkfd_submit_ib */
> +		if (!ring->no_scheduler)
> +			drm_sched_fini(&ring->sched);
> +
>   		r = amdgpu_fence_wait_empty(ring);
>   		if (r) {
>   			/* no need to trigger GPU reset as we are unloading */
> @@ -541,8 +547,7 @@ void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
>   		if (ring->fence_drv.irq_src)
>   			amdgpu_irq_put(adev, ring->fence_drv.irq_src,
>   				       ring->fence_drv.irq_type);
> -		if (!ring->no_scheduler)
> -			drm_sched_fini(&ring->sched);
> +
>   		del_timer_sync(&ring->fence_drv.fallback_timer);
>   	}
>   }

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Christian König]

I need to take another look at this part when I don't have a massive 
headache any more.

Maybe split the patch set up into different parts, something like:
1. Adding general infrastructure.
2. Making sure all memory is unpolated.
3. Job and fence handling

Christian.

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> Problem: If scheduler is already stopped by the time sched_entity
> is released and entity's job_queue not empty I encountred
> a hang in drm_sched_entity_flush. This is because drm_sched_entity_is_idle
> never becomes false.
>
> Fix: In drm_sched_fini detach all sched_entities from the
> scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
> Also wakeup all those processes stuck in sched_entity flushing
> as the scheduler main thread which wakes them up is stopped by now.
>
> v2:
> Reverse order of drm_sched_rq_remove_entity and marking
> s_entity as stopped to prevent reinserion back to rq due
> to race.
>
> v3:
> Drop drm_sched_rq_remove_entity, only modify entity->stopped
> and check for it in drm_sched_entity_is_idle
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> Reviewed-by: Christian König <christian.koenig@amd.com>
> ---
>   drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
>   drivers/gpu/drm/scheduler/sched_main.c   | 24 ++++++++++++++++++++++++
>   2 files changed, 26 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c
> index f0790e9471d1..cb58f692dad9 100644
> --- a/drivers/gpu/drm/scheduler/sched_entity.c
> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct drm_sched_entity *entity)
>   	rmb(); /* for list_empty to work without lock */
>   
>   	if (list_empty(&entity->list) ||
> -	    spsc_queue_count(&entity->job_queue) == 0)
> +	    spsc_queue_count(&entity->job_queue) == 0 ||
> +	    entity->stopped)
>   		return true;
>   
>   	return false;
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 908b0b56032d..ba087354d0a8 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
>    */
>   void drm_sched_fini(struct drm_gpu_scheduler *sched)
>   {
> +	struct drm_sched_entity *s_entity;
> +	int i;
> +
>   	if (sched->thread)
>   		kthread_stop(sched->thread);
>   
> +	for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {
> +		struct drm_sched_rq *rq = &sched->sched_rq[i];
> +
> +		if (!rq)
> +			continue;
> +
> +		spin_lock(&rq->lock);
> +		list_for_each_entry(s_entity, &rq->entities, list)
> +			/*
> +			 * Prevents reinsertion and marks job_queue as idle,
> +			 * it will removed from rq in drm_sched_entity_fini
> +			 * eventually
> +			 */
> +			s_entity->stopped = true;
> +		spin_unlock(&rq->lock);
> +
> +	}
> +
> +	/* Wakeup everyone stuck in drm_sched_entity_flush for this scheduler */
> +	wake_up_all(&sched->job_scheduled);
> +
>   	/* Confirm no work left behind accessing device structures */
>   	cancel_delayed_work_sync(&sched->work_tdr);
>   

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 16/27] drm/amdgpu: Unmap all MMIO mappings

[Christian König]

Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> Access to those must be prevented post pci_remove

That is certainly a no-go. We want to get rid of the kernel pointers in 
BOs, not add another one.

Christian.

>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  5 +++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 38 ++++++++++++++++++++--
>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 28 ++++++++++++++--
>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  5 +++
>   4 files changed, 71 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 30a24db5f4d1..3e4755fc10c8 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1056,6 +1056,11 @@ struct amdgpu_device {
>   	struct pci_saved_state          *pci_state;
>   
>   	struct list_head                device_bo_list;
> +
> +	/* List of all MMIO BOs */
> +	struct list_head                mmio_list;
> +	struct mutex                    mmio_list_lock;
> +
>   };
>   
>   static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 22b09c4db255..3ddad6cba62d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -3320,6 +3320,9 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>   	INIT_LIST_HEAD(&adev->shadow_list);
>   	mutex_init(&adev->shadow_list_lock);
>   
> +	INIT_LIST_HEAD(&adev->mmio_list);
> +	mutex_init(&adev->mmio_list_lock);
> +
>   	INIT_DELAYED_WORK(&adev->delayed_init_work,
>   			  amdgpu_device_delayed_init_work_handler);
>   	INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work,
> @@ -3636,6 +3639,36 @@ static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
>   	spin_unlock(&adev->mman.bdev.lru_lock);
>   }
>   
> +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
> +{
> +	struct amdgpu_bo *bo;
> +
> +	/* Clear all CPU mappings pointing to this device */
> +	unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1);
> +
> +	/* Unmap all MMIO mapped kernel BOs */
> +	mutex_lock(&adev->mmio_list_lock);
> +	list_for_each_entry(bo, &adev->mmio_list, mmio_list) {
> +		amdgpu_bo_kunmap(bo);
> +		if (*bo->kmap_ptr)
> +			*bo->kmap_ptr = NULL;
> +	}
> +	mutex_unlock(&adev->mmio_list_lock);
> +
> +	/* Unmap all mapped bars - Doorbell, registers and VRAM */
> +	amdgpu_device_doorbell_fini(adev);
> +
> +	iounmap(adev->rmmio);
> +	adev->rmmio = NULL;
> +	if (adev->mman.aper_base_kaddr)
> +		iounmap(adev->mman.aper_base_kaddr);
> +	adev->mman.aper_base_kaddr = NULL;
> +
> +	/* Memory manager related */
> +	arch_phys_wc_del(adev->gmc.vram_mtrr);
> +	arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size);
> +}
> +
>   /**
>    * amdgpu_device_fini - tear down the driver
>    *
> @@ -3683,6 +3716,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>   	amdgpu_clear_dma_mappings(adev);
>   
>   	amdgpu_gart_dummy_page_fini(adev);
> +
> +	amdgpu_device_unmap_mmio(adev);
>   }
>   
>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> @@ -3713,9 +3748,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>   	if (adev->rio_mem)
>   		pci_iounmap(adev->pdev, adev->rio_mem);
>   	adev->rio_mem = NULL;
> -	iounmap(adev->rmmio);
> -	adev->rmmio = NULL;
> -	amdgpu_device_doorbell_fini(adev);
>   
>   	if (IS_ENABLED(CONFIG_PERF_EVENTS))
>   		amdgpu_pmu_fini(adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> index 62d829f5e62c..9b05e3b96fa0 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> @@ -531,6 +531,9 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
>   		return -ENOMEM;
>   	drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size);
>   	INIT_LIST_HEAD(&bo->shadow_list);
> +
> +	INIT_LIST_HEAD(&bo->mmio_list);
> +
>   	bo->vm_bo = NULL;
>   	bo->preferred_domains = bp->preferred_domain ? bp->preferred_domain :
>   		bp->domain;
> @@ -774,9 +777,21 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr)
>   	if (r)
>   		return r;
>   
> -	if (ptr)
> +	if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {
> +		struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
> +
> +		mutex_lock(&adev->mmio_list_lock);
> +		list_add_tail(&bo->mmio_list, &adev->mmio_list);
> +		mutex_unlock(&adev->mmio_list_lock);
> +	}
> +
> +	if (ptr) {
>   		*ptr = amdgpu_bo_kptr(bo);
>   
> +		if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap)
> +			bo->kmap_ptr = ptr;
> +	}
> +
>   	return 0;
>   }
>   
> @@ -804,8 +819,17 @@ void *amdgpu_bo_kptr(struct amdgpu_bo *bo)
>    */
>   void amdgpu_bo_kunmap(struct amdgpu_bo *bo)
>   {
> -	if (bo->kmap.bo)
> +	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
> +
> +	if (bo->kmap.bo) {
> +		if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {
> +			mutex_lock(&adev->mmio_list_lock);
> +			list_del_init(&bo->mmio_list);
> +			mutex_unlock(&adev->mmio_list_lock);
> +		}
> +
>   		ttm_bo_kunmap(&bo->kmap);
> +	}
>   }
>   
>   /**
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> index 5ae8555ef275..3129d9bbfa22 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> @@ -112,6 +112,11 @@ struct amdgpu_bo {
>   	struct kgd_mem                  *kfd_bo;
>   
>   	struct list_head		bo;
> +
> +	struct list_head                mmio_list;
> +	/* Address of kernel VA pointer to MMIO so they can be updated post remap */
> +	void				**kmap_ptr;
> +
>   };
>   
>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> With this calling drm_dev_unplug will flush and block
> all in flight IOCTLs
> 
> Also, add feature such that if device supports graceful unplug
> we enclose entire IOCTL in SRCU critical section.
> 
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>

Nope.

The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.

Especially not with an opt-in flag so that it could be shrugged of as a
driver hack. Most of these ioctls should have absolutely no problem
working after hotunplug.

Also, doing this defeats the point since it pretty much guarantees
userspace will die in assert()s and stuff. E.g. on i915 the rough contract
is that only execbuf (and even that only when userspace has indicated
support for non-recoverable hw ctx) is allowed to fail. Anything else
might crash userspace.

You probably need similar (and very precisely defined) rules for amdgpu.
And those must definitely exclude any shard ioctls from randomly failing
with EIO, because that just kills the box and defeats the point of trying
to gracefully handling hotunplug and making sure userspace has a chance of
survival. E.g. for atomic everything should continue, including flip
completion, but we set all outputs to "disconnected" and send out the
uevent. Maybe crtc enabling can fail too, but that can also be handled
through the async status we're using to signal DP link failures to
userspace.

I guess we should clarify this in the hotunplug doc?

Cheers, Daniel

> ---
>  drivers/gpu/drm/drm_ioctl.c | 15 +++++++++++++--
>  include/drm/drm_drv.h       |  6 ++++++
>  2 files changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
> index d273d1a8603a..5882ef2183bb 100644
> --- a/drivers/gpu/drm/drm_ioctl.c
> +++ b/drivers/gpu/drm/drm_ioctl.c
> @@ -815,7 +815,7 @@ long drm_ioctl(struct file *filp,
>  	const struct drm_ioctl_desc *ioctl = NULL;
>  	drm_ioctl_t *func;
>  	unsigned int nr = DRM_IOCTL_NR(cmd);
> -	int retcode = -EINVAL;
> +	int idx, retcode = -EINVAL;
>  	char stack_kdata[128];
>  	char *kdata = NULL;
>  	unsigned int in_size, out_size, drv_size, ksize;
> @@ -884,7 +884,18 @@ long drm_ioctl(struct file *filp,
>  	if (ksize > in_size)
>  		memset(kdata + in_size, 0, ksize - in_size);
>  
> -	retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);
> +	if (drm_core_check_feature(dev, DRIVER_HOTUNPLUG_SUPPORT)) {
> +		if (drm_dev_enter(dev, &idx)) {
> +			retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);
> +			drm_dev_exit(idx);
> +		} else {
> +			retcode = -ENODEV;
> +			goto err_i1;
> +		}
> +	} else {
> +		retcode = drm_ioctl_kernel(filp, func, kdata, ioctl->flags);
> +	}
> +
>  	if (copy_to_user((void __user *)arg, kdata, out_size) != 0)
>  		retcode = -EFAULT;
>  
> diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h
> index b439ae1921b8..63e05cec46c1 100644
> --- a/include/drm/drm_drv.h
> +++ b/include/drm/drm_drv.h
> @@ -94,6 +94,12 @@ enum drm_driver_feature {
>  	 * synchronization of command submission.
>  	 */
>  	DRIVER_SYNCOBJ_TIMELINE         = BIT(6),
> +	/**
> +	 * @DRIVER_NO_HOTUNPLUG_SUPPORT:
> +	 *
> +	 * Driver support gracefull remove.
> +	 */
> +	DRIVER_HOTUNPLUG_SUPPORT         = BIT(7),
>  
>  	/* IMPORTANT: Below are all the legacy flags, add new ones above. */
>  
> -- 
> 2.25.1
> 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> > With this calling drm_dev_unplug will flush and block
> > all in flight IOCTLs
> > 
> > Also, add feature such that if device supports graceful unplug
> > we enclose entire IOCTL in SRCU critical section.
> > 
> > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> 
> Nope.
> 
> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
> 
> Especially not with an opt-in flag so that it could be shrugged of as a
> driver hack. Most of these ioctls should have absolutely no problem
> working after hotunplug.
> 
> Also, doing this defeats the point since it pretty much guarantees
> userspace will die in assert()s and stuff. E.g. on i915 the rough contract
> is that only execbuf (and even that only when userspace has indicated
> support for non-recoverable hw ctx) is allowed to fail. Anything else
> might crash userspace.
> 
> You probably need similar (and very precisely defined) rules for amdgpu.
> And those must definitely exclude any shard ioctls from randomly failing
> with EIO, because that just kills the box and defeats the point of trying
> to gracefully handling hotunplug and making sure userspace has a chance of
> survival. E.g. for atomic everything should continue, including flip
> completion, but we set all outputs to "disconnected" and send out the
> uevent. Maybe crtc enabling can fail too, but that can also be handled
> through the async status we're using to signal DP link failures to
> userspace.
> 
> I guess we should clarify this in the hotunplug doc?

To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
really make no sense, and where we're rather confident that all properly
implemented userspace will gracefully handle failures. Like a modeset, or
opening a device, or trying to import a dma-buf or stuff like that which
can already fail in normal operation for any kind of reason.

But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
after hotunplug.

And then there's the middle ground, like doing a pageflip or buffer flush,
which I guess some userspace might handle, but risky to inflict those
consequences on them. atomic modeset is especially fun since depending
what you're doing it can be both "failures expected" and "failures not
really expected in normal operation".

Also, this really should be consistent across drivers, not solved with a
driver flag for every possible combination.

If you look at the current hotunplug kms drivers, they have
drm_dev_enter/exit sprinkled in specific hw callback functions because of
the above problems. But maybe it makes sense to change things in a few
cases. But then we should do it across the board.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
>>> With this calling drm_dev_unplug will flush and block
>>> all in flight IOCTLs
>>>
>>> Also, add feature such that if device supports graceful unplug
>>> we enclose entire IOCTL in SRCU critical section.
>>>
>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>
>> Nope.
>>
>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.

Then I am confused why we have 
https://elixir.bootlin.com/linux/v5.12/source/drivers/gpu/drm/drm_ioctl.c#L826
currently in code ?

>>
>> Especially not with an opt-in flag so that it could be shrugged of as a
>> driver hack. Most of these ioctls should have absolutely no problem
>> working after hotunplug.
>>
>> Also, doing this defeats the point since it pretty much guarantees
>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract
>> is that only execbuf (and even that only when userspace has indicated
>> support for non-recoverable hw ctx) is allowed to fail. Anything else
>> might crash userspace.

Given that as I pointed above we already fail any IOCTls with -ENODEV
when device is unplugged, it seems those crashes don't happen that
often ? Also, in all my testing I don't think I saw a user space crash
I could attribute to this.

>>
>> You probably need similar (and very precisely defined) rules for amdgpu.
>> And those must definitely exclude any shard ioctls from randomly failing
>> with EIO, because that just kills the box and defeats the point of trying
>> to gracefully handling hotunplug and making sure userspace has a chance of
>> survival. E.g. for atomic everything should continue, including flip
>> completion, but we set all outputs to "disconnected" and send out the
>> uevent. Maybe crtc enabling can fail too, but that can also be handled
>> through the async status we're using to signal DP link failures to
>> userspace.

As I pointed before, because of the complexity of the topic I prefer to
take it step by step and solve first for secondary device use case, not
for primary, display attached device.

>>
>> I guess we should clarify this in the hotunplug doc?

Agree

> 
> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
> really make no sense, and where we're rather confident that all properly
> implemented userspace will gracefully handle failures. Like a modeset, or
> opening a device, or trying to import a dma-buf or stuff like that which
> can already fail in normal operation for any kind of reason.
> 
> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
> after hotunplug.

As I pointed above, this a bit confuses me given that we already do
blanker rejection of IOCTLs if device is unplugged.


> 
> And then there's the middle ground, like doing a pageflip or buffer flush,
> which I guess some userspace might handle, but risky to inflict those
> consequences on them. atomic modeset is especially fun since depending
> what you're doing it can be both "failures expected" and "failures not
> really expected in normal operation".
> 
> Also, this really should be consistent across drivers, not solved with a
> driver flag for every possible combination.
> 
> If you look at the current hotunplug kms drivers, they have
> drm_dev_enter/exit sprinkled in specific hw callback functions because of
> the above problems. But maybe it makes sense to change things in a few
> cases. But then we should do it across the board.

So as I understand your preferred approach is that I scope any back_end, 
HW specific function with drm_dev_enter/exit because that where MMIO
access takes place. But besides explicit MMIO access thorough
register accessors in the HW back-end there is also indirect MMIO access
taking place throughout the code in the driver because of various VRAM
BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
access is spread all over in the driver and even in mid-layers such as
TTM and not limited to HW back-end functions. It means it's much harder
to spot such places to surgically scope them with drm_dev_enter/exit and
also that any new such code introduced will immediately break hot unplug
because the developers can't be expected to remember making their code
robust to this specific use case. That why when we discussed internally
what approach to take to protecting code with drm_dev_enter/exit we
opted for using the widest available scope.

Andrey

> 
> Cheers, Daniel
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Felix Kuehling]

Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
> So as I understand your preferred approach is that I scope any
> back_end, HW specific function with drm_dev_enter/exit because that
> where MMIO
> access takes place. But besides explicit MMIO access thorough
> register accessors in the HW back-end there is also indirect MMIO access
> taking place throughout the code in the driver because of various VRAM
> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
> access is spread all over in the driver and even in mid-layers such as
> TTM and not limited to HW back-end functions. It means it's much harder
> to spot such places to surgically scope them with drm_dev_enter/exit and
> also that any new such code introduced will immediately break hot unplug
> because the developers can't be expected to remember making their code
> robust to this specific use case. That why when we discussed internally
> what approach to take to protecting code with drm_dev_enter/exit we
> opted for using the widest available scope.

VRAM can also be mapped in user mode. Is there anything preventing user
mode from accessing the memory after unplug? I guess the best you could
do is unmap it from the CPU page table and let the application segfault
on the next access. Or replace the mapping with a dummy page in system
memory?

Regards,
  Felix


>
> Andrey 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-04-29 12:15 p.m., Felix Kuehling wrote:
> Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
>> So as I understand your preferred approach is that I scope any
>> back_end, HW specific function with drm_dev_enter/exit because that
>> where MMIO
>> access takes place. But besides explicit MMIO access thorough
>> register accessors in the HW back-end there is also indirect MMIO access
>> taking place throughout the code in the driver because of various VRAM
>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>> access is spread all over in the driver and even in mid-layers such as
>> TTM and not limited to HW back-end functions. It means it's much harder
>> to spot such places to surgically scope them with drm_dev_enter/exit and
>> also that any new such code introduced will immediately break hot unplug
>> because the developers can't be expected to remember making their code
>> robust to this specific use case. That why when we discussed internally
>> what approach to take to protecting code with drm_dev_enter/exit we
>> opted for using the widest available scope.
> 
> VRAM can also be mapped in user mode. Is there anything preventing user
> mode from accessing the memory after unplug? I guess the best you could
> do is unmap it from the CPU page table and let the application segfault
> on the next access. Or replace the mapping with a dummy page in system
> memory?

We indeed unmap but instead of letting it segfault insert dummy page on
the next page fault. See here 
https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=6dde3330ffa450e2e6da4d19e2fd0bb94b66b6ce
And I am aware that this doesn't take care of KFD user mapping.
As you know, we had some discussions with you on this topic and it's on
my TODO list to follow up on this to solve it for KFD too.

Andrey

> 
> Regards,
>    Felix
> 
> 
>>
>> Andrey
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Felix Kuehling]

Am 2021-04-29 um 12:21 p.m. schrieb Andrey Grodzovsky:
>
>
> On 2021-04-29 12:15 p.m., Felix Kuehling wrote:
>> Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
>>> So as I understand your preferred approach is that I scope any
>>> back_end, HW specific function with drm_dev_enter/exit because that
>>> where MMIO
>>> access takes place. But besides explicit MMIO access thorough
>>> register accessors in the HW back-end there is also indirect MMIO
>>> access
>>> taking place throughout the code in the driver because of various VRAM
>>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>>> access is spread all over in the driver and even in mid-layers such as
>>> TTM and not limited to HW back-end functions. It means it's much harder
>>> to spot such places to surgically scope them with drm_dev_enter/exit
>>> and
>>> also that any new such code introduced will immediately break hot
>>> unplug
>>> because the developers can't be expected to remember making their code
>>> robust to this specific use case. That why when we discussed internally
>>> what approach to take to protecting code with drm_dev_enter/exit we
>>> opted for using the widest available scope.
>>
>> VRAM can also be mapped in user mode. Is there anything preventing user
>> mode from accessing the memory after unplug? I guess the best you could
>> do is unmap it from the CPU page table and let the application segfault
>> on the next access. Or replace the mapping with a dummy page in system
>> memory?
>
> We indeed unmap but instead of letting it segfault insert dummy page on
> the next page fault. See here
> https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=6dde3330ffa450e2e6da4d19e2fd0bb94b66b6ce
> And I am aware that this doesn't take care of KFD user mapping.
> As you know, we had some discussions with you on this topic and it's on
> my TODO list to follow up on this to solve it for KFD too.

ROCm user mode maps VRAM BOs using render nodes. So I'd expect
ttm_bo_vm_dummy_page to work for KFD as well.

I guess we'd need something special for KFD's doorbell and MMIO (HDP
flush) mappings. Was that the discussion about the file address space?

Regards,
  Felix


>
> Andrey
>
>>
>> Regards,
>>    Felix
>>
>>
>>>
>>> Andrey
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-04-29 12:29 p.m., Felix Kuehling wrote:
> Am 2021-04-29 um 12:21 p.m. schrieb Andrey Grodzovsky:
>>
>>
>> On 2021-04-29 12:15 p.m., Felix Kuehling wrote:
>>> Am 2021-04-29 um 12:04 p.m. schrieb Andrey Grodzovsky:
>>>> So as I understand your preferred approach is that I scope any
>>>> back_end, HW specific function with drm_dev_enter/exit because that
>>>> where MMIO
>>>> access takes place. But besides explicit MMIO access thorough
>>>> register accessors in the HW back-end there is also indirect MMIO
>>>> access
>>>> taking place throughout the code in the driver because of various VRAM
>>>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>>>> access is spread all over in the driver and even in mid-layers such as
>>>> TTM and not limited to HW back-end functions. It means it's much harder
>>>> to spot such places to surgically scope them with drm_dev_enter/exit
>>>> and
>>>> also that any new such code introduced will immediately break hot
>>>> unplug
>>>> because the developers can't be expected to remember making their code
>>>> robust to this specific use case. That why when we discussed internally
>>>> what approach to take to protecting code with drm_dev_enter/exit we
>>>> opted for using the widest available scope.
>>>
>>> VRAM can also be mapped in user mode. Is there anything preventing user
>>> mode from accessing the memory after unplug? I guess the best you could
>>> do is unmap it from the CPU page table and let the application segfault
>>> on the next access. Or replace the mapping with a dummy page in system
>>> memory?
>>
>> We indeed unmap but instead of letting it segfault insert dummy page on
>> the next page fault. See here
>> https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=6dde3330ffa450e2e6da4d19e2fd0bb94b66b6ce
>> And I am aware that this doesn't take care of KFD user mapping.
>> As you know, we had some discussions with you on this topic and it's on
>> my TODO list to follow up on this to solve it for KFD too.
> 
> ROCm user mode maps VRAM BOs using render nodes. So I'd expect
> ttm_bo_vm_dummy_page to work for KFD as well.
> 
> I guess we'd need something special for KFD's doorbell and MMIO (HDP
> flush) mappings. Was that the discussion about the file address space?

Yes

Andrey

> 
> Regards,
>    Felix
> 
> 
>>
>> Andrey
>>
>>>
>>> Regards,
>>>     Felix
>>>
>>>
>>>>
>>>> Andrey
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 08/27] PCI: add support for dev_groups to struct pci_device_driver

[Andrey Grodzovsky]

On 2021-04-28 12:53 p.m., Bjorn Helgaas wrote:
> In subject:
> 
> s/PCI: add support/PCI: Add support/
> 
> to match convention ("git log --oneline drivers/pci/pci-driver.c" to
> learn this).
> 
> On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
>> This is exact copy of 'USB: add support for dev_groups to
>> struct usb_device_driver' patch by Greg but just for
>> the PCI case.
> 
> Ideally this would be an imperative sentence telling us what the patch
> *does*, not just that it's a copy of something else.
> 
> I'd also like a brief comment about why this is useful, i.e., why you
> need this and what you're going to use it for.
> 
> The usual commit citation format is 7d9c1d2f7aca ("USB: add support
> for dev_groups to struct usb_device_driver") so it's easier to locate
> the commit.
> 
> I see there is also b71b283e3d6d ("USB: add support for dev_groups to
> struct usb_driver").  I don't know enough about USB to know whether
> 7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing
> here, but I do see that struct usb_driver is far more common than
> struct usb_device_driver.
> 
> PCI has struct pci_driver, but doesn't have the concept of a struct
> pci_device_driver.

Since we don't have pci_device_driver then pci_driver is the best place
for it then, no ?

Andrey

> 
>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>> Suggested-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>> ---
>>   drivers/pci/pci-driver.c | 1 +
>>   include/linux/pci.h      | 3 +++
>>   2 files changed, 4 insertions(+)
>>
>> diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c
>> index ec44a79e951a..3a72352aa5cf 100644
>> --- a/drivers/pci/pci-driver.c
>> +++ b/drivers/pci/pci-driver.c
>> @@ -1385,6 +1385,7 @@ int __pci_register_driver(struct pci_driver *drv, struct module *owner,
>>   	drv->driver.owner = owner;
>>   	drv->driver.mod_name = mod_name;
>>   	drv->driver.groups = drv->groups;
>> +	drv->driver.dev_groups = drv->dev_groups;
>>   
>>   	spin_lock_init(&drv->dynids.lock);
>>   	INIT_LIST_HEAD(&drv->dynids.list);
>> diff --git a/include/linux/pci.h b/include/linux/pci.h
>> index 86c799c97b77..b57755b03009 100644
>> --- a/include/linux/pci.h
>> +++ b/include/linux/pci.h
>> @@ -858,6 +858,8 @@ struct module;
>>    *		number of VFs to enable via sysfs "sriov_numvfs" file.
>>    * @err_handler: See Documentation/PCI/pci-error-recovery.rst
>>    * @groups:	Sysfs attribute groups.
>> + * @dev_groups: Attributes attached to the device that will be
>> + *              created once it is bound to the driver.
>>    * @driver:	Driver model structure.
>>    * @dynids:	List of dynamically added device IDs.
>>    */
>> @@ -873,6 +875,7 @@ struct pci_driver {
>>   	int  (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */
>>   	const struct pci_error_handlers *err_handler;
>>   	const struct attribute_group **groups;
>> +	const struct attribute_group **dev_groups;
>>   	struct device_driver	driver;
>>   	struct pci_dynids	dynids;
>>   };
>> -- 
>> 2.25.1
>>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 16/27] drm/amdgpu: Unmap all MMIO mappings

[Andrey Grodzovsky]

On 2021-04-29 3:19 a.m., Christian König wrote:
> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>> Access to those must be prevented post pci_remove
> 
> That is certainly a no-go. We want to get rid of the kernel pointers in 
> BOs, not add another one.
> 
> Christian.

As we discussed internally, will drop the entire explicit BOs unmapping
approach as unmapping the VRAM bar alone will give the same results.

Andrey

> 
>>
>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  5 +++
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 38 ++++++++++++++++++++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 28 ++++++++++++++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  5 +++
>>   4 files changed, 71 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index 30a24db5f4d1..3e4755fc10c8 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1056,6 +1056,11 @@ struct amdgpu_device {
>>       struct pci_saved_state          *pci_state;
>>       struct list_head                device_bo_list;
>> +
>> +    /* List of all MMIO BOs */
>> +    struct list_head                mmio_list;
>> +    struct mutex                    mmio_list_lock;
>> +
>>   };
>>   static inline struct amdgpu_device *drm_to_adev(struct drm_device 
>> *ddev)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 22b09c4db255..3ddad6cba62d 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -3320,6 +3320,9 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>>       INIT_LIST_HEAD(&adev->shadow_list);
>>       mutex_init(&adev->shadow_list_lock);
>> +    INIT_LIST_HEAD(&adev->mmio_list);
>> +    mutex_init(&adev->mmio_list_lock);
>> +
>>       INIT_DELAYED_WORK(&adev->delayed_init_work,
>>                 amdgpu_device_delayed_init_work_handler);
>>       INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work,
>> @@ -3636,6 +3639,36 @@ static void amdgpu_clear_dma_mappings(struct 
>> amdgpu_device *adev)
>>       spin_unlock(&adev->mman.bdev.lru_lock);
>>   }
>> +static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev)
>> +{
>> +    struct amdgpu_bo *bo;
>> +
>> +    /* Clear all CPU mappings pointing to this device */
>> +    unmap_mapping_range(adev->ddev.anon_inode->i_mapping, 0, 0, 1);
>> +
>> +    /* Unmap all MMIO mapped kernel BOs */
>> +    mutex_lock(&adev->mmio_list_lock);
>> +    list_for_each_entry(bo, &adev->mmio_list, mmio_list) {
>> +        amdgpu_bo_kunmap(bo);
>> +        if (*bo->kmap_ptr)
>> +            *bo->kmap_ptr = NULL;
>> +    }
>> +    mutex_unlock(&adev->mmio_list_lock);
>> +
>> +    /* Unmap all mapped bars - Doorbell, registers and VRAM */
>> +    amdgpu_device_doorbell_fini(adev);
>> +
>> +    iounmap(adev->rmmio);
>> +    adev->rmmio = NULL;
>> +    if (adev->mman.aper_base_kaddr)
>> +        iounmap(adev->mman.aper_base_kaddr);
>> +    adev->mman.aper_base_kaddr = NULL;
>> +
>> +    /* Memory manager related */
>> +    arch_phys_wc_del(adev->gmc.vram_mtrr);
>> +    arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size);
>> +}
>> +
>>   /**
>>    * amdgpu_device_fini - tear down the driver
>>    *
>> @@ -3683,6 +3716,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device 
>> *adev)
>>       amdgpu_clear_dma_mappings(adev);
>>       amdgpu_gart_dummy_page_fini(adev);
>> +
>> +    amdgpu_device_unmap_mmio(adev);
>>   }
>>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>> @@ -3713,9 +3748,6 @@ void amdgpu_device_fini_sw(struct amdgpu_device 
>> *adev)
>>       if (adev->rio_mem)
>>           pci_iounmap(adev->pdev, adev->rio_mem);
>>       adev->rio_mem = NULL;
>> -    iounmap(adev->rmmio);
>> -    adev->rmmio = NULL;
>> -    amdgpu_device_doorbell_fini(adev);
>>       if (IS_ENABLED(CONFIG_PERF_EVENTS))
>>           amdgpu_pmu_fini(adev);
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> index 62d829f5e62c..9b05e3b96fa0 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> @@ -531,6 +531,9 @@ static int amdgpu_bo_do_create(struct 
>> amdgpu_device *adev,
>>           return -ENOMEM;
>>       drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, 
>> size);
>>       INIT_LIST_HEAD(&bo->shadow_list);
>> +
>> +    INIT_LIST_HEAD(&bo->mmio_list);
>> +
>>       bo->vm_bo = NULL;
>>       bo->preferred_domains = bp->preferred_domain ? 
>> bp->preferred_domain :
>>           bp->domain;
>> @@ -774,9 +777,21 @@ int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr)
>>       if (r)
>>           return r;
>> -    if (ptr)
>> +    if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {
>> +        struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
>> +
>> +        mutex_lock(&adev->mmio_list_lock);
>> +        list_add_tail(&bo->mmio_list, &adev->mmio_list);
>> +        mutex_unlock(&adev->mmio_list_lock);
>> +    }
>> +
>> +    if (ptr) {
>>           *ptr = amdgpu_bo_kptr(bo);
>> +        if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap)
>> +            bo->kmap_ptr = ptr;
>> +    }
>> +
>>       return 0;
>>   }
>> @@ -804,8 +819,17 @@ void *amdgpu_bo_kptr(struct amdgpu_bo *bo)
>>    */
>>   void amdgpu_bo_kunmap(struct amdgpu_bo *bo)
>>   {
>> -    if (bo->kmap.bo)
>> +    struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
>> +
>> +    if (bo->kmap.bo) {
>> +        if (bo->kmap.bo_kmap_type == ttm_bo_map_iomap) {
>> +            mutex_lock(&adev->mmio_list_lock);
>> +            list_del_init(&bo->mmio_list);
>> +            mutex_unlock(&adev->mmio_list_lock);
>> +        }
>> +
>>           ttm_bo_kunmap(&bo->kmap);
>> +    }
>>   }
>>   /**
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> index 5ae8555ef275..3129d9bbfa22 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> @@ -112,6 +112,11 @@ struct amdgpu_bo {
>>       struct kgd_mem                  *kfd_bo;
>>       struct list_head        bo;
>> +
>> +    struct list_head                mmio_list;
>> +    /* Address of kernel VA pointer to MMIO so they can be updated 
>> post remap */
>> +    void                **kmap_ptr;
>> +
>>   };
>>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct 
>> ttm_buffer_object *tbo)
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Andrey Grodzovsky]

On 2021-04-29 3:18 a.m., Christian König wrote:
> I need to take another look at this part when I don't have a massive 
> headache any more.
> 
> Maybe split the patch set up into different parts, something like:
> 1. Adding general infrastructure.
> 2. Making sure all memory is unpolated.
> 3. Job and fence handling

I am not sure you mean this patch here, maybe another one ?
Also note you already RBed it.

Andrey

> 
> Christian.
> 
> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>> Problem: If scheduler is already stopped by the time sched_entity
>> is released and entity's job_queue not empty I encountred
>> a hang in drm_sched_entity_flush. This is because 
>> drm_sched_entity_is_idle
>> never becomes false.
>>
>> Fix: In drm_sched_fini detach all sched_entities from the
>> scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
>> Also wakeup all those processes stuck in sched_entity flushing
>> as the scheduler main thread which wakes them up is stopped by now.
>>
>> v2:
>> Reverse order of drm_sched_rq_remove_entity and marking
>> s_entity as stopped to prevent reinserion back to rq due
>> to race.
>>
>> v3:
>> Drop drm_sched_rq_remove_entity, only modify entity->stopped
>> and check for it in drm_sched_entity_is_idle
>>
>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>> Reviewed-by: Christian König <christian.koenig@amd.com>
>> ---
>>   drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
>>   drivers/gpu/drm/scheduler/sched_main.c   | 24 ++++++++++++++++++++++++
>>   2 files changed, 26 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c 
>> b/drivers/gpu/drm/scheduler/sched_entity.c
>> index f0790e9471d1..cb58f692dad9 100644
>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
>> @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct 
>> drm_sched_entity *entity)
>>       rmb(); /* for list_empty to work without lock */
>>       if (list_empty(&entity->list) ||
>> -        spsc_queue_count(&entity->job_queue) == 0)
>> +        spsc_queue_count(&entity->job_queue) == 0 ||
>> +        entity->stopped)
>>           return true;
>>       return false;
>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c 
>> b/drivers/gpu/drm/scheduler/sched_main.c
>> index 908b0b56032d..ba087354d0a8 100644
>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>> @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
>>    */
>>   void drm_sched_fini(struct drm_gpu_scheduler *sched)
>>   {
>> +    struct drm_sched_entity *s_entity;
>> +    int i;
>> +
>>       if (sched->thread)
>>           kthread_stop(sched->thread);
>> +    for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= 
>> DRM_SCHED_PRIORITY_MIN; i--) {
>> +        struct drm_sched_rq *rq = &sched->sched_rq[i];
>> +
>> +        if (!rq)
>> +            continue;
>> +
>> +        spin_lock(&rq->lock);
>> +        list_for_each_entry(s_entity, &rq->entities, list)
>> +            /*
>> +             * Prevents reinsertion and marks job_queue as idle,
>> +             * it will removed from rq in drm_sched_entity_fini
>> +             * eventually
>> +             */
>> +            s_entity->stopped = true;
>> +        spin_unlock(&rq->lock);
>> +
>> +    }
>> +
>> +    /* Wakeup everyone stuck in drm_sched_entity_flush for this 
>> scheduler */
>> +    wake_up_all(&sched->job_scheduled);
>> +
>>       /* Confirm no work left behind accessing device structures */
>>       cancel_delayed_work_sync(&sched->work_tdr);
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 13/27] drm/amdgpu: When filizing the fence driver. stop scheduler first.

[Andrey Grodzovsky]

On 2021-04-29 3:15 a.m., Christian König wrote:
> 
> Filizing the fences? You mean finishing the fences, don't you? :)

Yes, my bad.

Andrey

> 
> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>> No point calling amdgpu_fence_wait_empty before stopping the
>> SW scheduler otherwise there is always a chance another job sneaked
>> in after the wait.
>>
>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 9 +++++++--
>>   1 file changed, 7 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
>> index 34d51e962799..fd9282637549 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
>> @@ -533,6 +533,12 @@ void amdgpu_fence_driver_fini_hw(struct 
>> amdgpu_device *adev)
>>           if (!ring || !ring->fence_drv.initialized)
>>               continue;
>> +
>> +        /* Stop any new job submissions from sched before flushing 
>> the ring */
>> +        /* TODO Handle amdgpu_job_submit_direct and 
>> amdgpu_amdkfd_submit_ib */
>> +        if (!ring->no_scheduler)
>> +            drm_sched_fini(&ring->sched);
>> +
>>           r = amdgpu_fence_wait_empty(ring);
>>           if (r) {
>>               /* no need to trigger GPU reset as we are unloading */
>> @@ -541,8 +547,7 @@ void amdgpu_fence_driver_fini_hw(struct 
>> amdgpu_device *adev)
>>           if (ring->fence_drv.irq_src)
>>               amdgpu_irq_put(adev, ring->fence_drv.irq_src,
>>                          ring->fence_drv.irq_type);
>> -        if (!ring->no_scheduler)
>> -            drm_sched_fini(&ring->sched);
>> +
>>           del_timer_sync(&ring->fence_drv.fallback_timer);
>>       }
>>   }
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
> 
> 
> On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
> > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
> > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> > > > With this calling drm_dev_unplug will flush and block
> > > > all in flight IOCTLs
> > > > 
> > > > Also, add feature such that if device supports graceful unplug
> > > > we enclose entire IOCTL in SRCU critical section.
> > > > 
> > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > > 
> > > Nope.
> > > 
> > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
> 
> Then I am confused why we have https://elixir.bootlin.com/linux/v5.12/source/drivers/gpu/drm/drm_ioctl.c#L826
> currently in code ?

I forgot about this one, again. Thanks for reminding.

> > > Especially not with an opt-in flag so that it could be shrugged of as a
> > > driver hack. Most of these ioctls should have absolutely no problem
> > > working after hotunplug.
> > > 
> > > Also, doing this defeats the point since it pretty much guarantees
> > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract
> > > is that only execbuf (and even that only when userspace has indicated
> > > support for non-recoverable hw ctx) is allowed to fail. Anything else
> > > might crash userspace.
> 
> Given that as I pointed above we already fail any IOCTls with -ENODEV
> when device is unplugged, it seems those crashes don't happen that
> often ? Also, in all my testing I don't think I saw a user space crash
> I could attribute to this.

I guess it should be ok.

My reasons for making this work is both less trouble for userspace (did
you test with various wayland compositors out there, not just amdgpu x86
driver?), but also testing.

We still need a bunch of these checks in various places or you'll wait a
very long time for a pending modeset or similar to complete. Being able to
run that code easily after hotunplug has completed should help a lot with
testing.

Plus various drivers already acquired drm_dev_enter/exit and now I wonder
whether that was properly tested or not ...

I guess maybe we need a drm module option to disable this check, so that
we can exercise the code as if the ioctl has raced with hotunplug at the
worst possible moment.

Also atomic is really tricky here: I assume your testing has just done
normal synchronous commits, but anything that goes through atomic can be
done nonblocking in a separate thread. Which the ioctl catch-all here wont
capture.

> > > You probably need similar (and very precisely defined) rules for amdgpu.
> > > And those must definitely exclude any shard ioctls from randomly failing
> > > with EIO, because that just kills the box and defeats the point of trying
> > > to gracefully handling hotunplug and making sure userspace has a chance of
> > > survival. E.g. for atomic everything should continue, including flip
> > > completion, but we set all outputs to "disconnected" and send out the
> > > uevent. Maybe crtc enabling can fail too, but that can also be handled
> > > through the async status we're using to signal DP link failures to
> > > userspace.
> 
> As I pointed before, because of the complexity of the topic I prefer to
> take it step by step and solve first for secondary device use case, not
> for primary, display attached device.

Yeah makes sense. But then I think the right patch is to roll this out for
all drivers, properly justified with existing code. Not behind a driver
flag, because with all these different compositors the last thing we want
is a proliferation of driver-specific behaviour. That's imo the worst
option of all of them and needs to be avoided.

Cheers, Daniel


> 
> > > 
> > > I guess we should clarify this in the hotunplug doc?
> 
> Agree
> 
> > 
> > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
> > really make no sense, and where we're rather confident that all properly
> > implemented userspace will gracefully handle failures. Like a modeset, or
> > opening a device, or trying to import a dma-buf or stuff like that which
> > can already fail in normal operation for any kind of reason.
> > 
> > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
> > after hotunplug.
> 
> As I pointed above, this a bit confuses me given that we already do
> blanker rejection of IOCTLs if device is unplugged.

Well I'm confused about this too :-/

> > And then there's the middle ground, like doing a pageflip or buffer flush,
> > which I guess some userspace might handle, but risky to inflict those
> > consequences on them. atomic modeset is especially fun since depending
> > what you're doing it can be both "failures expected" and "failures not
> > really expected in normal operation".
> > 
> > Also, this really should be consistent across drivers, not solved with a
> > driver flag for every possible combination.
> > 
> > If you look at the current hotunplug kms drivers, they have
> > drm_dev_enter/exit sprinkled in specific hw callback functions because of
> > the above problems. But maybe it makes sense to change things in a few
> > cases. But then we should do it across the board.
> 
> So as I understand your preferred approach is that I scope any back_end, HW
> specific function with drm_dev_enter/exit because that where MMIO
> access takes place. But besides explicit MMIO access thorough
> register accessors in the HW back-end there is also indirect MMIO access
> taking place throughout the code in the driver because of various VRAM
> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
> access is spread all over in the driver and even in mid-layers such as
> TTM and not limited to HW back-end functions. It means it's much harder
> to spot such places to surgically scope them with drm_dev_enter/exit and
> also that any new such code introduced will immediately break hot unplug
> because the developers can't be expected to remember making their code
> robust to this specific use case. That why when we discussed internally
> what approach to take to protecting code with drm_dev_enter/exit we
> opted for using the widest available scope.

The thing is, you kinda have to anyway. There's enormous amounts of
asynchronous processing going on. E.g. nonblocking atomic commits also do
ttm unpinning and fun stuff like that, which if you sync things wrong can
happen way late. So the door for bad fallout is wide open :-(

I'm not sure where the right tradeoff is to make sure we catch them all,
and can make sure with testing that we've indeed caught them all.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 08/27] PCI: add support for dev_groups to struct pci_device_driver

[Bjorn Helgaas]

On Thu, Apr 29, 2021 at 12:53:15PM -0400, Andrey Grodzovsky wrote:
> On 2021-04-28 12:53 p.m., Bjorn Helgaas wrote:
> > On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
> > > This is exact copy of 'USB: add support for dev_groups to
> > > struct usb_device_driver' patch by Greg but just for
> > > the PCI case.

> > ...
> > The usual commit citation format is 7d9c1d2f7aca ("USB: add support
> > for dev_groups to struct usb_device_driver") so it's easier to locate
> > the commit.
> > 
> > I see there is also b71b283e3d6d ("USB: add support for dev_groups to
> > struct usb_driver").  I don't know enough about USB to know whether
> > 7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing
> > here, but I do see that struct usb_driver is far more common than
> > struct usb_device_driver.
> > 
> > PCI has struct pci_driver, but doesn't have the concept of a struct
> > pci_device_driver.
> 
> Since we don't have pci_device_driver then pci_driver is the best place
> for it then, no ?

Of course.  My point was just that maybe you should say this is
similar to b71b283e3d6d ("USB: add support for dev_groups to struct
usb_driver"), not similar to 7d9c1d2f7aca ("USB: add support for
dev_groups to struct usb_device_driver").

Bjorn
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
> On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
>>
>>
>> On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
>>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
>>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
>>>>> With this calling drm_dev_unplug will flush and block
>>>>> all in flight IOCTLs
>>>>>
>>>>> Also, add feature such that if device supports graceful unplug
>>>>> we enclose entire IOCTL in SRCU critical section.
>>>>>
>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>
>>>> Nope.
>>>>
>>>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
>>
>> Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C1821a19173a84ebae31108d90b41b2fa%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553199084555468%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=d6kXadWHv4CEDgODsm%2FOzIIjIDA9rZDLUuV11MmEU3A%3D&amp;reserved=0
>> currently in code ?
> 
> I forgot about this one, again. Thanks for reminding.
> 
>>>> Especially not with an opt-in flag so that it could be shrugged of as a
>>>> driver hack. Most of these ioctls should have absolutely no problem
>>>> working after hotunplug.
>>>>
>>>> Also, doing this defeats the point since it pretty much guarantees
>>>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract
>>>> is that only execbuf (and even that only when userspace has indicated
>>>> support for non-recoverable hw ctx) is allowed to fail. Anything else
>>>> might crash userspace.
>>
>> Given that as I pointed above we already fail any IOCTls with -ENODEV
>> when device is unplugged, it seems those crashes don't happen that
>> often ? Also, in all my testing I don't think I saw a user space crash
>> I could attribute to this.
> 
> I guess it should be ok.

What should be ok ?
> 
> My reasons for making this work is both less trouble for userspace (did
> you test with various wayland compositors out there, not just amdgpu x86

I didn't - will give it a try.

> driver?), but also testing.
> 
> We still need a bunch of these checks in various places or you'll wait a
> very long time for a pending modeset or similar to complete. Being able to
> run that code easily after hotunplug has completed should help a lot with
> testing.
> 
> Plus various drivers already acquired drm_dev_enter/exit and now I wonder
> whether that was properly tested or not ...
> 
> I guess maybe we need a drm module option to disable this check, so that
> we can exercise the code as if the ioctl has raced with hotunplug at the
> worst possible moment.
> 
> Also atomic is really tricky here: I assume your testing has just done
> normal synchronous commits, but anything that goes through atomic can be
> done nonblocking in a separate thread. Which the ioctl catch-all here wont
> capture.

Yes, async commit was on my mind and thanks for reminding me. Indeed
I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
drm_dev_enter/exit. Note that i have a bunch of patches, all name's
starting with 'Scope....' that just methodically put all the background
work items and timers the drivers schedules in drm_dev_enter/exit scope.
This was supposed to be part of the 'Scope Display code' patch.

> 
>>>> You probably need similar (and very precisely defined) rules for amdgpu.
>>>> And those must definitely exclude any shard ioctls from randomly failing
>>>> with EIO, because that just kills the box and defeats the point of trying
>>>> to gracefully handling hotunplug and making sure userspace has a chance of
>>>> survival. E.g. for atomic everything should continue, including flip
>>>> completion, but we set all outputs to "disconnected" and send out the
>>>> uevent. Maybe crtc enabling can fail too, but that can also be handled
>>>> through the async status we're using to signal DP link failures to
>>>> userspace.
>>
>> As I pointed before, because of the complexity of the topic I prefer to
>> take it step by step and solve first for secondary device use case, not
>> for primary, display attached device.
> 
> Yeah makes sense. But then I think the right patch is to roll this out for
> all drivers, properly justified with existing code. Not behind a driver
> flag, because with all these different compositors the last thing we want
> is a proliferation of driver-specific behaviour. That's imo the worst
> option of all of them and needs to be avoided.

So this kind of patch would be acceptable to you if I unconditionally
scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
I am worried to break other drivers with this, see patch 
https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=f0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6
Before setting drm_dev_unplug I go through a whole process of signalling
all possible fences in the system which some one some where might be
waiting on. My concern is that in the absence of HW those fences won't
signal and so unless I signal them myself srcu_synchrionize in
drm_dev_unplug will hang waiting for any such code scoped by
drm_dev_enter/exit.

Andrey

> 
> Cheers, Daniel
> 
> 
>>
>>>>
>>>> I guess we should clarify this in the hotunplug doc?
>>
>> Agree
>>
>>>
>>> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
>>> really make no sense, and where we're rather confident that all properly
>>> implemented userspace will gracefully handle failures. Like a modeset, or
>>> opening a device, or trying to import a dma-buf or stuff like that which
>>> can already fail in normal operation for any kind of reason.
>>>
>>> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
>>> after hotunplug.
>>
>> As I pointed above, this a bit confuses me given that we already do
>> blanker rejection of IOCTLs if device is unplugged.
> 
> Well I'm confused about this too :-/
> 
>>> And then there's the middle ground, like doing a pageflip or buffer flush,
>>> which I guess some userspace might handle, but risky to inflict those
>>> consequences on them. atomic modeset is especially fun since depending
>>> what you're doing it can be both "failures expected" and "failures not
>>> really expected in normal operation".
>>>
>>> Also, this really should be consistent across drivers, not solved with a
>>> driver flag for every possible combination.
>>>
>>> If you look at the current hotunplug kms drivers, they have
>>> drm_dev_enter/exit sprinkled in specific hw callback functions because of
>>> the above problems. But maybe it makes sense to change things in a few
>>> cases. But then we should do it across the board.
>>
>> So as I understand your preferred approach is that I scope any back_end, HW
>> specific function with drm_dev_enter/exit because that where MMIO
>> access takes place. But besides explicit MMIO access thorough
>> register accessors in the HW back-end there is also indirect MMIO access
>> taking place throughout the code in the driver because of various VRAM
>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>> access is spread all over in the driver and even in mid-layers such as
>> TTM and not limited to HW back-end functions. It means it's much harder
>> to spot such places to surgically scope them with drm_dev_enter/exit and
>> also that any new such code introduced will immediately break hot unplug
>> because the developers can't be expected to remember making their code
>> robust to this specific use case. That why when we discussed internally
>> what approach to take to protecting code with drm_dev_enter/exit we
>> opted for using the widest available scope.
> 
> The thing is, you kinda have to anyway. There's enormous amounts of
> asynchronous processing going on. E.g. nonblocking atomic commits also do
> ttm unpinning and fun stuff like that, which if you sync things wrong can
> happen way late. So the door for bad fallout is wide open :-(
> 
> I'm not sure where the right tradeoff is to make sure we catch them all,
> and can make sure with testing that we've indeed caught them all.
> -Daniel
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 08/27] PCI: add support for dev_groups to struct pci_device_driver

[Andrey Grodzovsky]

On 2021-04-29 3:23 p.m., Bjorn Helgaas wrote:
> On Thu, Apr 29, 2021 at 12:53:15PM -0400, Andrey Grodzovsky wrote:
>> On 2021-04-28 12:53 p.m., Bjorn Helgaas wrote:
>>> On Wed, Apr 28, 2021 at 11:11:48AM -0400, Andrey Grodzovsky wrote:
>>>> This is exact copy of 'USB: add support for dev_groups to
>>>> struct usb_device_driver' patch by Greg but just for
>>>> the PCI case.
> 
>>> ...
>>> The usual commit citation format is 7d9c1d2f7aca ("USB: add support
>>> for dev_groups to struct usb_device_driver") so it's easier to locate
>>> the commit.
>>>
>>> I see there is also b71b283e3d6d ("USB: add support for dev_groups to
>>> struct usb_driver").  I don't know enough about USB to know whether
>>> 7d9c1d2f7aca or b71b283e3d6d is a closer analogue to what you're doing
>>> here, but I do see that struct usb_driver is far more common than
>>> struct usb_device_driver.
>>>
>>> PCI has struct pci_driver, but doesn't have the concept of a struct
>>> pci_device_driver.
>>
>> Since we don't have pci_device_driver then pci_driver is the best place
>> for it then, no ?
> 
> Of course.  My point was just that maybe you should say this is
> similar to b71b283e3d6d ("USB: add support for dev_groups to struct
> usb_driver"), not similar to 7d9c1d2f7aca ("USB: add support for
> dev_groups to struct usb_device_driver").

Got it.

Andrey

> 
> Bjorn
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&data=04%7C01%7Candrey.grodzovsky%40amd.com%7C9778eea1c3164f9fbc5f08d90b443ba6%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553209952825202%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tkycS4EST1Q%2BkEWlmzocPjCxaONVk5sPzPnWmrmbfcg%3D&reserved=0
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late

[Alex Deucher]

On Thu, Apr 29, 2021 at 3:04 AM Christian König
<ckoenig.leichtzumerken@gmail.com> wrote:
>
>
>
> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> > Some of the stuff in amdgpu_device_fini such as HW interrupts
> > disable and pending fences finilization must be done right away on
> > pci_remove while most of the stuff which relates to finilizing and
> > releasing driver data structures can be kept until
> > drm_driver.release hook is called, i.e. when the last device
> > reference is dropped.
> >
> > v4: Change functions prefix early->hw and late->sw
> >
> > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>
> Acked-by: Christian König <christian.koenig@amd.com>
>
> But Alex should acknowledge this as well since it is general driver design.

Looks good to me as well.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>

>
> Christian.
>
> > ---
> >   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 ++++-
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++-------
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  7 ++----
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c  | 15 ++++++++++++-
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    | 26 +++++++++++++---------
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h    |  3 ++-
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 12 +++++++++-
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c    |  1 +
> >   drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h   |  3 ++-
> >   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  2 +-
> >   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  2 +-
> >   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  2 +-
> >   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  2 +-
> >   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  2 +-
> >   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  2 +-
> >   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  2 +-
> >   drivers/gpu/drm/amd/amdgpu/vega20_ih.c     |  2 +-
> >   17 files changed, 79 insertions(+), 36 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> > index 1af2fa1591fd..fddb82897e5d 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> > @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
> >
> >   int amdgpu_device_init(struct amdgpu_device *adev,
> >                      uint32_t flags);
> > -void amdgpu_device_fini(struct amdgpu_device *adev);
> > +void amdgpu_device_fini_hw(struct amdgpu_device *adev);
> > +void amdgpu_device_fini_sw(struct amdgpu_device *adev);
> > +
> >   int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
> >
> >   void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
> > @@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev);
> >   int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
> >   void amdgpu_driver_postclose_kms(struct drm_device *dev,
> >                                struct drm_file *file_priv);
> > +void amdgpu_driver_release_kms(struct drm_device *dev);
> > +
> >   int amdgpu_device_ip_suspend(struct amdgpu_device *adev);
> >   int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);
> >   int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > index 6447cd6ca5a8..8d22b79fc1cd 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> > @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev,
> >    * Tear down the driver info (all asics).
> >    * Called at driver shutdown.
> >    */
> > -void amdgpu_device_fini(struct amdgpu_device *adev)
> > +void amdgpu_device_fini_hw(struct amdgpu_device *adev)
> >   {
> >       dev_info(adev->dev, "amdgpu: finishing device.\n");
> >       flush_delayed_work(&adev->delayed_init_work);
> >       adev->shutdown = true;
> >
> > -     kfree(adev->pci_state);
> > -
> >       /* make sure IB test finished before entering exclusive mode
> >        * to avoid preemption on IB test
> >        * */
> > @@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
> >               else
> >                       drm_atomic_helper_shutdown(adev_to_drm(adev));
> >       }
> > -     amdgpu_fence_driver_fini(adev);
> > +     amdgpu_fence_driver_fini_hw(adev);
> > +
> >       if (adev->pm_sysfs_en)
> >               amdgpu_pm_sysfs_fini(adev);
> > +     if (adev->ucode_sysfs_en)
> > +             amdgpu_ucode_sysfs_fini(adev);
> > +     sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
> > +
> > +
> >       amdgpu_fbdev_fini(adev);
> > +
> > +     amdgpu_irq_fini_hw(adev);
> > +}
> > +
> > +void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> > +{
> >       amdgpu_device_ip_fini(adev);
> > +     amdgpu_fence_driver_fini_sw(adev);
> >       release_firmware(adev->firmware.gpu_info_fw);
> >       adev->firmware.gpu_info_fw = NULL;
> >       adev->accel_working = false;
> > @@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
> >       adev->rmmio = NULL;
> >       amdgpu_device_doorbell_fini(adev);
> >
> > -     if (adev->ucode_sysfs_en)
> > -             amdgpu_ucode_sysfs_fini(adev);
> > -
> > -     sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
> >       if (IS_ENABLED(CONFIG_PERF_EVENTS))
> >               amdgpu_pmu_fini(adev);
> >       if (adev->mman.discovery_bin)
> >               amdgpu_discovery_fini(adev);
> > +
> > +     kfree(adev->pci_state);
> > +
> >   }
> >
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> > index 671ec1002230..54cb5ee2f563 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> > @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev)
> >   {
> >       struct drm_device *dev = pci_get_drvdata(pdev);
> >
> > -#ifdef MODULE
> > -     if (THIS_MODULE->state != MODULE_STATE_GOING)
> > -#endif
> > -             DRM_ERROR("Hotplug removal is not supported\n");
> >       drm_dev_unplug(dev);
> >       amdgpu_driver_unload_kms(dev);
> > +
> >       pci_disable_device(pdev);
> > -     pci_set_drvdata(pdev, NULL);
> >   }
> >
> >   static void
> > @@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = {
> >       .dumb_create = amdgpu_mode_dumb_create,
> >       .dumb_map_offset = amdgpu_mode_dumb_mmap,
> >       .fops = &amdgpu_driver_kms_fops,
> > +     .release = &amdgpu_driver_release_kms,
> >
> >       .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
> >       .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> > index 8e0a5650d383..34d51e962799 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> > @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
> >    *
> >    * Tear down the fence driver for all possible rings (all asics).
> >    */
> > -void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
> > +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
> >   {
> >       unsigned i, j;
> >       int r;
> > @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
> >               if (!ring->no_scheduler)
> >                       drm_sched_fini(&ring->sched);
> >               del_timer_sync(&ring->fence_drv.fallback_timer);
> > +     }
> > +}
> > +
> > +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev)
> > +{
> > +     unsigned int i, j;
> > +
> > +     for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
> > +             struct amdgpu_ring *ring = adev->rings[i];
> > +
> > +             if (!ring || !ring->fence_drv.initialized)
> > +                     continue;
> > +
> >               for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j)
> >                       dma_fence_put(ring->fence_drv.fences[j]);
> >               kfree(ring->fence_drv.fences);
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> > index afbbec82a289..63e815c27585 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> > @@ -49,6 +49,7 @@
> >   #include <drm/drm_irq.h>
> >   #include <drm/drm_vblank.h>
> >   #include <drm/amdgpu_drm.h>
> > +#include <drm/drm_drv.h>
> >   #include "amdgpu.h"
> >   #include "amdgpu_ih.h"
> >   #include "atom.h"
> > @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
> >       return 0;
> >   }
> >
> > +
> > +void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
> > +{
> > +     if (adev->irq.installed) {
> > +             drm_irq_uninstall(&adev->ddev);
> > +             adev->irq.installed = false;
> > +             if (adev->irq.msi_enabled)
> > +                     pci_free_irq_vectors(adev->pdev);
> > +
> > +             if (!amdgpu_device_has_dc_support(adev))
> > +                     flush_work(&adev->hotplug_work);
> > +     }
> > +}
> > +
> >   /**
> >    * amdgpu_irq_fini - shut down interrupt handling
> >    *
> > @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
> >    * functionality, shuts down vblank, hotplug and reset interrupt handling,
> >    * turns off interrupts from all sources (all ASICs).
> >    */
> > -void amdgpu_irq_fini(struct amdgpu_device *adev)
> > +void amdgpu_irq_fini_sw(struct amdgpu_device *adev)
> >   {
> >       unsigned i, j;
> >
> > -     if (adev->irq.installed) {
> > -             drm_irq_uninstall(adev_to_drm(adev));
> > -             adev->irq.installed = false;
> > -             if (adev->irq.msi_enabled)
> > -                     pci_free_irq_vectors(adev->pdev);
> > -             if (!amdgpu_device_has_dc_support(adev))
> > -                     flush_work(&adev->hotplug_work);
> > -     }
> > -
> >       for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) {
> >               if (!adev->irq.client[i].sources)
> >                       continue;
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> > index ac527e5deae6..392a7324e2b1 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> > @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev);
> >   irqreturn_t amdgpu_irq_handler(int irq, void *arg);
> >
> >   int amdgpu_irq_init(struct amdgpu_device *adev);
> > -void amdgpu_irq_fini(struct amdgpu_device *adev);
> > +void amdgpu_irq_fini_sw(struct amdgpu_device *adev);
> > +void amdgpu_irq_fini_hw(struct amdgpu_device *adev);
> >   int amdgpu_irq_add_id(struct amdgpu_device *adev,
> >                     unsigned client_id, unsigned src_id,
> >                     struct amdgpu_irq_src *source);
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> > index 64beb3399604..1af3fba7bfd4 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> > @@ -29,6 +29,7 @@
> >   #include "amdgpu.h"
> >   #include <drm/drm_debugfs.h>
> >   #include <drm/amdgpu_drm.h>
> > +#include <drm/drm_drv.h>
> >   #include "amdgpu_uvd.h"
> >   #include "amdgpu_vce.h"
> >   #include "atom.h"
> > @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev)
> >       }
> >
> >       amdgpu_acpi_fini(adev);
> > -     amdgpu_device_fini(adev);
> > +     amdgpu_device_fini_hw(adev);
> >   }
> >
> >   void amdgpu_register_gpu_instance(struct amdgpu_device *adev)
> > @@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
> >       pm_runtime_put_autosuspend(dev->dev);
> >   }
> >
> > +
> > +void amdgpu_driver_release_kms(struct drm_device *dev)
> > +{
> > +     struct amdgpu_device *adev = drm_to_adev(dev);
> > +
> > +     amdgpu_device_fini_sw(adev);
> > +     pci_set_drvdata(adev->pdev, NULL);
> > +}
> > +
> >   /*
> >    * VBlank related functions.
> >    */
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> > index 1fb2a91ad30a..c0a16eac4923 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> > @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev)
> >       if (!con)
> >               return 0;
> >
> > +
> >       /* Need disable ras on all IPs here before ip [hw/sw]fini */
> >       amdgpu_ras_disable_all_features(adev, 0);
> >       amdgpu_ras_recovery_fini(adev);
> > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> > index 56acec1075ac..0f195f7bf797 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> > @@ -107,7 +107,8 @@ struct amdgpu_fence_driver {
> >   };
> >
> >   int amdgpu_fence_driver_init(struct amdgpu_device *adev);
> > -void amdgpu_fence_driver_fini(struct amdgpu_device *adev);
> > +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev);
> > +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev);
> >   void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
> >
> >   int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
> > diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> > index d3745711d55f..183d44a6583c 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> > @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> >       amdgpu_irq_remove_domain(adev);
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> > index 307c01301c87..d32743949003 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> > @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> >       amdgpu_irq_remove_domain(adev);
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> > index cc957471f31e..da96c6013477 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> > @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> >       amdgpu_irq_remove_domain(adev);
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> > index f4e4040bbd25..5eea4550b856 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> > @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> > diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> > index 51880f6ef634..751307f3252c 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> > @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> >
> >       return 0;
> > diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> > index 249fcbee7871..973d80ec7f6c 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> > @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> >       amdgpu_irq_remove_domain(adev);
> >
> > diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> > index 88626d83e07b..2d0094c276ca 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> > @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> > diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> > index 5a3c867d5881..9059b21b079f 100644
> > --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> > +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> > @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle)
> >   {
> >       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> >
> > -     amdgpu_irq_fini(adev);
> > +     amdgpu_irq_fini_sw(adev);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> >       amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Alex Deucher]

On Wed, Apr 28, 2021 at 11:13 AM Andrey Grodzovsky
<andrey.grodzovsky@amd.com> wrote:
>
> Handle all DMA IOMMU gropup related dependencies before the
> group is removed.
>
> v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>  drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>  drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>  drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>  drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>  drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>  drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>  drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>  14 files changed, 56 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index fddb82897e5d..30a24db5f4d1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>
>         bool                            in_pci_err_recovery;
>         struct pci_saved_state          *pci_state;
> +
> +       struct list_head                device_bo_list;
>  };
>
>  static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 46d646c40338..91594ddc2459 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -70,6 +70,7 @@
>  #include <drm/task_barrier.h>
>  #include <linux/pm_runtime.h>
>
> +
>  MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>  MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>  MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
> @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = {
>         NULL
>  };
>
> -
>  /**
>   * amdgpu_device_init - initialize the driver
>   *
> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>
>         INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>
> +       INIT_LIST_HEAD(&adev->device_bo_list);
> +
>         adev->gfx.gfx_off_req_count = 1;
>         adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>
> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>         return r;
>  }
>
> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)

Prefix this with amdgpu_device for consistency.  E.g.,
amdgpu_device_clear_dma_mappings()

> +{
> +       struct amdgpu_bo *bo = NULL;
> +
> +       /*
> +        * Unmaps all DMA mappings before device will be removed from it's
> +        * IOMMU group otherwise in case of IOMMU enabled system a crash
> +        * will happen.
> +        */
> +
> +       spin_lock(&adev->mman.bdev.lru_lock);
> +       while (!list_empty(&adev->device_bo_list)) {
> +               bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);
> +               list_del_init(&bo->bo);
> +               spin_unlock(&adev->mman.bdev.lru_lock);
> +               if (bo->tbo.ttm)
> +                       ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
> +               spin_lock(&adev->mman.bdev.lru_lock);
> +       }
> +       spin_unlock(&adev->mman.bdev.lru_lock);
> +}
> +
>  /**
>   * amdgpu_device_fini - tear down the driver
>   *
> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>                 amdgpu_ucode_sysfs_fini(adev);
>         sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>
> -
>         amdgpu_fbdev_fini(adev);
>
>         amdgpu_irq_fini_hw(adev);
>
>         amdgpu_device_ip_fini_early(adev);
> +
> +       amdgpu_clear_dma_mappings(adev);
> +
> +       amdgpu_gart_dummy_page_fini(adev);
>  }
>
>  void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> index fde2d899b2c4..49cdcaf8512d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
>   *
>   * Frees the dummy page used by the driver (all asics).
>   */
> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>  {
>         if (!adev->dummy_page_addr)
>                 return;
> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>         vfree(adev->gart.pages);
>         adev->gart.pages = NULL;
>  #endif
> -       amdgpu_gart_dummy_page_fini(adev);
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> index afa2e2877d87..5678d9c105ab 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev);
>  void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>  int amdgpu_gart_init(struct amdgpu_device *adev);
>  void amdgpu_gart_fini(struct amdgpu_device *adev);
> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>  int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>                        int pages);
>  int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> index 63e815c27585..a922154953a7 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
>                 if (!amdgpu_device_has_dc_support(adev))
>                         flush_work(&adev->hotplug_work);
>         }
> +
> +       if (adev->irq.ih_soft.ring)
> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> +       if (adev->irq.ih.ring)
> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> +       if (adev->irq.ih1.ring)
> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> +       if (adev->irq.ih2.ring)
> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>  }
>
>  /**
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> index 485f249d063a..62d829f5e62c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
>                 list_del_init(&bo->shadow_list);
>                 mutex_unlock(&adev->shadow_list_lock);
>         }
> -       amdgpu_bo_unref(&bo->parent);
>
> +
> +       spin_lock(&adev->mman.bdev.lru_lock);
> +       list_del(&bo->bo);
> +       spin_unlock(&adev->mman.bdev.lru_lock);
> +
> +       amdgpu_bo_unref(&bo->parent);
>         kfree(bo->metadata);
>         kfree(bo);
>  }
> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
>         if (bp->type == ttm_bo_type_device)
>                 bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>
> +       INIT_LIST_HEAD(&bo->bo);
> +
> +       spin_lock(&adev->mman.bdev.lru_lock);
> +       list_add_tail(&bo->bo, &adev->device_bo_list);
> +       spin_unlock(&adev->mman.bdev.lru_lock);
> +
>         return 0;
>
>  fail_unreserve:
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> index 9ac37569823f..5ae8555ef275 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>         struct list_head                shadow_list;
>
>         struct kgd_mem                  *kfd_bo;
> +
> +       struct list_head                bo;
>  };
>
>  static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> index 183d44a6583c..df385ffc9768 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>         struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>
>         amdgpu_irq_fini_sw(adev);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>         amdgpu_irq_remove_domain(adev);
>
>         return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> index d32743949003..b8c47e0cf37a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>         struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>
>         amdgpu_irq_fini_sw(adev);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>         amdgpu_irq_remove_domain(adev);
>
>         return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> index da96c6013477..ddfe4eaeea05 100644
> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>         struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>
>         amdgpu_irq_fini_sw(adev);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>         amdgpu_irq_remove_domain(adev);
>
>         return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> index 5eea4550b856..e171a9e78544 100644
> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>
>         amdgpu_irq_fini_sw(adev);
>         amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);

Shouldn't the soft ring be removed as well?

> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>
>         return 0;
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> index 751307f3252c..9a24f17a5750 100644
> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>         struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>
>         amdgpu_irq_fini_sw(adev);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>
>         return 0;
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> index 973d80ec7f6c..b08905d1c00f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>         struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>
>         amdgpu_irq_fini_sw(adev);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>         amdgpu_irq_remove_domain(adev);
>
>         return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> index 2d0094c276ca..8c8abc00f710 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>
>         amdgpu_irq_fini_sw(adev);
>         amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);

Same here?

> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>
>         return 0;
>  }
> --
> 2.25.1
>
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late

[Lazar, Lijo]

[-- Attachment #1.1: Type: text/plain, Size: 18339 bytes --]

[AMD Official Use Only - Internal Distribution Only]

sysfs cleanup is a sw cleanup to me but done inside hw fini. sw/hw separation is not strictly followed, or name it like stage1/stage2 fini.

Thanks,
Lijo
________________________________
From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of Andrey Grodzovsky <andrey.grodzovsky@amd.com>
Sent: Wednesday, April 28, 2021 8:41:43 PM
To: dri-devel@lists.freedesktop.org <dri-devel@lists.freedesktop.org>; amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>; linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>; ckoenig.leichtzumerken@gmail.com <ckoenig.leichtzumerken@gmail.com>; daniel.vetter@ffwll.ch <daniel.vetter@ffwll.ch>; Wentland, Harry <Harry.Wentland@amd.com>
Cc: Grodzovsky, Andrey <Andrey.Grodzovsky@amd.com>; gregkh@linuxfoundation.org <gregkh@linuxfoundation.org>; Kuehling, Felix <Felix.Kuehling@amd.com>; ppaalanen@gmail.com <ppaalanen@gmail.com>; helgaas@kernel.org <helgaas@kernel.org>; Deucher, Alexander <Alexander.Deucher@amd.com>
Subject: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late

Some of the stuff in amdgpu_device_fini such as HW interrupts
disable and pending fences finilization must be done right away on
pci_remove while most of the stuff which relates to finilizing and
releasing driver data structures can be kept until
drm_driver.release hook is called, i.e. when the last device
reference is dropped.

v4: Change functions prefix early->hw and late->sw

Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 ++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  7 ++----
 drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c  | 15 ++++++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    | 26 +++++++++++++---------
 drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h    |  3 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 12 +++++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c    |  1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h   |  3 ++-
 drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  2 +-
 drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  2 +-
 drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  2 +-
 drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  2 +-
 drivers/gpu/drm/amd/amdgpu/si_ih.c         |  2 +-
 drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  2 +-
 drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  2 +-
 drivers/gpu/drm/amd/amdgpu/vega20_ih.c     |  2 +-
 17 files changed, 79 insertions(+), 36 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 1af2fa1591fd..fddb82897e5d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1073,7 +1073,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)

 int amdgpu_device_init(struct amdgpu_device *adev,
                        uint32_t flags);
-void amdgpu_device_fini(struct amdgpu_device *adev);
+void amdgpu_device_fini_hw(struct amdgpu_device *adev);
+void amdgpu_device_fini_sw(struct amdgpu_device *adev);
+
 int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);

 void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
@@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev);
 int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
 void amdgpu_driver_postclose_kms(struct drm_device *dev,
                                  struct drm_file *file_priv);
+void amdgpu_driver_release_kms(struct drm_device *dev);
+
 int amdgpu_device_ip_suspend(struct amdgpu_device *adev);
 int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);
 int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 6447cd6ca5a8..8d22b79fc1cd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev,
  * Tear down the driver info (all asics).
  * Called at driver shutdown.
  */
-void amdgpu_device_fini(struct amdgpu_device *adev)
+void amdgpu_device_fini_hw(struct amdgpu_device *adev)
 {
         dev_info(adev->dev, "amdgpu: finishing device.\n");
         flush_delayed_work(&adev->delayed_init_work);
         adev->shutdown = true;

-       kfree(adev->pci_state);
-
         /* make sure IB test finished before entering exclusive mode
          * to avoid preemption on IB test
          * */
@@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
                 else
                         drm_atomic_helper_shutdown(adev_to_drm(adev));
         }
-       amdgpu_fence_driver_fini(adev);
+       amdgpu_fence_driver_fini_hw(adev);
+
         if (adev->pm_sysfs_en)
                 amdgpu_pm_sysfs_fini(adev);
+       if (adev->ucode_sysfs_en)
+               amdgpu_ucode_sysfs_fini(adev);
+       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
+
+
         amdgpu_fbdev_fini(adev);
+
+       amdgpu_irq_fini_hw(adev);
+}
+
+void amdgpu_device_fini_sw(struct amdgpu_device *adev)
+{
         amdgpu_device_ip_fini(adev);
+       amdgpu_fence_driver_fini_sw(adev);
         release_firmware(adev->firmware.gpu_info_fw);
         adev->firmware.gpu_info_fw = NULL;
         adev->accel_working = false;
@@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
         adev->rmmio = NULL;
         amdgpu_device_doorbell_fini(adev);

-       if (adev->ucode_sysfs_en)
-               amdgpu_ucode_sysfs_fini(adev);
-
-       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
         if (IS_ENABLED(CONFIG_PERF_EVENTS))
                 amdgpu_pmu_fini(adev);
         if (adev->mman.discovery_bin)
                 amdgpu_discovery_fini(adev);
+
+       kfree(adev->pci_state);
+
 }


diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 671ec1002230..54cb5ee2f563 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev)
 {
         struct drm_device *dev = pci_get_drvdata(pdev);

-#ifdef MODULE
-       if (THIS_MODULE->state != MODULE_STATE_GOING)
-#endif
-               DRM_ERROR("Hotplug removal is not supported\n");
         drm_dev_unplug(dev);
         amdgpu_driver_unload_kms(dev);
+
         pci_disable_device(pdev);
-       pci_set_drvdata(pdev, NULL);
 }

 static void
@@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = {
         .dumb_create = amdgpu_mode_dumb_create,
         .dumb_map_offset = amdgpu_mode_dumb_mmap,
         .fops = &amdgpu_driver_kms_fops,
+       .release = &amdgpu_driver_release_kms,

         .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
         .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index 8e0a5650d383..34d51e962799 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
  *
  * Tear down the fence driver for all possible rings (all asics).
  */
-void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
+void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
 {
         unsigned i, j;
         int r;
@@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
                 if (!ring->no_scheduler)
                         drm_sched_fini(&ring->sched);
                 del_timer_sync(&ring->fence_drv.fallback_timer);
+       }
+}
+
+void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev)
+{
+       unsigned int i, j;
+
+       for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
+               struct amdgpu_ring *ring = adev->rings[i];
+
+               if (!ring || !ring->fence_drv.initialized)
+                       continue;
+
                 for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j)
                         dma_fence_put(ring->fence_drv.fences[j]);
                 kfree(ring->fence_drv.fences);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index afbbec82a289..63e815c27585 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -49,6 +49,7 @@
 #include <drm/drm_irq.h>
 #include <drm/drm_vblank.h>
 #include <drm/amdgpu_drm.h>
+#include <drm/drm_drv.h>
 #include "amdgpu.h"
 #include "amdgpu_ih.h"
 #include "atom.h"
@@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
         return 0;
 }

+
+void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
+{
+       if (adev->irq.installed) {
+               drm_irq_uninstall(&adev->ddev);
+               adev->irq.installed = false;
+               if (adev->irq.msi_enabled)
+                       pci_free_irq_vectors(adev->pdev);
+
+               if (!amdgpu_device_has_dc_support(adev))
+                       flush_work(&adev->hotplug_work);
+       }
+}
+
 /**
  * amdgpu_irq_fini - shut down interrupt handling
  *
@@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
  * functionality, shuts down vblank, hotplug and reset interrupt handling,
  * turns off interrupts from all sources (all ASICs).
  */
-void amdgpu_irq_fini(struct amdgpu_device *adev)
+void amdgpu_irq_fini_sw(struct amdgpu_device *adev)
 {
         unsigned i, j;

-       if (adev->irq.installed) {
-               drm_irq_uninstall(adev_to_drm(adev));
-               adev->irq.installed = false;
-               if (adev->irq.msi_enabled)
-                       pci_free_irq_vectors(adev->pdev);
-               if (!amdgpu_device_has_dc_support(adev))
-                       flush_work(&adev->hotplug_work);
-       }
-
         for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) {
                 if (!adev->irq.client[i].sources)
                         continue;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
index ac527e5deae6..392a7324e2b1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
@@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev);
 irqreturn_t amdgpu_irq_handler(int irq, void *arg);

 int amdgpu_irq_init(struct amdgpu_device *adev);
-void amdgpu_irq_fini(struct amdgpu_device *adev);
+void amdgpu_irq_fini_sw(struct amdgpu_device *adev);
+void amdgpu_irq_fini_hw(struct amdgpu_device *adev);
 int amdgpu_irq_add_id(struct amdgpu_device *adev,
                       unsigned client_id, unsigned src_id,
                       struct amdgpu_irq_src *source);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index 64beb3399604..1af3fba7bfd4 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -29,6 +29,7 @@
 #include "amdgpu.h"
 #include <drm/drm_debugfs.h>
 #include <drm/amdgpu_drm.h>
+#include <drm/drm_drv.h>
 #include "amdgpu_uvd.h"
 #include "amdgpu_vce.h"
 #include "atom.h"
@@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev)
         }

         amdgpu_acpi_fini(adev);
-       amdgpu_device_fini(adev);
+       amdgpu_device_fini_hw(adev);
 }

 void amdgpu_register_gpu_instance(struct amdgpu_device *adev)
@@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
         pm_runtime_put_autosuspend(dev->dev);
 }

+
+void amdgpu_driver_release_kms(struct drm_device *dev)
+{
+       struct amdgpu_device *adev = drm_to_adev(dev);
+
+       amdgpu_device_fini_sw(adev);
+       pci_set_drvdata(adev->pdev, NULL);
+}
+
 /*
  * VBlank related functions.
  */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index 1fb2a91ad30a..c0a16eac4923 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev)
         if (!con)
                 return 0;

+
         /* Need disable ras on all IPs here before ip [hw/sw]fini */
         amdgpu_ras_disable_all_features(adev, 0);
         amdgpu_ras_recovery_fini(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
index 56acec1075ac..0f195f7bf797 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
@@ -107,7 +107,8 @@ struct amdgpu_fence_driver {
 };

 int amdgpu_fence_driver_init(struct amdgpu_device *adev);
-void amdgpu_fence_driver_fini(struct amdgpu_device *adev);
+void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev);
+void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev);
 void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);

 int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
index d3745711d55f..183d44a6583c 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
@@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih);
         amdgpu_irq_remove_domain(adev);

diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
index 307c01301c87..d32743949003 100644
--- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
@@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih);
         amdgpu_irq_remove_domain(adev);

diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
index cc957471f31e..da96c6013477 100644
--- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
@@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih);
         amdgpu_irq_remove_domain(adev);

diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
index f4e4040bbd25..5eea4550b856 100644
--- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
@@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
index 51880f6ef634..751307f3252c 100644
--- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
@@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih);

         return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
index 249fcbee7871..973d80ec7f6c 100644
--- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
@@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih);
         amdgpu_irq_remove_domain(adev);

diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
index 88626d83e07b..2d0094c276ca 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
@@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
index 5a3c867d5881..9059b21b079f 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
@@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle)
 {
         struct amdgpu_device *adev = (struct amdgpu_device *)handle;

-       amdgpu_irq_fini(adev);
+       amdgpu_irq_fini_sw(adev);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
         amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
--
2.25.1

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Clijo.lazar%40amd.com%7C71f92dcef6d04dfea28308d90a5818c9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637552195764790324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=fzGS6ZXYffGH0DE%2BEM4jxyB8yDGpYO%2FmT%2FKAtgig1Tw%3D&amp;reserved=0

[-- Attachment #1.2: Type: text/html, Size: 29880 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late

[Andrey Grodzovsky]

On 2021-04-30 1:19 a.m., Lazar, Lijo wrote:
> [AMD Official Use Only - Internal Distribution Only]
> 
> 
> sysfs cleanup is a sw cleanup to me but done inside hw fini. sw/hw 
> separation is not strictly followed, or name it like stage1/stage2 fini.

The thing is that it was called early_fini and late_fini before and then
according to Daniel's requirest it was changed to fini_hw and fini_sw

Andrey

> 
> Thanks,
> Lijo
> ------------------------------------------------------------------------
> *From:* amd-gfx <amd-gfx-bounces@lists.freedesktop.org> on behalf of 
> Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> *Sent:* Wednesday, April 28, 2021 8:41:43 PM
> *To:* dri-devel@lists.freedesktop.org <dri-devel@lists.freedesktop.org>; 
> amd-gfx@lists.freedesktop.org <amd-gfx@lists.freedesktop.org>; 
> linux-pci@vger.kernel.org <linux-pci@vger.kernel.org>; 
> ckoenig.leichtzumerken@gmail.com <ckoenig.leichtzumerken@gmail.com>; 
> daniel.vetter@ffwll.ch <daniel.vetter@ffwll.ch>; Wentland, Harry 
> <Harry.Wentland@amd.com>
> *Cc:* Grodzovsky, Andrey <Andrey.Grodzovsky@amd.com>; 
> gregkh@linuxfoundation.org <gregkh@linuxfoundation.org>; Kuehling, Felix 
> <Felix.Kuehling@amd.com>; ppaalanen@gmail.com <ppaalanen@gmail.com>; 
> helgaas@kernel.org <helgaas@kernel.org>; Deucher, Alexander 
> <Alexander.Deucher@amd.com>
> *Subject:* [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into 
> early and late
> Some of the stuff in amdgpu_device_fini such as HW interrupts
> disable and pending fences finilization must be done right away on
> pci_remove while most of the stuff which relates to finilizing and
> releasing driver data structures can be kept until
> drm_driver.release hook is called, i.e. when the last device
> reference is dropped.
> 
> v4: Change functions prefix early->hw and late->sw
> 
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 ++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++++++++++++++-------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  7 ++----
>   drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c  | 15 ++++++++++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    | 26 +++++++++++++---------
>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h    |  3 ++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 12 +++++++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c    |  1 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h   |  3 ++-
>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  2 +-
>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  2 +-
>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  2 +-
>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  2 +-
>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  2 +-
>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  2 +-
>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  2 +-
>   drivers/gpu/drm/amd/amdgpu/vega20_ih.c     |  2 +-
>   17 files changed, 79 insertions(+), 36 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index 1af2fa1591fd..fddb82897e5d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1073,7 +1073,9 @@ static inline struct amdgpu_device 
> *amdgpu_ttm_adev(struct ttm_device *bdev)
> 
>   int amdgpu_device_init(struct amdgpu_device *adev,
>                          uint32_t flags);
> -void amdgpu_device_fini(struct amdgpu_device *adev);
> +void amdgpu_device_fini_hw(struct amdgpu_device *adev);
> +void amdgpu_device_fini_sw(struct amdgpu_device *adev);
> +
>   int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
> 
>   void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
> @@ -1289,6 +1291,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device 
> *dev);
>   int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file 
> *file_priv);
>   void amdgpu_driver_postclose_kms(struct drm_device *dev,
>                                    struct drm_file *file_priv);
> +void amdgpu_driver_release_kms(struct drm_device *dev);
> +
>   int amdgpu_device_ip_suspend(struct amdgpu_device *adev);
>   int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);
>   int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 6447cd6ca5a8..8d22b79fc1cd 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -3590,14 +3590,12 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>    * Tear down the driver info (all asics).
>    * Called at driver shutdown.
>    */
> -void amdgpu_device_fini(struct amdgpu_device *adev)
> +void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>   {
>           dev_info(adev->dev, "amdgpu: finishing device.\n");
>           flush_delayed_work(&adev->delayed_init_work);
>           adev->shutdown = true;
> 
> -       kfree(adev->pci_state);
> -
>           /* make sure IB test finished before entering exclusive mode
>            * to avoid preemption on IB test
>            * */
> @@ -3614,11 +3612,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
>                   else
>                           drm_atomic_helper_shutdown(adev_to_drm(adev));
>           }
> -       amdgpu_fence_driver_fini(adev);
> +       amdgpu_fence_driver_fini_hw(adev);
> +
>           if (adev->pm_sysfs_en)
>                   amdgpu_pm_sysfs_fini(adev);
> +       if (adev->ucode_sysfs_en)
> +               amdgpu_ucode_sysfs_fini(adev);
> +       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
> +
> +
>           amdgpu_fbdev_fini(adev);
> +
> +       amdgpu_irq_fini_hw(adev);
> +}
> +
> +void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> +{
>           amdgpu_device_ip_fini(adev);
> +       amdgpu_fence_driver_fini_sw(adev);
>           release_firmware(adev->firmware.gpu_info_fw);
>           adev->firmware.gpu_info_fw = NULL;
>           adev->accel_working = false;
> @@ -3647,14 +3658,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
>           adev->rmmio = NULL;
>           amdgpu_device_doorbell_fini(adev);
> 
> -       if (adev->ucode_sysfs_en)
> -               amdgpu_ucode_sysfs_fini(adev);
> -
> -       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>           if (IS_ENABLED(CONFIG_PERF_EVENTS))
>                   amdgpu_pmu_fini(adev);
>           if (adev->mman.discovery_bin)
>                   amdgpu_discovery_fini(adev);
> +
> +       kfree(adev->pci_state);
> +
>   }
> 
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> index 671ec1002230..54cb5ee2f563 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
> @@ -1249,14 +1249,10 @@ amdgpu_pci_remove(struct pci_dev *pdev)
>   {
>           struct drm_device *dev = pci_get_drvdata(pdev);
> 
> -#ifdef MODULE
> -       if (THIS_MODULE->state != MODULE_STATE_GOING)
> -#endif
> -               DRM_ERROR("Hotplug removal is not supported\n");
>           drm_dev_unplug(dev);
>           amdgpu_driver_unload_kms(dev);
> +
>           pci_disable_device(pdev);
> -       pci_set_drvdata(pdev, NULL);
>   }
> 
>   static void
> @@ -1587,6 +1583,7 @@ static const struct drm_driver amdgpu_kms_driver = {
>           .dumb_create = amdgpu_mode_dumb_create,
>           .dumb_map_offset = amdgpu_mode_dumb_mmap,
>           .fops = &amdgpu_driver_kms_fops,
> +       .release = &amdgpu_driver_release_kms,
> 
>           .prime_handle_to_fd = drm_gem_prime_handle_to_fd,
>           .prime_fd_to_handle = drm_gem_prime_fd_to_handle,
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> index 8e0a5650d383..34d51e962799 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
> @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev)
>    *
>    * Tear down the fence driver for all possible rings (all asics).
>    */
> -void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
> +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
>   {
>           unsigned i, j;
>           int r;
> @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device 
> *adev)
>                   if (!ring->no_scheduler)
>                           drm_sched_fini(&ring->sched);
>                   del_timer_sync(&ring->fence_drv.fallback_timer);
> +       }
> +}
> +
> +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev)
> +{
> +       unsigned int i, j;
> +
> +       for (i = 0; i < AMDGPU_MAX_RINGS; i++) {
> +               struct amdgpu_ring *ring = adev->rings[i];
> +
> +               if (!ring || !ring->fence_drv.initialized)
> +                       continue;
> +
>                   for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j)
>                           dma_fence_put(ring->fence_drv.fences[j]);
>                   kfree(ring->fence_drv.fences);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> index afbbec82a289..63e815c27585 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> @@ -49,6 +49,7 @@
>   #include <drm/drm_irq.h>
>   #include <drm/drm_vblank.h>
>   #include <drm/amdgpu_drm.h>
> +#include <drm/drm_drv.h>
>   #include "amdgpu.h"
>   #include "amdgpu_ih.h"
>   #include "atom.h"
> @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
>           return 0;
>   }
> 
> +
> +void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
> +{
> +       if (adev->irq.installed) {
> +               drm_irq_uninstall(&adev->ddev);
> +               adev->irq.installed = false;
> +               if (adev->irq.msi_enabled)
> +                       pci_free_irq_vectors(adev->pdev);
> +
> +               if (!amdgpu_device_has_dc_support(adev))
> +                       flush_work(&adev->hotplug_work);
> +       }
> +}
> +
>   /**
>    * amdgpu_irq_fini - shut down interrupt handling
>    *
> @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev)
>    * functionality, shuts down vblank, hotplug and reset interrupt handling,
>    * turns off interrupts from all sources (all ASICs).
>    */
> -void amdgpu_irq_fini(struct amdgpu_device *adev)
> +void amdgpu_irq_fini_sw(struct amdgpu_device *adev)
>   {
>           unsigned i, j;
> 
> -       if (adev->irq.installed) {
> -               drm_irq_uninstall(adev_to_drm(adev));
> -               adev->irq.installed = false;
> -               if (adev->irq.msi_enabled)
> -                       pci_free_irq_vectors(adev->pdev);
> -               if (!amdgpu_device_has_dc_support(adev))
> -                       flush_work(&adev->hotplug_work);
> -       }
> -
>           for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) {
>                   if (!adev->irq.client[i].sources)
>                           continue;
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> index ac527e5deae6..392a7324e2b1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
> @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev);
>   irqreturn_t amdgpu_irq_handler(int irq, void *arg);
> 
>   int amdgpu_irq_init(struct amdgpu_device *adev);
> -void amdgpu_irq_fini(struct amdgpu_device *adev);
> +void amdgpu_irq_fini_sw(struct amdgpu_device *adev);
> +void amdgpu_irq_fini_hw(struct amdgpu_device *adev);
>   int amdgpu_irq_add_id(struct amdgpu_device *adev,
>                         unsigned client_id, unsigned src_id,
>                         struct amdgpu_irq_src *source);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> index 64beb3399604..1af3fba7bfd4 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
> @@ -29,6 +29,7 @@
>   #include "amdgpu.h"
>   #include <drm/drm_debugfs.h>
>   #include <drm/amdgpu_drm.h>
> +#include <drm/drm_drv.h>
>   #include "amdgpu_uvd.h"
>   #include "amdgpu_vce.h"
>   #include "atom.h"
> @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev)
>           }
> 
>           amdgpu_acpi_fini(adev);
> -       amdgpu_device_fini(adev);
> +       amdgpu_device_fini_hw(adev);
>   }
> 
>   void amdgpu_register_gpu_instance(struct amdgpu_device *adev)
> @@ -1151,6 +1152,15 @@ void amdgpu_driver_postclose_kms(struct 
> drm_device *dev,
>           pm_runtime_put_autosuspend(dev->dev);
>   }
> 
> +
> +void amdgpu_driver_release_kms(struct drm_device *dev)
> +{
> +       struct amdgpu_device *adev = drm_to_adev(dev);
> +
> +       amdgpu_device_fini_sw(adev);
> +       pci_set_drvdata(adev->pdev, NULL);
> +}
> +
>   /*
>    * VBlank related functions.
>    */
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> index 1fb2a91ad30a..c0a16eac4923 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
> @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev)
>           if (!con)
>                   return 0;
> 
> +
>           /* Need disable ras on all IPs here before ip [hw/sw]fini */
>           amdgpu_ras_disable_all_features(adev, 0);
>           amdgpu_ras_recovery_fini(adev);
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h 
> b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> index 56acec1075ac..0f195f7bf797 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
> @@ -107,7 +107,8 @@ struct amdgpu_fence_driver {
>   };
> 
>   int amdgpu_fence_driver_init(struct amdgpu_device *adev);
> -void amdgpu_fence_driver_fini(struct amdgpu_device *adev);
> +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev);
> +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev);
>   void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
> 
>   int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> index d3745711d55f..183d44a6583c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>           amdgpu_irq_remove_domain(adev);
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> index 307c01301c87..d32743949003 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> @@ -301,7 +301,7 @@ static int cz_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>           amdgpu_irq_remove_domain(adev);
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> index cc957471f31e..da96c6013477 100644
> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> @@ -300,7 +300,7 @@ static int iceland_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>           amdgpu_irq_remove_domain(adev);
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> index f4e4040bbd25..5eea4550b856 100644
> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> @@ -569,7 +569,7 @@ static int navi10_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> index 51880f6ef634..751307f3252c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> 
>           return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> index 249fcbee7871..973d80ec7f6c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> @@ -312,7 +312,7 @@ static int tonga_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>           amdgpu_irq_remove_domain(adev);
> 
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> index 88626d83e07b..2d0094c276ca 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> @@ -523,7 +523,7 @@ static int vega10_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c 
> b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> index 5a3c867d5881..9059b21b079f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
> @@ -558,7 +558,7 @@ static int vega20_ih_sw_fini(void *handle)
>   {
>           struct amdgpu_device *adev = (struct amdgpu_device *)handle;
> 
> -       amdgpu_irq_fini(adev);
> +       amdgpu_irq_fini_sw(adev);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>           amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> -- 
> 2.25.1
> 
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Clijo.lazar%40amd.com%7C71f92dcef6d04dfea28308d90a5818c9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637552195764790324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=fzGS6ZXYffGH0DE%2BEM4jxyB8yDGpYO%2FmT%2FKAtgig1Tw%3D&amp;reserved=0 
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Clijo.lazar%40amd.com%7C71f92dcef6d04dfea28308d90a5818c9%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637552195764790324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=fzGS6ZXYffGH0DE%2BEM4jxyB8yDGpYO%2FmT%2FKAtgig1Tw%3D&amp;reserved=0>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 03/27] drm/amdgpu: Split amdgpu_device_fini into early and late

[Lazar, Lijo]

[-- Attachment #1.1: Type: text/plain, Size: 236 bytes --]

[AMD Official Use Only - Internal Distribution Only]

That looks better to me :) As more things get added, I don't know how long you will be able to hold sw/hw cleanup separate and the name could confuse eventually.

Thanks,
Lijo

[-- Attachment #1.2: Type: text/html, Size: 892 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Christian König]

Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
>
>
> On 2021-04-29 3:18 a.m., Christian König wrote:
>> I need to take another look at this part when I don't have a massive 
>> headache any more.
>>
>> Maybe split the patch set up into different parts, something like:
>> 1. Adding general infrastructure.
>> 2. Making sure all memory is unpolated.
>> 3. Job and fence handling
>
> I am not sure you mean this patch here, maybe another one ?
> Also note you already RBed it.

No what I meant was to send out the patches before this one as #1 and #2.

That is the easier stuff which can easily go into the drm-misc-next or 
amd-staging-drm-next branch.

The scheduler stuff certainly need to go into drm-misc-next.

Christian.

>
> Andrey
>
>>
>> Christian.
>>
>> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>>> Problem: If scheduler is already stopped by the time sched_entity
>>> is released and entity's job_queue not empty I encountred
>>> a hang in drm_sched_entity_flush. This is because 
>>> drm_sched_entity_is_idle
>>> never becomes false.
>>>
>>> Fix: In drm_sched_fini detach all sched_entities from the
>>> scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
>>> Also wakeup all those processes stuck in sched_entity flushing
>>> as the scheduler main thread which wakes them up is stopped by now.
>>>
>>> v2:
>>> Reverse order of drm_sched_rq_remove_entity and marking
>>> s_entity as stopped to prevent reinserion back to rq due
>>> to race.
>>>
>>> v3:
>>> Drop drm_sched_rq_remove_entity, only modify entity->stopped
>>> and check for it in drm_sched_entity_is_idle
>>>
>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>> Reviewed-by: Christian König <christian.koenig@amd.com>
>>> ---
>>>   drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
>>>   drivers/gpu/drm/scheduler/sched_main.c   | 24 
>>> ++++++++++++++++++++++++
>>>   2 files changed, 26 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c 
>>> b/drivers/gpu/drm/scheduler/sched_entity.c
>>> index f0790e9471d1..cb58f692dad9 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
>>> @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct 
>>> drm_sched_entity *entity)
>>>       rmb(); /* for list_empty to work without lock */
>>>       if (list_empty(&entity->list) ||
>>> -        spsc_queue_count(&entity->job_queue) == 0)
>>> +        spsc_queue_count(&entity->job_queue) == 0 ||
>>> +        entity->stopped)
>>>           return true;
>>>       return false;
>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c 
>>> b/drivers/gpu/drm/scheduler/sched_main.c
>>> index 908b0b56032d..ba087354d0a8 100644
>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>> @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
>>>    */
>>>   void drm_sched_fini(struct drm_gpu_scheduler *sched)
>>>   {
>>> +    struct drm_sched_entity *s_entity;
>>> +    int i;
>>> +
>>>       if (sched->thread)
>>>           kthread_stop(sched->thread);
>>> +    for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= 
>>> DRM_SCHED_PRIORITY_MIN; i--) {
>>> +        struct drm_sched_rq *rq = &sched->sched_rq[i];
>>> +
>>> +        if (!rq)
>>> +            continue;
>>> +
>>> +        spin_lock(&rq->lock);
>>> +        list_for_each_entry(s_entity, &rq->entities, list)
>>> +            /*
>>> +             * Prevents reinsertion and marks job_queue as idle,
>>> +             * it will removed from rq in drm_sched_entity_fini
>>> +             * eventually
>>> +             */
>>> +            s_entity->stopped = true;
>>> +        spin_unlock(&rq->lock);
>>> +
>>> +    }
>>> +
>>> +    /* Wakeup everyone stuck in drm_sched_entity_flush for this 
>>> scheduler */
>>> +    wake_up_all(&sched->job_scheduled);
>>> +
>>>       /* Confirm no work left behind accessing device structures */
>>>       cancel_delayed_work_sync(&sched->work_tdr);
>>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
> 
> 
> On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
> > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
> > > 
> > > 
> > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
> > > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
> > > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> > > > > > With this calling drm_dev_unplug will flush and block
> > > > > > all in flight IOCTLs
> > > > > > 
> > > > > > Also, add feature such that if device supports graceful unplug
> > > > > > we enclose entire IOCTL in SRCU critical section.
> > > > > > 
> > > > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > > > > 
> > > > > Nope.
> > > > > 
> > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
> > > 
> > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C1821a19173a84ebae31108d90b41b2fa%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553199084555468%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=d6kXadWHv4CEDgODsm%2FOzIIjIDA9rZDLUuV11MmEU3A%3D&amp;reserved=0
> > > currently in code ?
> > 
> > I forgot about this one, again. Thanks for reminding.
> > 
> > > > > Especially not with an opt-in flag so that it could be shrugged of as a
> > > > > driver hack. Most of these ioctls should have absolutely no problem
> > > > > working after hotunplug.
> > > > > 
> > > > > Also, doing this defeats the point since it pretty much guarantees
> > > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract
> > > > > is that only execbuf (and even that only when userspace has indicated
> > > > > support for non-recoverable hw ctx) is allowed to fail. Anything else
> > > > > might crash userspace.
> > > 
> > > Given that as I pointed above we already fail any IOCTls with -ENODEV
> > > when device is unplugged, it seems those crashes don't happen that
> > > often ? Also, in all my testing I don't think I saw a user space crash
> > > I could attribute to this.
> > 
> > I guess it should be ok.
> 
> What should be ok ?

Your approach, but not your patch. If we go with this let's just lift it
to drm_ioctl() as the default behavior. No driver opt-in flag, because
that's definitely worse than any other approach because we really need to
get rid of driver specific behaviour for generic ioctls, especially
anything a compositor will use directly.

> > My reasons for making this work is both less trouble for userspace (did
> > you test with various wayland compositors out there, not just amdgpu x86
> 
> I didn't - will give it a try.
> 
> > driver?), but also testing.
> > 
> > We still need a bunch of these checks in various places or you'll wait a
> > very long time for a pending modeset or similar to complete. Being able to
> > run that code easily after hotunplug has completed should help a lot with
> > testing.
> > 
> > Plus various drivers already acquired drm_dev_enter/exit and now I wonder
> > whether that was properly tested or not ...
> > 
> > I guess maybe we need a drm module option to disable this check, so that
> > we can exercise the code as if the ioctl has raced with hotunplug at the
> > worst possible moment.
> > 
> > Also atomic is really tricky here: I assume your testing has just done
> > normal synchronous commits, but anything that goes through atomic can be
> > done nonblocking in a separate thread. Which the ioctl catch-all here wont
> > capture.
> 
> Yes, async commit was on my mind and thanks for reminding me. Indeed
> I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
> drm_dev_enter/exit. Note that i have a bunch of patches, all name's
> starting with 'Scope....' that just methodically put all the background
> work items and timers the drivers schedules in drm_dev_enter/exit scope.
> This was supposed to be part of the 'Scope Display code' patch.

That's too much. You still have to arrange that the flip completion event
gets sent out. So it's a bit tricky.

In other places the same problem applies, e.g. probe functions need to
make sure they report "disconnected".

> > > > > You probably need similar (and very precisely defined) rules for amdgpu.
> > > > > And those must definitely exclude any shard ioctls from randomly failing
> > > > > with EIO, because that just kills the box and defeats the point of trying
> > > > > to gracefully handling hotunplug and making sure userspace has a chance of
> > > > > survival. E.g. for atomic everything should continue, including flip
> > > > > completion, but we set all outputs to "disconnected" and send out the
> > > > > uevent. Maybe crtc enabling can fail too, but that can also be handled
> > > > > through the async status we're using to signal DP link failures to
> > > > > userspace.
> > > 
> > > As I pointed before, because of the complexity of the topic I prefer to
> > > take it step by step and solve first for secondary device use case, not
> > > for primary, display attached device.
> > 
> > Yeah makes sense. But then I think the right patch is to roll this out for
> > all drivers, properly justified with existing code. Not behind a driver
> > flag, because with all these different compositors the last thing we want
> > is a proliferation of driver-specific behaviour. That's imo the worst
> > option of all of them and needs to be avoided.
> 
> So this kind of patch would be acceptable to you if I unconditionally
> scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
> I am worried to break other drivers with this, see patch https://cgit.freedesktop.org/~agrodzov/linux/commit/?h=drm-misc-next&id=f0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6
> Before setting drm_dev_unplug I go through a whole process of signalling
> all possible fences in the system which some one some where might be
> waiting on. My concern is that in the absence of HW those fences won't
> signal and so unless I signal them myself srcu_synchrionize in
> drm_dev_unplug will hang waiting for any such code scoped by
> drm_dev_enter/exit.

Uh right. I forgot about this.

Which would kinda mean the top level scope is maybe not the best idea, and
perhaps we should indeed drill it down. But then the testing issue
definitely gets a lot worse.

So what if we'd push that drm_dev_is_unplugged check down into ioctls?
Then we can make a case-by case decision whether it should be converted to
drm_dev_enter/exit, needs to be pushed down further into drivers (due to
fence wait issues) or other concerns?

Also I guess we need to have a subsystem wide rule on whether you need to
force complete all fences before you call drm_dev_unplug, or afterwards.
If we have mixed behaviour on this there will be disappointment. And since
hotunplug and dma_fence completion are both userspace visible that
inconsistency might have bigger impact.

This is all very tricky indeed :-/

btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the
  DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to
  handle/where (with the aim to have the least amount of code flow
  different during hotunplug vs after hotunplug has finished, to reduce
  testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default

This would have us left with render ioctls, and I think the defensive
assumption there is that they're all hotunplug safe. We might hang on a
fence wait, but that's fixable, and it's better than blowing up on a
use-after-free security bug.

Thoughts?

It is unfortunately even more work until we've reached the goal, but I
think it's safest and most flexible approach overall.

Cheers, Daniel

> 
> Andrey
> 
> > 
> > Cheers, Daniel
> > 
> > 
> > > 
> > > > > 
> > > > > I guess we should clarify this in the hotunplug doc?
> > > 
> > > Agree
> > > 
> > > > 
> > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
> > > > really make no sense, and where we're rather confident that all properly
> > > > implemented userspace will gracefully handle failures. Like a modeset, or
> > > > opening a device, or trying to import a dma-buf or stuff like that which
> > > > can already fail in normal operation for any kind of reason.
> > > > 
> > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
> > > > after hotunplug.
> > > 
> > > As I pointed above, this a bit confuses me given that we already do
> > > blanker rejection of IOCTLs if device is unplugged.
> > 
> > Well I'm confused about this too :-/
> > 
> > > > And then there's the middle ground, like doing a pageflip or buffer flush,
> > > > which I guess some userspace might handle, but risky to inflict those
> > > > consequences on them. atomic modeset is especially fun since depending
> > > > what you're doing it can be both "failures expected" and "failures not
> > > > really expected in normal operation".
> > > > 
> > > > Also, this really should be consistent across drivers, not solved with a
> > > > driver flag for every possible combination.
> > > > 
> > > > If you look at the current hotunplug kms drivers, they have
> > > > drm_dev_enter/exit sprinkled in specific hw callback functions because of
> > > > the above problems. But maybe it makes sense to change things in a few
> > > > cases. But then we should do it across the board.
> > > 
> > > So as I understand your preferred approach is that I scope any back_end, HW
> > > specific function with drm_dev_enter/exit because that where MMIO
> > > access takes place. But besides explicit MMIO access thorough
> > > register accessors in the HW back-end there is also indirect MMIO access
> > > taking place throughout the code in the driver because of various VRAM
> > > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
> > > access is spread all over in the driver and even in mid-layers such as
> > > TTM and not limited to HW back-end functions. It means it's much harder
> > > to spot such places to surgically scope them with drm_dev_enter/exit and
> > > also that any new such code introduced will immediately break hot unplug
> > > because the developers can't be expected to remember making their code
> > > robust to this specific use case. That why when we discussed internally
> > > what approach to take to protecting code with drm_dev_enter/exit we
> > > opted for using the widest available scope.
> > 
> > The thing is, you kinda have to anyway. There's enormous amounts of
> > asynchronous processing going on. E.g. nonblocking atomic commits also do
> > ttm unpinning and fun stuff like that, which if you sync things wrong can
> > happen way late. So the door for bad fallout is wide open :-(
> > 
> > I'm not sure where the right tradeoff is to make sure we catch them all,
> > and can make sure with testing that we've indeed caught them all.
> > -Daniel
> > 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Andrey Grodzovsky]

On 2021-04-30 2:47 a.m., Christian König wrote:
> 
> 
> Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
>>
>>
>> On 2021-04-29 3:18 a.m., Christian König wrote:
>>> I need to take another look at this part when I don't have a massive 
>>> headache any more.
>>>
>>> Maybe split the patch set up into different parts, something like:
>>> 1. Adding general infrastructure.
>>> 2. Making sure all memory is unpolated.
>>> 3. Job and fence handling
>>
>> I am not sure you mean this patch here, maybe another one ?
>> Also note you already RBed it.
> 
> No what I meant was to send out the patches before this one as #1 and #2.
> 
> That is the easier stuff which can easily go into the drm-misc-next or 
> amd-staging-drm-next branch.
> 
> The scheduler stuff certainly need to go into drm-misc-next.
> 
> Christian.

Got you. I am fine with it. What we have here is a working hot-unplug
code but, one with potential use after free MMIO ranges frpom the zombie
device. The followup patches after this patch are all about preventing
this and so the patch-set up until this patch including, is functional
on it's own. While it's necessary to solve the above issue, it's has
complications as can be seen from the discussion with Daniel on later
patch in this series. Still, in my opinion it's better to rollout some
initial support to hot-unplug without use after free protection then
having no support for hot-unplug at all. It will also make the merge
work easier as I need to constantly rebase the patches on top latest
kernel and solve new regressions.

Daniel - given the arguments above can you sound your opinion on this
approach ?

Andrey
> 
>>
>> Andrey
>>
>>>
>>> Christian.
>>>
>>> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>>>> Problem: If scheduler is already stopped by the time sched_entity
>>>> is released and entity's job_queue not empty I encountred
>>>> a hang in drm_sched_entity_flush. This is because 
>>>> drm_sched_entity_is_idle
>>>> never becomes false.
>>>>
>>>> Fix: In drm_sched_fini detach all sched_entities from the
>>>> scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
>>>> Also wakeup all those processes stuck in sched_entity flushing
>>>> as the scheduler main thread which wakes them up is stopped by now.
>>>>
>>>> v2:
>>>> Reverse order of drm_sched_rq_remove_entity and marking
>>>> s_entity as stopped to prevent reinserion back to rq due
>>>> to race.
>>>>
>>>> v3:
>>>> Drop drm_sched_rq_remove_entity, only modify entity->stopped
>>>> and check for it in drm_sched_entity_is_idle
>>>>
>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>> Reviewed-by: Christian König <christian.koenig@amd.com>
>>>> ---
>>>>   drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
>>>>   drivers/gpu/drm/scheduler/sched_main.c   | 24 
>>>> ++++++++++++++++++++++++
>>>>   2 files changed, 26 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c 
>>>> b/drivers/gpu/drm/scheduler/sched_entity.c
>>>> index f0790e9471d1..cb58f692dad9 100644
>>>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
>>>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
>>>> @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct 
>>>> drm_sched_entity *entity)
>>>>       rmb(); /* for list_empty to work without lock */
>>>>       if (list_empty(&entity->list) ||
>>>> -        spsc_queue_count(&entity->job_queue) == 0)
>>>> +        spsc_queue_count(&entity->job_queue) == 0 ||
>>>> +        entity->stopped)
>>>>           return true;
>>>>       return false;
>>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c 
>>>> b/drivers/gpu/drm/scheduler/sched_main.c
>>>> index 908b0b56032d..ba087354d0a8 100644
>>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>>> @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
>>>>    */
>>>>   void drm_sched_fini(struct drm_gpu_scheduler *sched)
>>>>   {
>>>> +    struct drm_sched_entity *s_entity;
>>>> +    int i;
>>>> +
>>>>       if (sched->thread)
>>>>           kthread_stop(sched->thread);
>>>> +    for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= 
>>>> DRM_SCHED_PRIORITY_MIN; i--) {
>>>> +        struct drm_sched_rq *rq = &sched->sched_rq[i];
>>>> +
>>>> +        if (!rq)
>>>> +            continue;
>>>> +
>>>> +        spin_lock(&rq->lock);
>>>> +        list_for_each_entry(s_entity, &rq->entities, list)
>>>> +            /*
>>>> +             * Prevents reinsertion and marks job_queue as idle,
>>>> +             * it will removed from rq in drm_sched_entity_fini
>>>> +             * eventually
>>>> +             */
>>>> +            s_entity->stopped = true;
>>>> +        spin_unlock(&rq->lock);
>>>> +
>>>> +    }
>>>> +
>>>> +    /* Wakeup everyone stuck in drm_sched_entity_flush for this 
>>>> scheduler */
>>>> +    wake_up_all(&sched->job_scheduled);
>>>> +
>>>>       /* Confirm no work left behind accessing device structures */
>>>>       cancel_delayed_work_sync(&sched->work_tdr);
>>>
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
> On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
>>
>>
>> On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
>>> On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
>>>>
>>>>
>>>> On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
>>>>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
>>>>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
>>>>>>> With this calling drm_dev_unplug will flush and block
>>>>>>> all in flight IOCTLs
>>>>>>>
>>>>>>> Also, add feature such that if device supports graceful unplug
>>>>>>> we enclose entire IOCTL in SRCU critical section.
>>>>>>>
>>>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>>>
>>>>>> Nope.
>>>>>>
>>>>>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
>>>>
>>>> Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cf4c0568093cc462f625808d90bc23a3c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553751106596888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PPKrQYBrgRMjpwlL0r8n5zenIhQMFWc6gniHgUTxTAY%3D&amp;reserved=0
>>>> currently in code ?
>>>
>>> I forgot about this one, again. Thanks for reminding.
>>>
>>>>>> Especially not with an opt-in flag so that it could be shrugged of as a
>>>>>> driver hack. Most of these ioctls should have absolutely no problem
>>>>>> working after hotunplug.
>>>>>>
>>>>>> Also, doing this defeats the point since it pretty much guarantees
>>>>>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract
>>>>>> is that only execbuf (and even that only when userspace has indicated
>>>>>> support for non-recoverable hw ctx) is allowed to fail. Anything else
>>>>>> might crash userspace.
>>>>
>>>> Given that as I pointed above we already fail any IOCTls with -ENODEV
>>>> when device is unplugged, it seems those crashes don't happen that
>>>> often ? Also, in all my testing I don't think I saw a user space crash
>>>> I could attribute to this.
>>>
>>> I guess it should be ok.
>>
>> What should be ok ?
> 
> Your approach, but not your patch. If we go with this let's just lift it
> to drm_ioctl() as the default behavior. No driver opt-in flag, because
> that's definitely worse than any other approach because we really need to
> get rid of driver specific behaviour for generic ioctls, especially
> anything a compositor will use directly.
> 
>>> My reasons for making this work is both less trouble for userspace (did
>>> you test with various wayland compositors out there, not just amdgpu x86
>>
>> I didn't - will give it a try.

Weston worked without crashes, run the egl tester cube there.

>>
>>> driver?), but also testing.
>>>
>>> We still need a bunch of these checks in various places or you'll wait a
>>> very long time for a pending modeset or similar to complete. Being able to
>>> run that code easily after hotunplug has completed should help a lot with
>>> testing.
>>>
>>> Plus various drivers already acquired drm_dev_enter/exit and now I wonder
>>> whether that was properly tested or not ...
>>>
>>> I guess maybe we need a drm module option to disable this check, so that
>>> we can exercise the code as if the ioctl has raced with hotunplug at the
>>> worst possible moment.
>>>
>>> Also atomic is really tricky here: I assume your testing has just done
>>> normal synchronous commits, but anything that goes through atomic can be
>>> done nonblocking in a separate thread. Which the ioctl catch-all here wont
>>> capture.
>>
>> Yes, async commit was on my mind and thanks for reminding me. Indeed
>> I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
>> drm_dev_enter/exit. Note that i have a bunch of patches, all name's
>> starting with 'Scope....' that just methodically put all the background
>> work items and timers the drivers schedules in drm_dev_enter/exit scope.
>> This was supposed to be part of the 'Scope Display code' patch.
> 
> That's too much. You still have to arrange that the flip completion event
> gets sent out. So it's a bit tricky.
> 
> In other places the same problem applies, e.g. probe functions need to
> make sure they report "disconnected".

I see, well, this is all part of KMS support which I defer for now
anyway. Will tackle it then.

> 
>>>>>> You probably need similar (and very precisely defined) rules for amdgpu.
>>>>>> And those must definitely exclude any shard ioctls from randomly failing
>>>>>> with EIO, because that just kills the box and defeats the point of trying
>>>>>> to gracefully handling hotunplug and making sure userspace has a chance of
>>>>>> survival. E.g. for atomic everything should continue, including flip
>>>>>> completion, but we set all outputs to "disconnected" and send out the
>>>>>> uevent. Maybe crtc enabling can fail too, but that can also be handled
>>>>>> through the async status we're using to signal DP link failures to
>>>>>> userspace.
>>>>
>>>> As I pointed before, because of the complexity of the topic I prefer to
>>>> take it step by step and solve first for secondary device use case, not
>>>> for primary, display attached device.
>>>
>>> Yeah makes sense. But then I think the right patch is to roll this out for
>>> all drivers, properly justified with existing code. Not behind a driver
>>> flag, because with all these different compositors the last thing we want
>>> is a proliferation of driver-specific behaviour. That's imo the worst
>>> option of all of them and needs to be avoided.
>>
>> So this kind of patch would be acceptable to you if I unconditionally
>> scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
>> I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cf4c0568093cc462f625808d90bc23a3c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553751106596888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2F3Jq6SvTm%2BZX7AVpaxEepfOj0C3O7%2Bo2Wm3y0gxrmKI%3D&amp;reserved=0
>> Before setting drm_dev_unplug I go through a whole process of signalling
>> all possible fences in the system which some one some where might be
>> waiting on. My concern is that in the absence of HW those fences won't
>> signal and so unless I signal them myself srcu_synchrionize in
>> drm_dev_unplug will hang waiting for any such code scoped by
>> drm_dev_enter/exit.
> 
> Uh right. I forgot about this.
> 
> Which would kinda mean the top level scope is maybe not the best idea, and
> perhaps we should indeed drill it down. But then the testing issue
> definitely gets a lot worse.
> 
> So what if we'd push that drm_dev_is_unplugged check down into ioctls?
> Then we can make a case-by case decision whether it should be converted to
> drm_dev_enter/exit, needs to be pushed down further into drivers (due to
> fence wait issues) or other concerns?
> 
> Also I guess we need to have a subsystem wide rule on whether you need to
> force complete all fences before you call drm_dev_unplug, or afterwards.

I don't see how you can handle it afterwards. If a thread is stuck in
dma_fence_wait in non interruptible wait (any kernel thread) and with no
timeout there is nothing you can do to stop the wait. Any such code
scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
The only way then is to preemptively force signal all such fences before
calling drm_dev_unplug - as I do in the above mentioned patch.

> If we have mixed behaviour on this there will be disappointment. And since
> hotunplug and dma_fence completion are both userspace visible that
> inconsistency might have bigger impact.
> 
> This is all very tricky indeed :-/
> 
> btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
> with that: We could do the same trick we've done for DRM_UNLOCKED:
> - drm_dev_enter/exit is called for any ioctl that has not set the
>    DRM_HOTUNPLUG_SAFE flag
> - for drm core ioctls we push them into all ioctls and decide how to
>    handle/where (with the aim to have the least amount of code flow
>    different during hotunplug vs after hotunplug has finished, to reduce
>    testing scope)
> - then we make DRM_HOTUNPLUG_SAFE the implied default
> 
> This would have us left with render ioctls, and I think the defensive
> assumption there is that they're all hotunplug safe. We might hang on a
> fence wait, but that's fixable, and it's better than blowing up on a
> use-after-free security bug.
> 
> Thoughts?

I don't fully see a difference between the approach described above and
the full drill down to each driver and even within the driver, to the HW
back-ends - what criteria I would use to decide if for a given IOCTL i
scope with drm_dev_enter/exit at the highest level while for another
i go all the way down ? If we would agree that signaling the fences
preemptively before engaging drm_dev_unplug is generically the right
approach maybe we can then scope drm_ioctl unconditionally with
drm_dev_enter/exit and then for each driver go through the same process
I do for amdgpu - writing driver specific function which takes care of
all the fences. We could then just create a drm callback which would
be called from drm_ioctl before drm_dev_unplug is called.

Andrey

> 
> It is unfortunately even more work until we've reached the goal, but I
> think it's safest and most flexible approach overall.
> 
> Cheers, Daniel
> 
>>
>> Andrey
>>
>>>
>>> Cheers, Daniel
>>>
>>>
>>>>
>>>>>>
>>>>>> I guess we should clarify this in the hotunplug doc?
>>>>
>>>> Agree
>>>>
>>>>>
>>>>> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
>>>>> really make no sense, and where we're rather confident that all properly
>>>>> implemented userspace will gracefully handle failures. Like a modeset, or
>>>>> opening a device, or trying to import a dma-buf or stuff like that which
>>>>> can already fail in normal operation for any kind of reason.
>>>>>
>>>>> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
>>>>> after hotunplug.
>>>>
>>>> As I pointed above, this a bit confuses me given that we already do
>>>> blanker rejection of IOCTLs if device is unplugged.
>>>
>>> Well I'm confused about this too :-/
>>>
>>>>> And then there's the middle ground, like doing a pageflip or buffer flush,
>>>>> which I guess some userspace might handle, but risky to inflict those
>>>>> consequences on them. atomic modeset is especially fun since depending
>>>>> what you're doing it can be both "failures expected" and "failures not
>>>>> really expected in normal operation".
>>>>>
>>>>> Also, this really should be consistent across drivers, not solved with a
>>>>> driver flag for every possible combination.
>>>>>
>>>>> If you look at the current hotunplug kms drivers, they have
>>>>> drm_dev_enter/exit sprinkled in specific hw callback functions because of
>>>>> the above problems. But maybe it makes sense to change things in a few
>>>>> cases. But then we should do it across the board.
>>>>
>>>> So as I understand your preferred approach is that I scope any back_end, HW
>>>> specific function with drm_dev_enter/exit because that where MMIO
>>>> access takes place. But besides explicit MMIO access thorough
>>>> register accessors in the HW back-end there is also indirect MMIO access
>>>> taking place throughout the code in the driver because of various VRAM
>>>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>>>> access is spread all over in the driver and even in mid-layers such as
>>>> TTM and not limited to HW back-end functions. It means it's much harder
>>>> to spot such places to surgically scope them with drm_dev_enter/exit and
>>>> also that any new such code introduced will immediately break hot unplug
>>>> because the developers can't be expected to remember making their code
>>>> robust to this specific use case. That why when we discussed internally
>>>> what approach to take to protecting code with drm_dev_enter/exit we
>>>> opted for using the widest available scope.
>>>
>>> The thing is, you kinda have to anyway. There's enormous amounts of
>>> asynchronous processing going on. E.g. nonblocking atomic commits also do
>>> ttm unpinning and fun stuff like that, which if you sync things wrong can
>>> happen way late. So the door for bad fallout is wide open :-(
>>>
>>> I'm not sure where the right tradeoff is to make sure we catch them all,
>>> and can make sure with testing that we've indeed caught them all.
>>> -Daniel
>>>
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Andrey Grodzovsky]

On 2021-04-29 11:13 p.m., Alex Deucher wrote:
> On Wed, Apr 28, 2021 at 11:13 AM Andrey Grodzovsky
> <andrey.grodzovsky@amd.com> wrote:
>>
>> Handle all DMA IOMMU gropup related dependencies before the
>> group is removed.
>>
>> v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
>>
>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>>   14 files changed, 56 insertions(+), 16 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index fddb82897e5d..30a24db5f4d1 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>>
>>          bool                            in_pci_err_recovery;
>>          struct pci_saved_state          *pci_state;
>> +
>> +       struct list_head                device_bo_list;
>>   };
>>
>>   static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 46d646c40338..91594ddc2459 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -70,6 +70,7 @@
>>   #include <drm/task_barrier.h>
>>   #include <linux/pm_runtime.h>
>>
>> +
>>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
>> @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = {
>>          NULL
>>   };
>>
>> -
>>   /**
>>    * amdgpu_device_init - initialize the driver
>>    *
>> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>>
>>          INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>>
>> +       INIT_LIST_HEAD(&adev->device_bo_list);
>> +
>>          adev->gfx.gfx_off_req_count = 1;
>>          adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>>
>> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>>          return r;
>>   }
>>
>> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
> 
> Prefix this with amdgpu_device for consistency.  E.g.,
> amdgpu_device_clear_dma_mappings()
> 
>> +{
>> +       struct amdgpu_bo *bo = NULL;
>> +
>> +       /*
>> +        * Unmaps all DMA mappings before device will be removed from it's
>> +        * IOMMU group otherwise in case of IOMMU enabled system a crash
>> +        * will happen.
>> +        */
>> +
>> +       spin_lock(&adev->mman.bdev.lru_lock);
>> +       while (!list_empty(&adev->device_bo_list)) {
>> +               bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);
>> +               list_del_init(&bo->bo);
>> +               spin_unlock(&adev->mman.bdev.lru_lock);
>> +               if (bo->tbo.ttm)
>> +                       ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
>> +               spin_lock(&adev->mman.bdev.lru_lock);
>> +       }
>> +       spin_unlock(&adev->mman.bdev.lru_lock);
>> +}
>> +
>>   /**
>>    * amdgpu_device_fini - tear down the driver
>>    *
>> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>>                  amdgpu_ucode_sysfs_fini(adev);
>>          sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>>
>> -
>>          amdgpu_fbdev_fini(adev);
>>
>>          amdgpu_irq_fini_hw(adev);
>>
>>          amdgpu_device_ip_fini_early(adev);
>> +
>> +       amdgpu_clear_dma_mappings(adev);
>> +
>> +       amdgpu_gart_dummy_page_fini(adev);
>>   }
>>
>>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> index fde2d899b2c4..49cdcaf8512d 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
>>    *
>>    * Frees the dummy page used by the driver (all asics).
>>    */
>> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>   {
>>          if (!adev->dummy_page_addr)
>>                  return;
>> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>>          vfree(adev->gart.pages);
>>          adev->gart.pages = NULL;
>>   #endif
>> -       amdgpu_gart_dummy_page_fini(adev);
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> index afa2e2877d87..5678d9c105ab 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev);
>>   void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>>   int amdgpu_gart_init(struct amdgpu_device *adev);
>>   void amdgpu_gart_fini(struct amdgpu_device *adev);
>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>>   int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>>                         int pages);
>>   int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> index 63e815c27585..a922154953a7 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
>>                  if (!amdgpu_device_has_dc_support(adev))
>>                          flush_work(&adev->hotplug_work);
>>          }
>> +
>> +       if (adev->irq.ih_soft.ring)
>> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>> +       if (adev->irq.ih.ring)
>> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>> +       if (adev->irq.ih1.ring)
>> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> +       if (adev->irq.ih2.ring)
>> +               amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>   }
>>
>>   /**
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> index 485f249d063a..62d829f5e62c 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
>>                  list_del_init(&bo->shadow_list);
>>                  mutex_unlock(&adev->shadow_list_lock);
>>          }
>> -       amdgpu_bo_unref(&bo->parent);
>>
>> +
>> +       spin_lock(&adev->mman.bdev.lru_lock);
>> +       list_del(&bo->bo);
>> +       spin_unlock(&adev->mman.bdev.lru_lock);
>> +
>> +       amdgpu_bo_unref(&bo->parent);
>>          kfree(bo->metadata);
>>          kfree(bo);
>>   }
>> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
>>          if (bp->type == ttm_bo_type_device)
>>                  bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>>
>> +       INIT_LIST_HEAD(&bo->bo);
>> +
>> +       spin_lock(&adev->mman.bdev.lru_lock);
>> +       list_add_tail(&bo->bo, &adev->device_bo_list);
>> +       spin_unlock(&adev->mman.bdev.lru_lock);
>> +
>>          return 0;
>>
>>   fail_unreserve:
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> index 9ac37569823f..5ae8555ef275 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>>          struct list_head                shadow_list;
>>
>>          struct kgd_mem                  *kfd_bo;
>> +
>> +       struct list_head                bo;
>>   };
>>
>>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> index 183d44a6583c..df385ffc9768 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>>          struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>
>>          amdgpu_irq_fini_sw(adev);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>          amdgpu_irq_remove_domain(adev);
>>
>>          return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> index d32743949003..b8c47e0cf37a 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>>          struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>
>>          amdgpu_irq_fini_sw(adev);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>          amdgpu_irq_remove_domain(adev);
>>
>>          return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> index da96c6013477..ddfe4eaeea05 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>>          struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>
>>          amdgpu_irq_fini_sw(adev);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>          amdgpu_irq_remove_domain(adev);
>>
>>          return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> index 5eea4550b856..e171a9e78544 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>>
>>          amdgpu_irq_fini_sw(adev);
>>          amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> 
> Shouldn't the soft ring be removed as well?

Right, thanks for noticing.

> 
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>
>>          return 0;
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> index 751307f3252c..9a24f17a5750 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>>          struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>
>>          amdgpu_irq_fini_sw(adev);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>
>>          return 0;
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> index 973d80ec7f6c..b08905d1c00f 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>>          struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>
>>          amdgpu_irq_fini_sw(adev);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>          amdgpu_irq_remove_domain(adev);
>>
>>          return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> index 2d0094c276ca..8c8abc00f710 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>>
>>          amdgpu_irq_fini_sw(adev);
>>          amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> 
> Same here?

Yep.

Andrey

> 
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> -       amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>
>>          return 0;
>>   }
>> --
>> 2.25.1
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C6b1f10b7bfb5491e025008d90b86044a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553492499443932%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WagbcvZG9xsaX6cKlMG%2FzMD%2FlYQlUSgxSQjnUHB8Myc%3D&amp;reserved=0
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Andrey Grodzovsky]

On 2021-04-29 3:08 a.m., Christian König wrote:
> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>> Handle all DMA IOMMU gropup related dependencies before the
>> group is removed.
>>
>> v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
> 
> Maybe split that up into more patches.
> 
>>
>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>>   14 files changed, 56 insertions(+), 16 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index fddb82897e5d..30a24db5f4d1 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>>       bool                            in_pci_err_recovery;
>>       struct pci_saved_state          *pci_state;
>> +
>> +    struct list_head                device_bo_list;
>>   };
>>   static inline struct amdgpu_device *drm_to_adev(struct drm_device 
>> *ddev)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 46d646c40338..91594ddc2459 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -70,6 +70,7 @@
>>   #include <drm/task_barrier.h>
>>   #include <linux/pm_runtime.h>
>> +
>>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
>> @@ -3211,7 +3212,6 @@ static const struct attribute 
>> *amdgpu_dev_attributes[] = {
>>       NULL
>>   };
>> -
>>   /**
>>    * amdgpu_device_init - initialize the driver
>>    *
>> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>>       INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>> +    INIT_LIST_HEAD(&adev->device_bo_list);
>> +
>>       adev->gfx.gfx_off_req_count = 1;
>>       adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>>       return r;
>>   }
>> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
>> +{
>> +    struct amdgpu_bo *bo = NULL;
>> +
>> +    /*
>> +     * Unmaps all DMA mappings before device will be removed from it's
>> +     * IOMMU group otherwise in case of IOMMU enabled system a crash
>> +     * will happen.
>> +     */
>> +
>> +    spin_lock(&adev->mman.bdev.lru_lock);
>> +    while (!list_empty(&adev->device_bo_list)) {
>> +        bo = list_first_entry(&adev->device_bo_list, struct 
>> amdgpu_bo, bo);
>> +        list_del_init(&bo->bo);
>> +        spin_unlock(&adev->mman.bdev.lru_lock);
>> +        if (bo->tbo.ttm)
>> +            ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
>> +        spin_lock(&adev->mman.bdev.lru_lock);
>> +    }
>> +    spin_unlock(&adev->mman.bdev.lru_lock);
> 
> Can you try to use the same approach as amdgpu_gtt_mgr_recover() instead 
> of adding something to the BO?
> 
> Christian.

Are you sure that dma mappings limit themself only to GTT BOs
which have allocated mm nodes ? Otherwsie we will crash and burn
on missing IOMMU group when unampping post device remove.
Problem for me to test this as in 5.12 kernel I don't crash even
when removing this entire patch.  Looks like iommu_dma_unmap_page
was changed since 5.9 when I introdiced this patch.

Andrey

> 
>> +}
>> +
>>   /**
>>    * amdgpu_device_fini - tear down the driver
>>    *
>> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct 
>> amdgpu_device *adev)
>>           amdgpu_ucode_sysfs_fini(adev);
>>       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>> -
>>       amdgpu_fbdev_fini(adev);
>>       amdgpu_irq_fini_hw(adev);
>>       amdgpu_device_ip_fini_early(adev);
>> +
>> +    amdgpu_clear_dma_mappings(adev);
>> +
>> +    amdgpu_gart_dummy_page_fini(adev);
>>   }
>>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> index fde2d899b2c4..49cdcaf8512d 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct 
>> amdgpu_device *adev)
>>    *
>>    * Frees the dummy page used by the driver (all asics).
>>    */
>> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>   {
>>       if (!adev->dummy_page_addr)
>>           return;
>> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>>       vfree(adev->gart.pages);
>>       adev->gart.pages = NULL;
>>   #endif
>> -    amdgpu_gart_dummy_page_fini(adev);
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> index afa2e2877d87..5678d9c105ab 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device 
>> *adev);
>>   void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>>   int amdgpu_gart_init(struct amdgpu_device *adev);
>>   void amdgpu_gart_fini(struct amdgpu_device *adev);
>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>>   int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>>                  int pages);
>>   int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> index 63e815c27585..a922154953a7 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
>>           if (!amdgpu_device_has_dc_support(adev))
>>               flush_work(&adev->hotplug_work);
>>       }
>> +
>> +    if (adev->irq.ih_soft.ring)
>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>> +    if (adev->irq.ih.ring)
>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>> +    if (adev->irq.ih1.ring)
>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> +    if (adev->irq.ih2.ring)
>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>   }
>>   /**
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> index 485f249d063a..62d829f5e62c 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct 
>> ttm_buffer_object *tbo)
>>           list_del_init(&bo->shadow_list);
>>           mutex_unlock(&adev->shadow_list_lock);
>>       }
>> -    amdgpu_bo_unref(&bo->parent);
>> +
>> +    spin_lock(&adev->mman.bdev.lru_lock);
>> +    list_del(&bo->bo);
>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>> +
>> +    amdgpu_bo_unref(&bo->parent);
>>       kfree(bo->metadata);
>>       kfree(bo);
>>   }
>> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct 
>> amdgpu_device *adev,
>>       if (bp->type == ttm_bo_type_device)
>>           bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>> +    INIT_LIST_HEAD(&bo->bo);
>> +
>> +    spin_lock(&adev->mman.bdev.lru_lock);
>> +    list_add_tail(&bo->bo, &adev->device_bo_list);
>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>> +
>>       return 0;
>>   fail_unreserve:
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h 
>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> index 9ac37569823f..5ae8555ef275 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>>       struct list_head        shadow_list;
>>       struct kgd_mem                  *kfd_bo;
>> +
>> +    struct list_head        bo;
>>   };
>>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct 
>> ttm_buffer_object *tbo)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c 
>> b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> index 183d44a6583c..df385ffc9768 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>       amdgpu_irq_fini_sw(adev);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>       amdgpu_irq_remove_domain(adev);
>>       return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c 
>> b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> index d32743949003..b8c47e0cf37a 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>       amdgpu_irq_fini_sw(adev);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>       amdgpu_irq_remove_domain(adev);
>>       return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c 
>> b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> index da96c6013477..ddfe4eaeea05 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>       amdgpu_irq_fini_sw(adev);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>       amdgpu_irq_remove_domain(adev);
>>       return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c 
>> b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> index 5eea4550b856..e171a9e78544 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>>       amdgpu_irq_fini_sw(adev);
>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>       return 0;
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c 
>> b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> index 751307f3252c..9a24f17a5750 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>       amdgpu_irq_fini_sw(adev);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>       return 0;
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c 
>> b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> index 973d80ec7f6c..b08905d1c00f 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>       amdgpu_irq_fini_sw(adev);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>       amdgpu_irq_remove_domain(adev);
>>       return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c 
>> b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> index 2d0094c276ca..8c8abc00f710 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>>       amdgpu_irq_fini_sw(adev);
>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>       return 0;
>>   }
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Christian König]

Am 03.05.21 um 22:43 schrieb Andrey Grodzovsky:
>
>
> On 2021-04-29 3:08 a.m., Christian König wrote:
>> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>>> Handle all DMA IOMMU gropup related dependencies before the
>>> group is removed.
>>>
>>> v5: Drop IOMMU notifier and switch to lockless call to 
>>> ttm_tt_unpopulate
>>
>> Maybe split that up into more patches.
>>
>>>
>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>> ---
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 
>>> ++++++++++++++++++++--
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>>>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>>>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>>>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>>>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>>>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>>>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>>>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>>>   14 files changed, 56 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> index fddb82897e5d..30a24db5f4d1 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>>>       bool                            in_pci_err_recovery;
>>>       struct pci_saved_state          *pci_state;
>>> +
>>> +    struct list_head                device_bo_list;
>>>   };
>>>   static inline struct amdgpu_device *drm_to_adev(struct drm_device 
>>> *ddev)
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> index 46d646c40338..91594ddc2459 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>> @@ -70,6 +70,7 @@
>>>   #include <drm/task_barrier.h>
>>>   #include <linux/pm_runtime.h>
>>> +
>>>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>>>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>>>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
>>> @@ -3211,7 +3212,6 @@ static const struct attribute 
>>> *amdgpu_dev_attributes[] = {
>>>       NULL
>>>   };
>>> -
>>>   /**
>>>    * amdgpu_device_init - initialize the driver
>>>    *
>>> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device 
>>> *adev,
>>>       INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>>> +    INIT_LIST_HEAD(&adev->device_bo_list);
>>> +
>>>       adev->gfx.gfx_off_req_count = 1;
>>>       adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>>> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device 
>>> *adev,
>>>       return r;
>>>   }
>>> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
>>> +{
>>> +    struct amdgpu_bo *bo = NULL;
>>> +
>>> +    /*
>>> +     * Unmaps all DMA mappings before device will be removed from it's
>>> +     * IOMMU group otherwise in case of IOMMU enabled system a crash
>>> +     * will happen.
>>> +     */
>>> +
>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>> +    while (!list_empty(&adev->device_bo_list)) {
>>> +        bo = list_first_entry(&adev->device_bo_list, struct 
>>> amdgpu_bo, bo);
>>> +        list_del_init(&bo->bo);
>>> +        spin_unlock(&adev->mman.bdev.lru_lock);
>>> +        if (bo->tbo.ttm)
>>> +            ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
>>> +        spin_lock(&adev->mman.bdev.lru_lock);
>>> +    }
>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>
>> Can you try to use the same approach as amdgpu_gtt_mgr_recover() 
>> instead of adding something to the BO?
>>
>> Christian.
>
> Are you sure that dma mappings limit themself only to GTT BOs
> which have allocated mm nodes ?

Yes, you would also need the system domain BOs. But those can be put on 
a similar list.

> Otherwsie we will crash and burn
> on missing IOMMU group when unampping post device remove.
> Problem for me to test this as in 5.12 kernel I don't crash even
> when removing this entire patch.  Looks like iommu_dma_unmap_page
> was changed since 5.9 when I introdiced this patch.

Do we really still need that stuff then? What exactly has changed?

Christian.

>
> Andrey
>
>>
>>> +}
>>> +
>>>   /**
>>>    * amdgpu_device_fini - tear down the driver
>>>    *
>>> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct 
>>> amdgpu_device *adev)
>>>           amdgpu_ucode_sysfs_fini(adev);
>>>       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>>> -
>>>       amdgpu_fbdev_fini(adev);
>>>       amdgpu_irq_fini_hw(adev);
>>>       amdgpu_device_ip_fini_early(adev);
>>> +
>>> +    amdgpu_clear_dma_mappings(adev);
>>> +
>>> +    amdgpu_gart_dummy_page_fini(adev);
>>>   }
>>>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>> index fde2d899b2c4..49cdcaf8512d 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct 
>>> amdgpu_device *adev)
>>>    *
>>>    * Frees the dummy page used by the driver (all asics).
>>>    */
>>> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>>   {
>>>       if (!adev->dummy_page_addr)
>>>           return;
>>> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>>>       vfree(adev->gart.pages);
>>>       adev->gart.pages = NULL;
>>>   #endif
>>> -    amdgpu_gart_dummy_page_fini(adev);
>>>   }
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>> index afa2e2877d87..5678d9c105ab 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct 
>>> amdgpu_device *adev);
>>>   void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>>>   int amdgpu_gart_init(struct amdgpu_device *adev);
>>>   void amdgpu_gart_fini(struct amdgpu_device *adev);
>>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>>>   int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>>>                  int pages);
>>>   int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>> index 63e815c27585..a922154953a7 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device 
>>> *adev)
>>>           if (!amdgpu_device_has_dc_support(adev))
>>>               flush_work(&adev->hotplug_work);
>>>       }
>>> +
>>> +    if (adev->irq.ih_soft.ring)
>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>> +    if (adev->irq.ih.ring)
>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>> +    if (adev->irq.ih1.ring)
>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>> +    if (adev->irq.ih2.ring)
>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>>   }
>>>   /**
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>> index 485f249d063a..62d829f5e62c 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct 
>>> ttm_buffer_object *tbo)
>>>           list_del_init(&bo->shadow_list);
>>>           mutex_unlock(&adev->shadow_list_lock);
>>>       }
>>> -    amdgpu_bo_unref(&bo->parent);
>>> +
>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>> +    list_del(&bo->bo);
>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>> +
>>> +    amdgpu_bo_unref(&bo->parent);
>>>       kfree(bo->metadata);
>>>       kfree(bo);
>>>   }
>>> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct 
>>> amdgpu_device *adev,
>>>       if (bp->type == ttm_bo_type_device)
>>>           bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>>> +    INIT_LIST_HEAD(&bo->bo);
>>> +
>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>> +    list_add_tail(&bo->bo, &adev->device_bo_list);
>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>> +
>>>       return 0;
>>>   fail_unreserve:
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h 
>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>> index 9ac37569823f..5ae8555ef275 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>>>       struct list_head        shadow_list;
>>>       struct kgd_mem                  *kfd_bo;
>>> +
>>> +    struct list_head        bo;
>>>   };
>>>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct 
>>> ttm_buffer_object *tbo)
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c 
>>> b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>> index 183d44a6583c..df385ffc9768 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>       amdgpu_irq_fini_sw(adev);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>       amdgpu_irq_remove_domain(adev);
>>>       return 0;
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c 
>>> b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>> index d32743949003..b8c47e0cf37a 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>       amdgpu_irq_fini_sw(adev);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>       amdgpu_irq_remove_domain(adev);
>>>       return 0;
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c 
>>> b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>> index da96c6013477..ddfe4eaeea05 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>       amdgpu_irq_fini_sw(adev);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>       amdgpu_irq_remove_domain(adev);
>>>       return 0;
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c 
>>> b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>> index 5eea4550b856..e171a9e78544 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>>>       amdgpu_irq_fini_sw(adev);
>>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>       return 0;
>>>   }
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c 
>>> b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>> index 751307f3252c..9a24f17a5750 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>       amdgpu_irq_fini_sw(adev);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>       return 0;
>>>   }
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c 
>>> b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>> index 973d80ec7f6c..b08905d1c00f 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>       amdgpu_irq_fini_sw(adev);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>       amdgpu_irq_remove_domain(adev);
>>>       return 0;
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c 
>>> b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>> index 2d0094c276ca..8c8abc00f710 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>>>       amdgpu_irq_fini_sw(adev);
>>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>       return 0;
>>>   }
>>

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Andrey Grodzovsky]

On 2021-05-04 3:03 a.m., Christian König wrote:
> Am 03.05.21 um 22:43 schrieb Andrey Grodzovsky:
>>
>>
>> On 2021-04-29 3:08 a.m., Christian König wrote:
>>> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>>>> Handle all DMA IOMMU gropup related dependencies before the
>>>> group is removed.
>>>>
>>>> v5: Drop IOMMU notifier and switch to lockless call to 
>>>> ttm_tt_unpopulate
>>>
>>> Maybe split that up into more patches.
>>>
>>>>
>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>> ---
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 
>>>> ++++++++++++++++++++--
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>>>>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>>>>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>>>>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>>>>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>>>>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>>>>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>>>>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>>>>   14 files changed, 56 insertions(+), 16 deletions(-)
>>>>
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> index fddb82897e5d..30a24db5f4d1 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>>>>       bool                            in_pci_err_recovery;
>>>>       struct pci_saved_state          *pci_state;
>>>> +
>>>> +    struct list_head                device_bo_list;
>>>>   };
>>>>   static inline struct amdgpu_device *drm_to_adev(struct drm_device 
>>>> *ddev)
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> index 46d646c40338..91594ddc2459 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>> @@ -70,6 +70,7 @@
>>>>   #include <drm/task_barrier.h>
>>>>   #include <linux/pm_runtime.h>
>>>> +
>>>>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>>>>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>>>>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
>>>> @@ -3211,7 +3212,6 @@ static const struct attribute 
>>>> *amdgpu_dev_attributes[] = {
>>>>       NULL
>>>>   };
>>>> -
>>>>   /**
>>>>    * amdgpu_device_init - initialize the driver
>>>>    *
>>>> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device 
>>>> *adev,
>>>>       INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>>>> +    INIT_LIST_HEAD(&adev->device_bo_list);
>>>> +
>>>>       adev->gfx.gfx_off_req_count = 1;
>>>>       adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>>>> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device 
>>>> *adev,
>>>>       return r;
>>>>   }
>>>> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
>>>> +{
>>>> +    struct amdgpu_bo *bo = NULL;
>>>> +
>>>> +    /*
>>>> +     * Unmaps all DMA mappings before device will be removed from it's
>>>> +     * IOMMU group otherwise in case of IOMMU enabled system a crash
>>>> +     * will happen.
>>>> +     */
>>>> +
>>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>>> +    while (!list_empty(&adev->device_bo_list)) {
>>>> +        bo = list_first_entry(&adev->device_bo_list, struct 
>>>> amdgpu_bo, bo);
>>>> +        list_del_init(&bo->bo);
>>>> +        spin_unlock(&adev->mman.bdev.lru_lock);
>>>> +        if (bo->tbo.ttm)
>>>> +            ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
>>>> +        spin_lock(&adev->mman.bdev.lru_lock);
>>>> +    }
>>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>>
>>> Can you try to use the same approach as amdgpu_gtt_mgr_recover() 
>>> instead of adding something to the BO?
>>>
>>> Christian.
>>
>> Are you sure that dma mappings limit themself only to GTT BOs
>> which have allocated mm nodes ?
> 
> Yes, you would also need the system domain BOs. But those can be put on 
> a similar list.

What list ? Those BOs don't have ttm_resource_manager and so no
drm_mm_node list they all bound to. Should I maintain a list for them
spcifically for the unmap purpuse ?

> 
>> Otherwsie we will crash and burn
>> on missing IOMMU group when unampping post device remove.
>> Problem for me to test this as in 5.12 kernel I don't crash even
>> when removing this entire patch.  Looks like iommu_dma_unmap_page
>> was changed since 5.9 when I introdiced this patch.
> 
> Do we really still need that stuff then? What exactly has changed?

At first I assumed that because of this change 'iommu: Allow the 
dma-iommu api to use bounce buffers'
Which changed iommu_dma_unmap_page to call __iommu_dma_unmap_swiotlb
instead if __iommu_dma_unmap directly. But then i looked inside
__iommu_dma_unmap_swiotlb and it still calls __iommu_dma_unmap
evenetually. So maybe the fact that I moved the amd_ip_funcs.hw_fini
call to inside amdgpu_pci_remove helps.

Andrey


> 
> Christian.
> 
>>
>> Andrey
>>
>>>
>>>> +}
>>>> +
>>>>   /**
>>>>    * amdgpu_device_fini - tear down the driver
>>>>    *
>>>> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct 
>>>> amdgpu_device *adev)
>>>>           amdgpu_ucode_sysfs_fini(adev);
>>>>       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>>>> -
>>>>       amdgpu_fbdev_fini(adev);
>>>>       amdgpu_irq_fini_hw(adev);
>>>>       amdgpu_device_ip_fini_early(adev);
>>>> +
>>>> +    amdgpu_clear_dma_mappings(adev);
>>>> +
>>>> +    amdgpu_gart_dummy_page_fini(adev);
>>>>   }
>>>>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>>> index fde2d899b2c4..49cdcaf8512d 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>>> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct 
>>>> amdgpu_device *adev)
>>>>    *
>>>>    * Frees the dummy page used by the driver (all asics).
>>>>    */
>>>> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>>>   {
>>>>       if (!adev->dummy_page_addr)
>>>>           return;
>>>> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>>>>       vfree(adev->gart.pages);
>>>>       adev->gart.pages = NULL;
>>>>   #endif
>>>> -    amdgpu_gart_dummy_page_fini(adev);
>>>>   }
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>>> index afa2e2877d87..5678d9c105ab 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>>> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct 
>>>> amdgpu_device *adev);
>>>>   void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>>>>   int amdgpu_gart_init(struct amdgpu_device *adev);
>>>>   void amdgpu_gart_fini(struct amdgpu_device *adev);
>>>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>>>>   int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>>>>                  int pages);
>>>>   int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>>> index 63e815c27585..a922154953a7 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>>> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device 
>>>> *adev)
>>>>           if (!amdgpu_device_has_dc_support(adev))
>>>>               flush_work(&adev->hotplug_work);
>>>>       }
>>>> +
>>>> +    if (adev->irq.ih_soft.ring)
>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>>> +    if (adev->irq.ih.ring)
>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>> +    if (adev->irq.ih1.ring)
>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>>> +    if (adev->irq.ih2.ring)
>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>>>   }
>>>>   /**
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>> index 485f249d063a..62d829f5e62c 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct 
>>>> ttm_buffer_object *tbo)
>>>>           list_del_init(&bo->shadow_list);
>>>>           mutex_unlock(&adev->shadow_list_lock);
>>>>       }
>>>> -    amdgpu_bo_unref(&bo->parent);
>>>> +
>>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>>> +    list_del(&bo->bo);
>>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>>> +
>>>> +    amdgpu_bo_unref(&bo->parent);
>>>>       kfree(bo->metadata);
>>>>       kfree(bo);
>>>>   }
>>>> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct 
>>>> amdgpu_device *adev,
>>>>       if (bp->type == ttm_bo_type_device)
>>>>           bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>>>> +    INIT_LIST_HEAD(&bo->bo);
>>>> +
>>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>>> +    list_add_tail(&bo->bo, &adev->device_bo_list);
>>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>>> +
>>>>       return 0;
>>>>   fail_unreserve:
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h 
>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>> index 9ac37569823f..5ae8555ef275 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>>>>       struct list_head        shadow_list;
>>>>       struct kgd_mem                  *kfd_bo;
>>>> +
>>>> +    struct list_head        bo;
>>>>   };
>>>>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct 
>>>> ttm_buffer_object *tbo)
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>>> index 183d44a6583c..df385ffc9768 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>>> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>       amdgpu_irq_fini_sw(adev);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>       amdgpu_irq_remove_domain(adev);
>>>>       return 0;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>>> index d32743949003..b8c47e0cf37a 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>>> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>       amdgpu_irq_fini_sw(adev);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>       amdgpu_irq_remove_domain(adev);
>>>>       return 0;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>>> index da96c6013477..ddfe4eaeea05 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>>> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>       amdgpu_irq_fini_sw(adev);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>       amdgpu_irq_remove_domain(adev);
>>>>       return 0;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>>> index 5eea4550b856..e171a9e78544 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>>> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>>>>       amdgpu_irq_fini_sw(adev);
>>>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>       return 0;
>>>>   }
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>>> index 751307f3252c..9a24f17a5750 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>>> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>       amdgpu_irq_fini_sw(adev);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>       return 0;
>>>>   }
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>>> index 973d80ec7f6c..b08905d1c00f 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>>> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>       amdgpu_irq_fini_sw(adev);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>       amdgpu_irq_remove_domain(adev);
>>>>       return 0;
>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c 
>>>> b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>>> index 2d0094c276ca..8c8abc00f710 100644
>>>> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>>> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>>> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>>>>       amdgpu_irq_fini_sw(adev);
>>>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>       return 0;
>>>>   }
>>>
> 
> _______________________________________________
> amd-gfx mailing list
> amd-gfx@lists.freedesktop.org
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C1cee392c0b934cda6c7608d90ecabc41%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637557086175078458%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=C8QBsUQhJa1eWV1YYdQaykUVQGwmCn6OIoWQSrDkWoU%3D&amp;reserved=0 
> 
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Felix Kuehling]

Am 2021-04-28 um 11:11 a.m. schrieb Andrey Grodzovsky:
> Handle all DMA IOMMU gropup related dependencies before the
> group is removed.
>
> v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
>
> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>  drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>  drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>  drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>  drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>  drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>  drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>  drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>  drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>  drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>  14 files changed, 56 insertions(+), 16 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> index fddb82897e5d..30a24db5f4d1 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>  
>  	bool                            in_pci_err_recovery;
>  	struct pci_saved_state          *pci_state;
> +
> +	struct list_head                device_bo_list;
>  };
>  
>  static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> index 46d646c40338..91594ddc2459 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
> @@ -70,6 +70,7 @@
>  #include <drm/task_barrier.h>
>  #include <linux/pm_runtime.h>
>  
> +
>  MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>  MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>  MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
> @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = {
>  	NULL
>  };
>  
> -
>  /**
>   * amdgpu_device_init - initialize the driver
>   *
> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>  
>  	INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>  
> +	INIT_LIST_HEAD(&adev->device_bo_list);
> +
>  	adev->gfx.gfx_off_req_count = 1;
>  	adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>  
> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>  	return r;
>  }
>  
> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
> +{
> +	struct amdgpu_bo *bo = NULL;
> +
> +	/*
> +	 * Unmaps all DMA mappings before device will be removed from it's
> +	 * IOMMU group otherwise in case of IOMMU enabled system a crash
> +	 * will happen.
> +	 */
> +
> +	spin_lock(&adev->mman.bdev.lru_lock);
> +	while (!list_empty(&adev->device_bo_list)) {
> +		bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);
> +		list_del_init(&bo->bo);
> +		spin_unlock(&adev->mman.bdev.lru_lock);
> +		if (bo->tbo.ttm)
> +			ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);

I have a patch pending (reviewed by Christian) that moves the
dma-unmapping to amdgpu_ttm_backend_unbind. With that patch,
ttm_tt_unpopulate would no longer be the right way to remove the DMA
mapping.

Maybe I'd need to add a check in ttm_tt_unpopulate to call
backend_unbind first, if necessary. Or is there some other mechanism
that moves the BO to the CPU domain before unpopulating it?

Regards,
  Felix


> +		spin_lock(&adev->mman.bdev.lru_lock);
> +	}
> +	spin_unlock(&adev->mman.bdev.lru_lock);
> +}
> +
>  /**
>   * amdgpu_device_fini - tear down the driver
>   *
> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>  		amdgpu_ucode_sysfs_fini(adev);
>  	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>  
> -
>  	amdgpu_fbdev_fini(adev);
>  
>  	amdgpu_irq_fini_hw(adev);
>  
>  	amdgpu_device_ip_fini_early(adev);
> +
> +	amdgpu_clear_dma_mappings(adev);
> +
> +	amdgpu_gart_dummy_page_fini(adev);
>  }
>  
>  void amdgpu_device_fini_sw(struct amdgpu_device *adev)
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> index fde2d899b2c4..49cdcaf8512d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
>   *
>   * Frees the dummy page used by the driver (all asics).
>   */
> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>  {
>  	if (!adev->dummy_page_addr)
>  		return;
> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>  	vfree(adev->gart.pages);
>  	adev->gart.pages = NULL;
>  #endif
> -	amdgpu_gart_dummy_page_fini(adev);
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> index afa2e2877d87..5678d9c105ab 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev);
>  void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>  int amdgpu_gart_init(struct amdgpu_device *adev);
>  void amdgpu_gart_fini(struct amdgpu_device *adev);
> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>  int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>  		       int pages);
>  int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> index 63e815c27585..a922154953a7 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
>  		if (!amdgpu_device_has_dc_support(adev))
>  			flush_work(&adev->hotplug_work);
>  	}
> +
> +	if (adev->irq.ih_soft.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> +	if (adev->irq.ih.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih);
> +	if (adev->irq.ih1.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> +	if (adev->irq.ih2.ring)
> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>  }
>  
>  /**
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> index 485f249d063a..62d829f5e62c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
>  		list_del_init(&bo->shadow_list);
>  		mutex_unlock(&adev->shadow_list_lock);
>  	}
> -	amdgpu_bo_unref(&bo->parent);
>  
> +
> +	spin_lock(&adev->mman.bdev.lru_lock);
> +	list_del(&bo->bo);
> +	spin_unlock(&adev->mman.bdev.lru_lock);
> +
> +	amdgpu_bo_unref(&bo->parent);
>  	kfree(bo->metadata);
>  	kfree(bo);
>  }
> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
>  	if (bp->type == ttm_bo_type_device)
>  		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>  
> +	INIT_LIST_HEAD(&bo->bo);
> +
> +	spin_lock(&adev->mman.bdev.lru_lock);
> +	list_add_tail(&bo->bo, &adev->device_bo_list);
> +	spin_unlock(&adev->mman.bdev.lru_lock);
> +
>  	return 0;
>  
>  fail_unreserve:
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> index 9ac37569823f..5ae8555ef275 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>  	struct list_head		shadow_list;
>  
>  	struct kgd_mem                  *kfd_bo;
> +
> +	struct list_head		bo;
>  };
>  
>  static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> index 183d44a6583c..df385ffc9768 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>  
>  	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>  	amdgpu_irq_remove_domain(adev);
>  
>  	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> index d32743949003..b8c47e0cf37a 100644
> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>  
>  	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>  	amdgpu_irq_remove_domain(adev);
>  
>  	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> index da96c6013477..ddfe4eaeea05 100644
> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>  
>  	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>  	amdgpu_irq_remove_domain(adev);
>  
>  	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> index 5eea4550b856..e171a9e78544 100644
> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>  
>  	amdgpu_irq_fini_sw(adev);
>  	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>  
>  	return 0;
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> index 751307f3252c..9a24f17a5750 100644
> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>  
>  	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>  
>  	return 0;
>  }
> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> index 973d80ec7f6c..b08905d1c00f 100644
> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>  	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>  
>  	amdgpu_irq_fini_sw(adev);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>  	amdgpu_irq_remove_domain(adev);
>  
>  	return 0;
> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> index 2d0094c276ca..8c8abc00f710 100644
> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>  
>  	amdgpu_irq_fini_sw(adev);
>  	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>  
>  	return 0;
>  }
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Andrey Grodzovsky]

Ping

Andrey

On 2021-04-30 12:10 p.m., Andrey Grodzovsky wrote:
> 
> 
> On 2021-04-30 2:47 a.m., Christian König wrote:
>>
>>
>> Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
>>>
>>>
>>> On 2021-04-29 3:18 a.m., Christian König wrote:
>>>> I need to take another look at this part when I don't have a massive 
>>>> headache any more.
>>>>
>>>> Maybe split the patch set up into different parts, something like:
>>>> 1. Adding general infrastructure.
>>>> 2. Making sure all memory is unpolated.
>>>> 3. Job and fence handling
>>>
>>> I am not sure you mean this patch here, maybe another one ?
>>> Also note you already RBed it.
>>
>> No what I meant was to send out the patches before this one as #1 and #2.
>>
>> That is the easier stuff which can easily go into the drm-misc-next or 
>> amd-staging-drm-next branch.
>>
>> The scheduler stuff certainly need to go into drm-misc-next.
>>
>> Christian.
> 
> Got you. I am fine with it. What we have here is a working hot-unplug
> code but, one with potential use after free MMIO ranges frpom the zombie
> device. The followup patches after this patch are all about preventing
> this and so the patch-set up until this patch including, is functional
> on it's own. While it's necessary to solve the above issue, it's has
> complications as can be seen from the discussion with Daniel on later
> patch in this series. Still, in my opinion it's better to rollout some
> initial support to hot-unplug without use after free protection then
> having no support for hot-unplug at all. It will also make the merge
> work easier as I need to constantly rebase the patches on top latest
> kernel and solve new regressions.
> 
> Daniel - given the arguments above can you sound your opinion on this
> approach ?
> 
> Andrey
>>
>>>
>>> Andrey
>>>
>>>>
>>>> Christian.
>>>>
>>>> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>>>>> Problem: If scheduler is already stopped by the time sched_entity
>>>>> is released and entity's job_queue not empty I encountred
>>>>> a hang in drm_sched_entity_flush. This is because 
>>>>> drm_sched_entity_is_idle
>>>>> never becomes false.
>>>>>
>>>>> Fix: In drm_sched_fini detach all sched_entities from the
>>>>> scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
>>>>> Also wakeup all those processes stuck in sched_entity flushing
>>>>> as the scheduler main thread which wakes them up is stopped by now.
>>>>>
>>>>> v2:
>>>>> Reverse order of drm_sched_rq_remove_entity and marking
>>>>> s_entity as stopped to prevent reinserion back to rq due
>>>>> to race.
>>>>>
>>>>> v3:
>>>>> Drop drm_sched_rq_remove_entity, only modify entity->stopped
>>>>> and check for it in drm_sched_entity_is_idle
>>>>>
>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>> Reviewed-by: Christian König <christian.koenig@amd.com>
>>>>> ---
>>>>>   drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
>>>>>   drivers/gpu/drm/scheduler/sched_main.c   | 24 
>>>>> ++++++++++++++++++++++++
>>>>>   2 files changed, 26 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c 
>>>>> b/drivers/gpu/drm/scheduler/sched_entity.c
>>>>> index f0790e9471d1..cb58f692dad9 100644
>>>>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
>>>>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
>>>>> @@ -116,7 +116,8 @@ static bool drm_sched_entity_is_idle(struct 
>>>>> drm_sched_entity *entity)
>>>>>       rmb(); /* for list_empty to work without lock */
>>>>>       if (list_empty(&entity->list) ||
>>>>> -        spsc_queue_count(&entity->job_queue) == 0)
>>>>> +        spsc_queue_count(&entity->job_queue) == 0 ||
>>>>> +        entity->stopped)
>>>>>           return true;
>>>>>       return false;
>>>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c 
>>>>> b/drivers/gpu/drm/scheduler/sched_main.c
>>>>> index 908b0b56032d..ba087354d0a8 100644
>>>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>>>> @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
>>>>>    */
>>>>>   void drm_sched_fini(struct drm_gpu_scheduler *sched)
>>>>>   {
>>>>> +    struct drm_sched_entity *s_entity;
>>>>> +    int i;
>>>>> +
>>>>>       if (sched->thread)
>>>>>           kthread_stop(sched->thread);
>>>>> +    for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= 
>>>>> DRM_SCHED_PRIORITY_MIN; i--) {
>>>>> +        struct drm_sched_rq *rq = &sched->sched_rq[i];
>>>>> +
>>>>> +        if (!rq)
>>>>> +            continue;
>>>>> +
>>>>> +        spin_lock(&rq->lock);
>>>>> +        list_for_each_entry(s_entity, &rq->entities, list)
>>>>> +            /*
>>>>> +             * Prevents reinsertion and marks job_queue as idle,
>>>>> +             * it will removed from rq in drm_sched_entity_fini
>>>>> +             * eventually
>>>>> +             */
>>>>> +            s_entity->stopped = true;
>>>>> +        spin_unlock(&rq->lock);
>>>>> +
>>>>> +    }
>>>>> +
>>>>> +    /* Wakeup everyone stuck in drm_sched_entity_flush for this 
>>>>> scheduler */
>>>>> +    wake_up_all(&sched->job_scheduled);
>>>>> +
>>>>>       /* Confirm no work left behind accessing device structures */
>>>>>       cancel_delayed_work_sync(&sched->work_tdr);
>>>>
>>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

Ping

Andrey

On 2021-04-30 1:27 p.m., Andrey Grodzovsky wrote:
> 
> 
> On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
>> On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
>>>
>>>
>>> On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
>>>> On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
>>>>>
>>>>>
>>>>> On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
>>>>>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
>>>>>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
>>>>>>>> With this calling drm_dev_unplug will flush and block
>>>>>>>> all in flight IOCTLs
>>>>>>>>
>>>>>>>> Also, add feature such that if device supports graceful unplug
>>>>>>>> we enclose entire IOCTL in SRCU critical section.
>>>>>>>>
>>>>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>>>>
>>>>>>> Nope.
>>>>>>>
>>>>>>> The idea of drm_dev_enter/exit is to mark up hw access. Not 
>>>>>>> entire ioctl.
>>>>>
>>>>> Then I am confused why we have 
>>>>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cf4c0568093cc462f625808d90bc23a3c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553751106596888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PPKrQYBrgRMjpwlL0r8n5zenIhQMFWc6gniHgUTxTAY%3D&amp;reserved=0 
>>>>>
>>>>> currently in code ?
>>>>
>>>> I forgot about this one, again. Thanks for reminding.
>>>>
>>>>>>> Especially not with an opt-in flag so that it could be shrugged 
>>>>>>> of as a
>>>>>>> driver hack. Most of these ioctls should have absolutely no problem
>>>>>>> working after hotunplug.
>>>>>>>
>>>>>>> Also, doing this defeats the point since it pretty much guarantees
>>>>>>> userspace will die in assert()s and stuff. E.g. on i915 the rough 
>>>>>>> contract
>>>>>>> is that only execbuf (and even that only when userspace has 
>>>>>>> indicated
>>>>>>> support for non-recoverable hw ctx) is allowed to fail. Anything 
>>>>>>> else
>>>>>>> might crash userspace.
>>>>>
>>>>> Given that as I pointed above we already fail any IOCTls with -ENODEV
>>>>> when device is unplugged, it seems those crashes don't happen that
>>>>> often ? Also, in all my testing I don't think I saw a user space crash
>>>>> I could attribute to this.
>>>>
>>>> I guess it should be ok.
>>>
>>> What should be ok ?
>>
>> Your approach, but not your patch. If we go with this let's just lift it
>> to drm_ioctl() as the default behavior. No driver opt-in flag, because
>> that's definitely worse than any other approach because we really need to
>> get rid of driver specific behaviour for generic ioctls, especially
>> anything a compositor will use directly.
>>
>>>> My reasons for making this work is both less trouble for userspace (did
>>>> you test with various wayland compositors out there, not just amdgpu 
>>>> x86
>>>
>>> I didn't - will give it a try.
> 
> Weston worked without crashes, run the egl tester cube there.
> 
>>>
>>>> driver?), but also testing.
>>>>
>>>> We still need a bunch of these checks in various places or you'll 
>>>> wait a
>>>> very long time for a pending modeset or similar to complete. Being 
>>>> able to
>>>> run that code easily after hotunplug has completed should help a lot 
>>>> with
>>>> testing.
>>>>
>>>> Plus various drivers already acquired drm_dev_enter/exit and now I 
>>>> wonder
>>>> whether that was properly tested or not ...
>>>>
>>>> I guess maybe we need a drm module option to disable this check, so 
>>>> that
>>>> we can exercise the code as if the ioctl has raced with hotunplug at 
>>>> the
>>>> worst possible moment.
>>>>
>>>> Also atomic is really tricky here: I assume your testing has just done
>>>> normal synchronous commits, but anything that goes through atomic 
>>>> can be
>>>> done nonblocking in a separate thread. Which the ioctl catch-all 
>>>> here wont
>>>> capture.
>>>
>>> Yes, async commit was on my mind and thanks for reminding me. Indeed
>>> I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
>>> drm_dev_enter/exit. Note that i have a bunch of patches, all name's
>>> starting with 'Scope....' that just methodically put all the background
>>> work items and timers the drivers schedules in drm_dev_enter/exit scope.
>>> This was supposed to be part of the 'Scope Display code' patch.
>>
>> That's too much. You still have to arrange that the flip completion event
>> gets sent out. So it's a bit tricky.
>>
>> In other places the same problem applies, e.g. probe functions need to
>> make sure they report "disconnected".
> 
> I see, well, this is all part of KMS support which I defer for now
> anyway. Will tackle it then.
> 
>>
>>>>>>> You probably need similar (and very precisely defined) rules for 
>>>>>>> amdgpu.
>>>>>>> And those must definitely exclude any shard ioctls from randomly 
>>>>>>> failing
>>>>>>> with EIO, because that just kills the box and defeats the point 
>>>>>>> of trying
>>>>>>> to gracefully handling hotunplug and making sure userspace has a 
>>>>>>> chance of
>>>>>>> survival. E.g. for atomic everything should continue, including flip
>>>>>>> completion, but we set all outputs to "disconnected" and send out 
>>>>>>> the
>>>>>>> uevent. Maybe crtc enabling can fail too, but that can also be 
>>>>>>> handled
>>>>>>> through the async status we're using to signal DP link failures to
>>>>>>> userspace.
>>>>>
>>>>> As I pointed before, because of the complexity of the topic I 
>>>>> prefer to
>>>>> take it step by step and solve first for secondary device use case, 
>>>>> not
>>>>> for primary, display attached device.
>>>>
>>>> Yeah makes sense. But then I think the right patch is to roll this 
>>>> out for
>>>> all drivers, properly justified with existing code. Not behind a driver
>>>> flag, because with all these different compositors the last thing we 
>>>> want
>>>> is a proliferation of driver-specific behaviour. That's imo the worst
>>>> option of all of them and needs to be avoided.
>>>
>>> So this kind of patch would be acceptable to you if I unconditionally
>>> scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
>>> I am worried to break other drivers with this, see patch 
>>> https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cf4c0568093cc462f625808d90bc23a3c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553751106596888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2F3Jq6SvTm%2BZX7AVpaxEepfOj0C3O7%2Bo2Wm3y0gxrmKI%3D&amp;reserved=0 
>>>
>>> Before setting drm_dev_unplug I go through a whole process of signalling
>>> all possible fences in the system which some one some where might be
>>> waiting on. My concern is that in the absence of HW those fences won't
>>> signal and so unless I signal them myself srcu_synchrionize in
>>> drm_dev_unplug will hang waiting for any such code scoped by
>>> drm_dev_enter/exit.
>>
>> Uh right. I forgot about this.
>>
>> Which would kinda mean the top level scope is maybe not the best idea, 
>> and
>> perhaps we should indeed drill it down. But then the testing issue
>> definitely gets a lot worse.
>>
>> So what if we'd push that drm_dev_is_unplugged check down into ioctls?
>> Then we can make a case-by case decision whether it should be 
>> converted to
>> drm_dev_enter/exit, needs to be pushed down further into drivers (due to
>> fence wait issues) or other concerns?
>>
>> Also I guess we need to have a subsystem wide rule on whether you need to
>> force complete all fences before you call drm_dev_unplug, or afterwards.
> 
> I don't see how you can handle it afterwards. If a thread is stuck in
> dma_fence_wait in non interruptible wait (any kernel thread) and with no
> timeout there is nothing you can do to stop the wait. Any such code
> scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
> The only way then is to preemptively force signal all such fences before
> calling drm_dev_unplug - as I do in the above mentioned patch.
> 
>> If we have mixed behaviour on this there will be disappointment. And 
>> since
>> hotunplug and dma_fence completion are both userspace visible that
>> inconsistency might have bigger impact.
>>
>> This is all very tricky indeed :-/
>>
>> btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
>> with that: We could do the same trick we've done for DRM_UNLOCKED:
>> - drm_dev_enter/exit is called for any ioctl that has not set the
>>    DRM_HOTUNPLUG_SAFE flag
>> - for drm core ioctls we push them into all ioctls and decide how to
>>    handle/where (with the aim to have the least amount of code flow
>>    different during hotunplug vs after hotunplug has finished, to reduce
>>    testing scope)
>> - then we make DRM_HOTUNPLUG_SAFE the implied default
>>
>> This would have us left with render ioctls, and I think the defensive
>> assumption there is that they're all hotunplug safe. We might hang on a
>> fence wait, but that's fixable, and it's better than blowing up on a
>> use-after-free security bug.
>>
>> Thoughts?
> 
> I don't fully see a difference between the approach described above and
> the full drill down to each driver and even within the driver, to the HW
> back-ends - what criteria I would use to decide if for a given IOCTL i
> scope with drm_dev_enter/exit at the highest level while for another
> i go all the way down ? If we would agree that signaling the fences
> preemptively before engaging drm_dev_unplug is generically the right
> approach maybe we can then scope drm_ioctl unconditionally with
> drm_dev_enter/exit and then for each driver go through the same process
> I do for amdgpu - writing driver specific function which takes care of
> all the fences. We could then just create a drm callback which would
> be called from drm_ioctl before drm_dev_unplug is called.
> 
> Andrey
> 
>>
>> It is unfortunately even more work until we've reached the goal, but I
>> think it's safest and most flexible approach overall.
>>
>> Cheers, Daniel
>>
>>>
>>> Andrey
>>>
>>>>
>>>> Cheers, Daniel
>>>>
>>>>
>>>>>
>>>>>>>
>>>>>>> I guess we should clarify this in the hotunplug doc?
>>>>>
>>>>> Agree
>>>>>
>>>>>>
>>>>>> To clarify: I'm not against throwing an ENODEV at userspace for 
>>>>>> ioctl that
>>>>>> really make no sense, and where we're rather confident that all 
>>>>>> properly
>>>>>> implemented userspace will gracefully handle failures. Like a 
>>>>>> modeset, or
>>>>>> opening a device, or trying to import a dma-buf or stuff like that 
>>>>>> which
>>>>>> can already fail in normal operation for any kind of reason.
>>>>>>
>>>>>> But stuff that never fails, like GETRESOURCES ioctl, really 
>>>>>> shouldn't fail
>>>>>> after hotunplug.
>>>>>
>>>>> As I pointed above, this a bit confuses me given that we already do
>>>>> blanker rejection of IOCTLs if device is unplugged.
>>>>
>>>> Well I'm confused about this too :-/
>>>>
>>>>>> And then there's the middle ground, like doing a pageflip or 
>>>>>> buffer flush,
>>>>>> which I guess some userspace might handle, but risky to inflict those
>>>>>> consequences on them. atomic modeset is especially fun since 
>>>>>> depending
>>>>>> what you're doing it can be both "failures expected" and "failures 
>>>>>> not
>>>>>> really expected in normal operation".
>>>>>>
>>>>>> Also, this really should be consistent across drivers, not solved 
>>>>>> with a
>>>>>> driver flag for every possible combination.
>>>>>>
>>>>>> If you look at the current hotunplug kms drivers, they have
>>>>>> drm_dev_enter/exit sprinkled in specific hw callback functions 
>>>>>> because of
>>>>>> the above problems. But maybe it makes sense to change things in a 
>>>>>> few
>>>>>> cases. But then we should do it across the board.
>>>>>
>>>>> So as I understand your preferred approach is that I scope any 
>>>>> back_end, HW
>>>>> specific function with drm_dev_enter/exit because that where MMIO
>>>>> access takes place. But besides explicit MMIO access thorough
>>>>> register accessors in the HW back-end there is also indirect MMIO 
>>>>> access
>>>>> taking place throughout the code in the driver because of various VRAM
>>>>> BOs which provide CPU access to VRAM through the VRAM BAR. This 
>>>>> kind of
>>>>> access is spread all over in the driver and even in mid-layers such as
>>>>> TTM and not limited to HW back-end functions. It means it's much 
>>>>> harder
>>>>> to spot such places to surgically scope them with 
>>>>> drm_dev_enter/exit and
>>>>> also that any new such code introduced will immediately break hot 
>>>>> unplug
>>>>> because the developers can't be expected to remember making their code
>>>>> robust to this specific use case. That why when we discussed 
>>>>> internally
>>>>> what approach to take to protecting code with drm_dev_enter/exit we
>>>>> opted for using the widest available scope.
>>>>
>>>> The thing is, you kinda have to anyway. There's enormous amounts of
>>>> asynchronous processing going on. E.g. nonblocking atomic commits 
>>>> also do
>>>> ttm unpinning and fun stuff like that, which if you sync things 
>>>> wrong can
>>>> happen way late. So the door for bad fallout is wide open :-(
>>>>
>>>> I'm not sure where the right tradeoff is to make sure we catch them 
>>>> all,
>>>> and can make sure with testing that we've indeed caught them all.
>>>> -Daniel
>>>>
>>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Andrey Grodzovsky]

On 2021-05-04 1:05 p.m., Felix Kuehling wrote:
> 
> Am 2021-04-28 um 11:11 a.m. schrieb Andrey Grodzovsky:
>> Handle all DMA IOMMU gropup related dependencies before the
>> group is removed.
>>
>> v5: Drop IOMMU notifier and switch to lockless call to ttm_tt_unpopulate
>>
>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>> ---
>>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 ++++++++++++++++++++--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>>   14 files changed, 56 insertions(+), 16 deletions(-)
>>
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> index fddb82897e5d..30a24db5f4d1 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>>   
>>   	bool                            in_pci_err_recovery;
>>   	struct pci_saved_state          *pci_state;
>> +
>> +	struct list_head                device_bo_list;
>>   };
>>   
>>   static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> index 46d646c40338..91594ddc2459 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>> @@ -70,6 +70,7 @@
>>   #include <drm/task_barrier.h>
>>   #include <linux/pm_runtime.h>
>>   
>> +
>>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
>> @@ -3211,7 +3212,6 @@ static const struct attribute *amdgpu_dev_attributes[] = {
>>   	NULL
>>   };
>>   
>> -
>>   /**
>>    * amdgpu_device_init - initialize the driver
>>    *
>> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>>   
>>   	INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func);
>>   
>> +	INIT_LIST_HEAD(&adev->device_bo_list);
>> +
>>   	adev->gfx.gfx_off_req_count = 1;
>>   	adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>>   
>> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
>>   	return r;
>>   }
>>   
>> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
>> +{
>> +	struct amdgpu_bo *bo = NULL;
>> +
>> +	/*
>> +	 * Unmaps all DMA mappings before device will be removed from it's
>> +	 * IOMMU group otherwise in case of IOMMU enabled system a crash
>> +	 * will happen.
>> +	 */
>> +
>> +	spin_lock(&adev->mman.bdev.lru_lock);
>> +	while (!list_empty(&adev->device_bo_list)) {
>> +		bo = list_first_entry(&adev->device_bo_list, struct amdgpu_bo, bo);
>> +		list_del_init(&bo->bo);
>> +		spin_unlock(&adev->mman.bdev.lru_lock);
>> +		if (bo->tbo.ttm)
>> +			ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
> 
> I have a patch pending (reviewed by Christian) that moves the
> dma-unmapping to amdgpu_ttm_backend_unbind. With that patch,
> ttm_tt_unpopulate would no longer be the right way to remove the DMA
> mapping.
> 
> Maybe I'd need to add a check in ttm_tt_unpopulate to call
> backend_unbind first, if necessary. Or is there some other mechanism
> that moves the BO to the CPU domain before unpopulating it?
> 
> Regards,
>    Felix

At least in the context of this patch we don't move the BO to system
domain but rather preemptively remove DMA mappings before IOMMU grpoup
is gone post pci_remove. So yes, I would say yes, we need to check here
for backend_unbind first.

Andrey

> 
> 
>> +		spin_lock(&adev->mman.bdev.lru_lock);
>> +	}
>> +	spin_unlock(&adev->mman.bdev.lru_lock);
>> +}
>> +
>>   /**
>>    * amdgpu_device_fini - tear down the driver
>>    *
>> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev)
>>   		amdgpu_ucode_sysfs_fini(adev);
>>   	sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>>   
>> -
>>   	amdgpu_fbdev_fini(adev);
>>   
>>   	amdgpu_irq_fini_hw(adev);
>>   
>>   	amdgpu_device_ip_fini_early(adev);
>> +
>> +	amdgpu_clear_dma_mappings(adev);
>> +
>> +	amdgpu_gart_dummy_page_fini(adev);
>>   }
>>   
>>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> index fde2d899b2c4..49cdcaf8512d 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
>>    *
>>    * Frees the dummy page used by the driver (all asics).
>>    */
>> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>   {
>>   	if (!adev->dummy_page_addr)
>>   		return;
>> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>>   	vfree(adev->gart.pages);
>>   	adev->gart.pages = NULL;
>>   #endif
>> -	amdgpu_gart_dummy_page_fini(adev);
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> index afa2e2877d87..5678d9c105ab 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev);
>>   void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>>   int amdgpu_gart_init(struct amdgpu_device *adev);
>>   void amdgpu_gart_fini(struct amdgpu_device *adev);
>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>>   int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>>   		       int pages);
>>   int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> index 63e815c27585..a922154953a7 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device *adev)
>>   		if (!amdgpu_device_has_dc_support(adev))
>>   			flush_work(&adev->hotplug_work);
>>   	}
>> +
>> +	if (adev->irq.ih_soft.ring)
>> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>> +	if (adev->irq.ih.ring)
>> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>> +	if (adev->irq.ih1.ring)
>> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> +	if (adev->irq.ih2.ring)
>> +		amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>   }
>>   
>>   /**
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> index 485f249d063a..62d829f5e62c 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
>>   		list_del_init(&bo->shadow_list);
>>   		mutex_unlock(&adev->shadow_list_lock);
>>   	}
>> -	amdgpu_bo_unref(&bo->parent);
>>   
>> +
>> +	spin_lock(&adev->mman.bdev.lru_lock);
>> +	list_del(&bo->bo);
>> +	spin_unlock(&adev->mman.bdev.lru_lock);
>> +
>> +	amdgpu_bo_unref(&bo->parent);
>>   	kfree(bo->metadata);
>>   	kfree(bo);
>>   }
>> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
>>   	if (bp->type == ttm_bo_type_device)
>>   		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>>   
>> +	INIT_LIST_HEAD(&bo->bo);
>> +
>> +	spin_lock(&adev->mman.bdev.lru_lock);
>> +	list_add_tail(&bo->bo, &adev->device_bo_list);
>> +	spin_unlock(&adev->mman.bdev.lru_lock);
>> +
>>   	return 0;
>>   
>>   fail_unreserve:
>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> index 9ac37569823f..5ae8555ef275 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>>   	struct list_head		shadow_list;
>>   
>>   	struct kgd_mem                  *kfd_bo;
>> +
>> +	struct list_head		bo;
>>   };
>>   
>>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
>> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> index 183d44a6583c..df385ffc9768 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>   
>>   	amdgpu_irq_fini_sw(adev);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>   	amdgpu_irq_remove_domain(adev);
>>   
>>   	return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> index d32743949003..b8c47e0cf37a 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>   
>>   	amdgpu_irq_fini_sw(adev);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>   	amdgpu_irq_remove_domain(adev);
>>   
>>   	return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> index da96c6013477..ddfe4eaeea05 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>   
>>   	amdgpu_irq_fini_sw(adev);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>   	amdgpu_irq_remove_domain(adev);
>>   
>>   	return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> index 5eea4550b856..e171a9e78544 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>>   
>>   	amdgpu_irq_fini_sw(adev);
>>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>   
>>   	return 0;
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> index 751307f3252c..9a24f17a5750 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>   
>>   	amdgpu_irq_fini_sw(adev);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>   
>>   	return 0;
>>   }
>> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> index 973d80ec7f6c..b08905d1c00f 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>>   	struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>   
>>   	amdgpu_irq_fini_sw(adev);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>   	amdgpu_irq_remove_domain(adev);
>>   
>>   	return 0;
>> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> index 2d0094c276ca..8c8abc00f710 100644
>> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>>   
>>   	amdgpu_irq_fini_sw(adev);
>>   	amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>> -	amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>   
>>   	return 0;
>>   }
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 06/27] drm/amdgpu: Handle IOMMU enabled case.

[Andrey Grodzovsky]

Ping

Andrey

On 2021-05-04 11:43 a.m., Andrey Grodzovsky wrote:
> 
> 
> On 2021-05-04 3:03 a.m., Christian König wrote:
>> Am 03.05.21 um 22:43 schrieb Andrey Grodzovsky:
>>>
>>>
>>> On 2021-04-29 3:08 a.m., Christian König wrote:
>>>> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>>>>> Handle all DMA IOMMU gropup related dependencies before the
>>>>> group is removed.
>>>>>
>>>>> v5: Drop IOMMU notifier and switch to lockless call to 
>>>>> ttm_tt_unpopulate
>>>>
>>>> Maybe split that up into more patches.
>>>>
>>>>>
>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>> ---
>>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  2 ++
>>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 31 
>>>>> ++++++++++++++++++++--
>>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c   |  3 +--
>>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h   |  1 +
>>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c    |  9 +++++++
>>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 13 ++++++++-
>>>>>   drivers/gpu/drm/amd/amdgpu/amdgpu_object.h |  2 ++
>>>>>   drivers/gpu/drm/amd/amdgpu/cik_ih.c        |  1 -
>>>>>   drivers/gpu/drm/amd/amdgpu/cz_ih.c         |  1 -
>>>>>   drivers/gpu/drm/amd/amdgpu/iceland_ih.c    |  1 -
>>>>>   drivers/gpu/drm/amd/amdgpu/navi10_ih.c     |  3 ---
>>>>>   drivers/gpu/drm/amd/amdgpu/si_ih.c         |  1 -
>>>>>   drivers/gpu/drm/amd/amdgpu/tonga_ih.c      |  1 -
>>>>>   drivers/gpu/drm/amd/amdgpu/vega10_ih.c     |  3 ---
>>>>>   14 files changed, 56 insertions(+), 16 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h 
>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>> index fddb82897e5d..30a24db5f4d1 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
>>>>> @@ -1054,6 +1054,8 @@ struct amdgpu_device {
>>>>>       bool                            in_pci_err_recovery;
>>>>>       struct pci_saved_state          *pci_state;
>>>>> +
>>>>> +    struct list_head                device_bo_list;
>>>>>   };
>>>>>   static inline struct amdgpu_device *drm_to_adev(struct drm_device 
>>>>> *ddev)
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>> index 46d646c40338..91594ddc2459 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
>>>>> @@ -70,6 +70,7 @@
>>>>>   #include <drm/task_barrier.h>
>>>>>   #include <linux/pm_runtime.h>
>>>>> +
>>>>>   MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin");
>>>>>   MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin");
>>>>>   MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin");
>>>>> @@ -3211,7 +3212,6 @@ static const struct attribute 
>>>>> *amdgpu_dev_attributes[] = {
>>>>>       NULL
>>>>>   };
>>>>> -
>>>>>   /**
>>>>>    * amdgpu_device_init - initialize the driver
>>>>>    *
>>>>> @@ -3316,6 +3316,8 @@ int amdgpu_device_init(struct amdgpu_device 
>>>>> *adev,
>>>>>       INIT_WORK(&adev->xgmi_reset_work, 
>>>>> amdgpu_device_xgmi_reset_func);
>>>>> +    INIT_LIST_HEAD(&adev->device_bo_list);
>>>>> +
>>>>>       adev->gfx.gfx_off_req_count = 1;
>>>>>       adev->pm.ac_power = power_supply_is_system_supplied() > 0;
>>>>> @@ -3601,6 +3603,28 @@ int amdgpu_device_init(struct amdgpu_device 
>>>>> *adev,
>>>>>       return r;
>>>>>   }
>>>>> +static void amdgpu_clear_dma_mappings(struct amdgpu_device *adev)
>>>>> +{
>>>>> +    struct amdgpu_bo *bo = NULL;
>>>>> +
>>>>> +    /*
>>>>> +     * Unmaps all DMA mappings before device will be removed from 
>>>>> it's
>>>>> +     * IOMMU group otherwise in case of IOMMU enabled system a crash
>>>>> +     * will happen.
>>>>> +     */
>>>>> +
>>>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>>>> +    while (!list_empty(&adev->device_bo_list)) {
>>>>> +        bo = list_first_entry(&adev->device_bo_list, struct 
>>>>> amdgpu_bo, bo);
>>>>> +        list_del_init(&bo->bo);
>>>>> +        spin_unlock(&adev->mman.bdev.lru_lock);
>>>>> +        if (bo->tbo.ttm)
>>>>> +            ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm);
>>>>> +        spin_lock(&adev->mman.bdev.lru_lock);
>>>>> +    }
>>>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>>>
>>>> Can you try to use the same approach as amdgpu_gtt_mgr_recover() 
>>>> instead of adding something to the BO?
>>>>
>>>> Christian.
>>>
>>> Are you sure that dma mappings limit themself only to GTT BOs
>>> which have allocated mm nodes ?
>>
>> Yes, you would also need the system domain BOs. But those can be put 
>> on a similar list.
> 
> What list ? Those BOs don't have ttm_resource_manager and so no
> drm_mm_node list they all bound to. Should I maintain a list for them
> spcifically for the unmap purpuse ?
> 
>>
>>> Otherwsie we will crash and burn
>>> on missing IOMMU group when unampping post device remove.
>>> Problem for me to test this as in 5.12 kernel I don't crash even
>>> when removing this entire patch.  Looks like iommu_dma_unmap_page
>>> was changed since 5.9 when I introdiced this patch.
>>
>> Do we really still need that stuff then? What exactly has changed?
> 
> At first I assumed that because of this change 'iommu: Allow the 
> dma-iommu api to use bounce buffers'
> Which changed iommu_dma_unmap_page to call __iommu_dma_unmap_swiotlb
> instead if __iommu_dma_unmap directly. But then i looked inside
> __iommu_dma_unmap_swiotlb and it still calls __iommu_dma_unmap
> evenetually. So maybe the fact that I moved the amd_ip_funcs.hw_fini
> call to inside amdgpu_pci_remove helps.
> 
> Andrey
> 
> 
>>
>> Christian.
>>
>>>
>>> Andrey
>>>
>>>>
>>>>> +}
>>>>> +
>>>>>   /**
>>>>>    * amdgpu_device_fini - tear down the driver
>>>>>    *
>>>>> @@ -3639,12 +3663,15 @@ void amdgpu_device_fini_hw(struct 
>>>>> amdgpu_device *adev)
>>>>>           amdgpu_ucode_sysfs_fini(adev);
>>>>>       sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes);
>>>>> -
>>>>>       amdgpu_fbdev_fini(adev);
>>>>>       amdgpu_irq_fini_hw(adev);
>>>>>       amdgpu_device_ip_fini_early(adev);
>>>>> +
>>>>> +    amdgpu_clear_dma_mappings(adev);
>>>>> +
>>>>> +    amdgpu_gart_dummy_page_fini(adev);
>>>>>   }
>>>>>   void amdgpu_device_fini_sw(struct amdgpu_device *adev)
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>>>> index fde2d899b2c4..49cdcaf8512d 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
>>>>> @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct 
>>>>> amdgpu_device *adev)
>>>>>    *
>>>>>    * Frees the dummy page used by the driver (all asics).
>>>>>    */
>>>>> -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>>>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev)
>>>>>   {
>>>>>       if (!adev->dummy_page_addr)
>>>>>           return;
>>>>> @@ -397,5 +397,4 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)
>>>>>       vfree(adev->gart.pages);
>>>>>       adev->gart.pages = NULL;
>>>>>   #endif
>>>>> -    amdgpu_gart_dummy_page_fini(adev);
>>>>>   }
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h 
>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>>>> index afa2e2877d87..5678d9c105ab 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
>>>>> @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct 
>>>>> amdgpu_device *adev);
>>>>>   void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev);
>>>>>   int amdgpu_gart_init(struct amdgpu_device *adev);
>>>>>   void amdgpu_gart_fini(struct amdgpu_device *adev);
>>>>> +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev);
>>>>>   int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
>>>>>                  int pages);
>>>>>   int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset,
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>>>> index 63e815c27585..a922154953a7 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
>>>>> @@ -326,6 +326,15 @@ void amdgpu_irq_fini_hw(struct amdgpu_device 
>>>>> *adev)
>>>>>           if (!amdgpu_device_has_dc_support(adev))
>>>>>               flush_work(&adev->hotplug_work);
>>>>>       }
>>>>> +
>>>>> +    if (adev->irq.ih_soft.ring)
>>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>>>> +    if (adev->irq.ih.ring)
>>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>> +    if (adev->irq.ih1.ring)
>>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>>>> +    if (adev->irq.ih2.ring)
>>>>> +        amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>>>>   }
>>>>>   /**
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>>> index 485f249d063a..62d829f5e62c 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
>>>>> @@ -68,8 +68,13 @@ static void amdgpu_bo_destroy(struct 
>>>>> ttm_buffer_object *tbo)
>>>>>           list_del_init(&bo->shadow_list);
>>>>>           mutex_unlock(&adev->shadow_list_lock);
>>>>>       }
>>>>> -    amdgpu_bo_unref(&bo->parent);
>>>>> +
>>>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>>>> +    list_del(&bo->bo);
>>>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>>>> +
>>>>> +    amdgpu_bo_unref(&bo->parent);
>>>>>       kfree(bo->metadata);
>>>>>       kfree(bo);
>>>>>   }
>>>>> @@ -585,6 +590,12 @@ static int amdgpu_bo_do_create(struct 
>>>>> amdgpu_device *adev,
>>>>>       if (bp->type == ttm_bo_type_device)
>>>>>           bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
>>>>> +    INIT_LIST_HEAD(&bo->bo);
>>>>> +
>>>>> +    spin_lock(&adev->mman.bdev.lru_lock);
>>>>> +    list_add_tail(&bo->bo, &adev->device_bo_list);
>>>>> +    spin_unlock(&adev->mman.bdev.lru_lock);
>>>>> +
>>>>>       return 0;
>>>>>   fail_unreserve:
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h 
>>>>> b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>>> index 9ac37569823f..5ae8555ef275 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
>>>>> @@ -110,6 +110,8 @@ struct amdgpu_bo {
>>>>>       struct list_head        shadow_list;
>>>>>       struct kgd_mem                  *kfd_bo;
>>>>> +
>>>>> +    struct list_head        bo;
>>>>>   };
>>>>>   static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct 
>>>>> ttm_buffer_object *tbo)
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>>>> index 183d44a6583c..df385ffc9768 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c
>>>>> @@ -310,7 +310,6 @@ static int cik_ih_sw_fini(void *handle)
>>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>>       amdgpu_irq_fini_sw(adev);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>>       amdgpu_irq_remove_domain(adev);
>>>>>       return 0;
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>>>> index d32743949003..b8c47e0cf37a 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c
>>>>> @@ -302,7 +302,6 @@ static int cz_ih_sw_fini(void *handle)
>>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>>       amdgpu_irq_fini_sw(adev);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>>       amdgpu_irq_remove_domain(adev);
>>>>>       return 0;
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>>>> index da96c6013477..ddfe4eaeea05 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c
>>>>> @@ -301,7 +301,6 @@ static int iceland_ih_sw_fini(void *handle)
>>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>>       amdgpu_irq_fini_sw(adev);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>>       amdgpu_irq_remove_domain(adev);
>>>>>       return 0;
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>>>> index 5eea4550b856..e171a9e78544 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c
>>>>> @@ -571,9 +571,6 @@ static int navi10_ih_sw_fini(void *handle)
>>>>>       amdgpu_irq_fini_sw(adev);
>>>>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>>       return 0;
>>>>>   }
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>>>> index 751307f3252c..9a24f17a5750 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c
>>>>> @@ -176,7 +176,6 @@ static int si_ih_sw_fini(void *handle)
>>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>>       amdgpu_irq_fini_sw(adev);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>>       return 0;
>>>>>   }
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>>>> index 973d80ec7f6c..b08905d1c00f 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c
>>>>> @@ -313,7 +313,6 @@ static int tonga_ih_sw_fini(void *handle)
>>>>>       struct amdgpu_device *adev = (struct amdgpu_device *)handle;
>>>>>       amdgpu_irq_fini_sw(adev);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>>       amdgpu_irq_remove_domain(adev);
>>>>>       return 0;
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c 
>>>>> b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>>>> index 2d0094c276ca..8c8abc00f710 100644
>>>>> --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
>>>>> @@ -525,9 +525,6 @@ static int vega10_ih_sw_fini(void *handle)
>>>>>       amdgpu_irq_fini_sw(adev);
>>>>>       amdgpu_ih_ring_fini(adev, &adev->irq.ih_soft);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih2);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih1);
>>>>> -    amdgpu_ih_ring_fini(adev, &adev->irq.ih);
>>>>>       return 0;
>>>>>   }
>>>>
>>
>> _______________________________________________
>> amd-gfx mailing list
>> amd-gfx@lists.freedesktop.org
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfx&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C1cee392c0b934cda6c7608d90ecabc41%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637557086175078458%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=C8QBsUQhJa1eWV1YYdQaykUVQGwmCn6OIoWQSrDkWoU%3D&amp;reserved=0 
>>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
> 
> 
> On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
> > On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
> > > 
> > > 
> > > On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
> > > > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
> > > > > 
> > > > > 
> > > > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
> > > > > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
> > > > > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> > > > > > > > With this calling drm_dev_unplug will flush and block
> > > > > > > > all in flight IOCTLs
> > > > > > > > 
> > > > > > > > Also, add feature such that if device supports graceful unplug
> > > > > > > > we enclose entire IOCTL in SRCU critical section.
> > > > > > > > 
> > > > > > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > > > > > > 
> > > > > > > Nope.
> > > > > > > 
> > > > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
> > > > > 
> > > > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cf4c0568093cc462f625808d90bc23a3c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553751106596888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PPKrQYBrgRMjpwlL0r8n5zenIhQMFWc6gniHgUTxTAY%3D&amp;reserved=0
> > > > > currently in code ?
> > > > 
> > > > I forgot about this one, again. Thanks for reminding.
> > > > 
> > > > > > > Especially not with an opt-in flag so that it could be shrugged of as a
> > > > > > > driver hack. Most of these ioctls should have absolutely no problem
> > > > > > > working after hotunplug.
> > > > > > > 
> > > > > > > Also, doing this defeats the point since it pretty much guarantees
> > > > > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract
> > > > > > > is that only execbuf (and even that only when userspace has indicated
> > > > > > > support for non-recoverable hw ctx) is allowed to fail. Anything else
> > > > > > > might crash userspace.
> > > > > 
> > > > > Given that as I pointed above we already fail any IOCTls with -ENODEV
> > > > > when device is unplugged, it seems those crashes don't happen that
> > > > > often ? Also, in all my testing I don't think I saw a user space crash
> > > > > I could attribute to this.
> > > > 
> > > > I guess it should be ok.
> > > 
> > > What should be ok ?
> > 
> > Your approach, but not your patch. If we go with this let's just lift it
> > to drm_ioctl() as the default behavior. No driver opt-in flag, because
> > that's definitely worse than any other approach because we really need to
> > get rid of driver specific behaviour for generic ioctls, especially
> > anything a compositor will use directly.
> > 
> > > > My reasons for making this work is both less trouble for userspace (did
> > > > you test with various wayland compositors out there, not just amdgpu x86
> > > 
> > > I didn't - will give it a try.
> 
> Weston worked without crashes, run the egl tester cube there.
> 
> > > 
> > > > driver?), but also testing.
> > > > 
> > > > We still need a bunch of these checks in various places or you'll wait a
> > > > very long time for a pending modeset or similar to complete. Being able to
> > > > run that code easily after hotunplug has completed should help a lot with
> > > > testing.
> > > > 
> > > > Plus various drivers already acquired drm_dev_enter/exit and now I wonder
> > > > whether that was properly tested or not ...
> > > > 
> > > > I guess maybe we need a drm module option to disable this check, so that
> > > > we can exercise the code as if the ioctl has raced with hotunplug at the
> > > > worst possible moment.
> > > > 
> > > > Also atomic is really tricky here: I assume your testing has just done
> > > > normal synchronous commits, but anything that goes through atomic can be
> > > > done nonblocking in a separate thread. Which the ioctl catch-all here wont
> > > > capture.
> > > 
> > > Yes, async commit was on my mind and thanks for reminding me. Indeed
> > > I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
> > > drm_dev_enter/exit. Note that i have a bunch of patches, all name's
> > > starting with 'Scope....' that just methodically put all the background
> > > work items and timers the drivers schedules in drm_dev_enter/exit scope.
> > > This was supposed to be part of the 'Scope Display code' patch.
> > 
> > That's too much. You still have to arrange that the flip completion event
> > gets sent out. So it's a bit tricky.
> > 
> > In other places the same problem applies, e.g. probe functions need to
> > make sure they report "disconnected".
> 
> I see, well, this is all part of KMS support which I defer for now
> anyway. Will tackle it then.
> 
> > 
> > > > > > > You probably need similar (and very precisely defined) rules for amdgpu.
> > > > > > > And those must definitely exclude any shard ioctls from randomly failing
> > > > > > > with EIO, because that just kills the box and defeats the point of trying
> > > > > > > to gracefully handling hotunplug and making sure userspace has a chance of
> > > > > > > survival. E.g. for atomic everything should continue, including flip
> > > > > > > completion, but we set all outputs to "disconnected" and send out the
> > > > > > > uevent. Maybe crtc enabling can fail too, but that can also be handled
> > > > > > > through the async status we're using to signal DP link failures to
> > > > > > > userspace.
> > > > > 
> > > > > As I pointed before, because of the complexity of the topic I prefer to
> > > > > take it step by step and solve first for secondary device use case, not
> > > > > for primary, display attached device.
> > > > 
> > > > Yeah makes sense. But then I think the right patch is to roll this out for
> > > > all drivers, properly justified with existing code. Not behind a driver
> > > > flag, because with all these different compositors the last thing we want
> > > > is a proliferation of driver-specific behaviour. That's imo the worst
> > > > option of all of them and needs to be avoided.
> > > 
> > > So this kind of patch would be acceptable to you if I unconditionally
> > > scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
> > > I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Cf4c0568093cc462f625808d90bc23a3c%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637553751106596888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2F3Jq6SvTm%2BZX7AVpaxEepfOj0C3O7%2Bo2Wm3y0gxrmKI%3D&amp;reserved=0
> > > Before setting drm_dev_unplug I go through a whole process of signalling
> > > all possible fences in the system which some one some where might be
> > > waiting on. My concern is that in the absence of HW those fences won't
> > > signal and so unless I signal them myself srcu_synchrionize in
> > > drm_dev_unplug will hang waiting for any such code scoped by
> > > drm_dev_enter/exit.
> > 
> > Uh right. I forgot about this.
> > 
> > Which would kinda mean the top level scope is maybe not the best idea, and
> > perhaps we should indeed drill it down. But then the testing issue
> > definitely gets a lot worse.
> > 
> > So what if we'd push that drm_dev_is_unplugged check down into ioctls?
> > Then we can make a case-by case decision whether it should be converted to
> > drm_dev_enter/exit, needs to be pushed down further into drivers (due to
> > fence wait issues) or other concerns?
> > 
> > Also I guess we need to have a subsystem wide rule on whether you need to
> > force complete all fences before you call drm_dev_unplug, or afterwards.
> 
> I don't see how you can handle it afterwards. If a thread is stuck in
> dma_fence_wait in non interruptible wait (any kernel thread) and with no
> timeout there is nothing you can do to stop the wait. Any such code
> scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
> The only way then is to preemptively force signal all such fences before
> calling drm_dev_unplug - as I do in the above mentioned patch.

Yeah, which is why I don't think top-level drm_dev_enter/exit is a good
idea.

> > If we have mixed behaviour on this there will be disappointment. And since
> > hotunplug and dma_fence completion are both userspace visible that
> > inconsistency might have bigger impact.
> > 
> > This is all very tricky indeed :-/
> > 
> > btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
> > with that: We could do the same trick we've done for DRM_UNLOCKED:
> > - drm_dev_enter/exit is called for any ioctl that has not set the
> >    DRM_HOTUNPLUG_SAFE flag
> > - for drm core ioctls we push them into all ioctls and decide how to
> >    handle/where (with the aim to have the least amount of code flow
> >    different during hotunplug vs after hotunplug has finished, to reduce
> >    testing scope)
> > - then we make DRM_HOTUNPLUG_SAFE the implied default
> > 
> > This would have us left with render ioctls, and I think the defensive
> > assumption there is that they're all hotunplug safe. We might hang on a
> > fence wait, but that's fixable, and it's better than blowing up on a
> > use-after-free security bug.
> > 
> > Thoughts?
> 
> I don't fully see a difference between the approach described above and
> the full drill down to each driver and even within the driver, to the HW
> back-ends - what criteria I would use to decide if for a given IOCTL i
> scope with drm_dev_enter/exit at the highest level while for another
> i go all the way down ? If we would agree that signaling the fences
> preemptively before engaging drm_dev_unplug is generically the right
> approach maybe we can then scope drm_ioctl unconditionally with
> drm_dev_enter/exit and then for each driver go through the same process
> I do for amdgpu - writing driver specific function which takes care of
> all the fences. We could then just create a drm callback which would
> be called from drm_ioctl before drm_dev_unplug is called.

So I see the appeal of just nuking all the fences, but I'm not sure that's
a good plan. We've done this in the old i915 gpu reset code too, and the
issue is it's defacto inverting the locking. But also the hw is truly
gone, so it also makes sense.

The problem is a bit roll-out, if we state that dma_fence_wait is allowed
with a drm_dev_enter/exit, then all drivers need to force-retire their
fences.

The other option would be that we require that dma_fence_wait is _not_
allowed in drm_dev_enter/exit, and that therefore these areas must be
marked up more fine-grained to avoid deadlocks. I like this more from the
testing aspect (it makes it easier to be reasonable sure your code handles
concurrent hotunplug), but also it's pretty easy to validate with the
dma_fence lockdep annotations we have I think.

A third reasons for not requiring force-retiring of dma_fence before
drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new
fences from happening, but until you've stopped new fences it's hard to
guarantee they're all retired. How do you solve this currently.

Finally there's still hangcheck and all that, so if we go with forbidding
dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't
need to have additional tricky code to force-retire fences. TDR will take
care already (albeit with maybe a slightly annoying long timeout, which
we can shorten to "time out everything immediately" after drm_dev_unplug).

What we definitely can't have is half the drivers doing it one way, and
the other half the other way. So your driver flag to wrap the ioctl
optionally in a drm_dev_enter/exit path is a no-go still I think.

I guess my tldr; is: I definitely see how your current approach gives
quicker results for amdgpu right now, but long term I'm seeing more
positives on the other one. At least I expect less special cases due to
hotunplug with that.

Cheers, Daniel

> 
> Andrey
> 
> > 
> > It is unfortunately even more work until we've reached the goal, but I
> > think it's safest and most flexible approach overall.
> > 
> > Cheers, Daniel
> > 
> > > 
> > > Andrey
> > > 
> > > > 
> > > > Cheers, Daniel
> > > > 
> > > > 
> > > > > 
> > > > > > > 
> > > > > > > I guess we should clarify this in the hotunplug doc?
> > > > > 
> > > > > Agree
> > > > > 
> > > > > > 
> > > > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
> > > > > > really make no sense, and where we're rather confident that all properly
> > > > > > implemented userspace will gracefully handle failures. Like a modeset, or
> > > > > > opening a device, or trying to import a dma-buf or stuff like that which
> > > > > > can already fail in normal operation for any kind of reason.
> > > > > > 
> > > > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
> > > > > > after hotunplug.
> > > > > 
> > > > > As I pointed above, this a bit confuses me given that we already do
> > > > > blanker rejection of IOCTLs if device is unplugged.
> > > > 
> > > > Well I'm confused about this too :-/
> > > > 
> > > > > > And then there's the middle ground, like doing a pageflip or buffer flush,
> > > > > > which I guess some userspace might handle, but risky to inflict those
> > > > > > consequences on them. atomic modeset is especially fun since depending
> > > > > > what you're doing it can be both "failures expected" and "failures not
> > > > > > really expected in normal operation".
> > > > > > 
> > > > > > Also, this really should be consistent across drivers, not solved with a
> > > > > > driver flag for every possible combination.
> > > > > > 
> > > > > > If you look at the current hotunplug kms drivers, they have
> > > > > > drm_dev_enter/exit sprinkled in specific hw callback functions because of
> > > > > > the above problems. But maybe it makes sense to change things in a few
> > > > > > cases. But then we should do it across the board.
> > > > > 
> > > > > So as I understand your preferred approach is that I scope any back_end, HW
> > > > > specific function with drm_dev_enter/exit because that where MMIO
> > > > > access takes place. But besides explicit MMIO access thorough
> > > > > register accessors in the HW back-end there is also indirect MMIO access
> > > > > taking place throughout the code in the driver because of various VRAM
> > > > > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
> > > > > access is spread all over in the driver and even in mid-layers such as
> > > > > TTM and not limited to HW back-end functions. It means it's much harder
> > > > > to spot such places to surgically scope them with drm_dev_enter/exit and
> > > > > also that any new such code introduced will immediately break hot unplug
> > > > > because the developers can't be expected to remember making their code
> > > > > robust to this specific use case. That why when we discussed internally
> > > > > what approach to take to protecting code with drm_dev_enter/exit we
> > > > > opted for using the widest available scope.
> > > > 
> > > > The thing is, you kinda have to anyway. There's enormous amounts of
> > > > asynchronous processing going on. E.g. nonblocking atomic commits also do
> > > > ttm unpinning and fun stuff like that, which if you sync things wrong can
> > > > happen way late. So the door for bad fallout is wide open :-(
> > > > 
> > > > I'm not sure where the right tradeoff is to make sure we catch them all,
> > > > and can make sure with testing that we've indeed caught them all.
> > > > -Daniel
> > > > 
> > 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
> On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
>>
>>
>> On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
>>> On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
>>>>
>>>>
>>>> On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
>>>>> On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
>>>>>>
>>>>>>
>>>>>> On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
>>>>>>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
>>>>>>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
>>>>>>>>> With this calling drm_dev_unplug will flush and block
>>>>>>>>> all in flight IOCTLs
>>>>>>>>>
>>>>>>>>> Also, add feature such that if device supports graceful unplug
>>>>>>>>> we enclose entire IOCTL in SRCU critical section.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>>>>>
>>>>>>>> Nope.
>>>>>>>>
>>>>>>>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
>>>>>>
>>>>>> Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ca0ca5bdab20a4533491c08d91072fe2a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637558908355926609%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=SESZFWQEcQUHGGek8d1cNi9Iwo9XOmXqxg9MieRkxNU%3D&amp;reserved=0
>>>>>> currently in code ?
>>>>>
>>>>> I forgot about this one, again. Thanks for reminding.
>>>>>
>>>>>>>> Especially not with an opt-in flag so that it could be shrugged of as a
>>>>>>>> driver hack. Most of these ioctls should have absolutely no problem
>>>>>>>> working after hotunplug.
>>>>>>>>
>>>>>>>> Also, doing this defeats the point since it pretty much guarantees
>>>>>>>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract
>>>>>>>> is that only execbuf (and even that only when userspace has indicated
>>>>>>>> support for non-recoverable hw ctx) is allowed to fail. Anything else
>>>>>>>> might crash userspace.
>>>>>>
>>>>>> Given that as I pointed above we already fail any IOCTls with -ENODEV
>>>>>> when device is unplugged, it seems those crashes don't happen that
>>>>>> often ? Also, in all my testing I don't think I saw a user space crash
>>>>>> I could attribute to this.
>>>>>
>>>>> I guess it should be ok.
>>>>
>>>> What should be ok ?
>>>
>>> Your approach, but not your patch. If we go with this let's just lift it
>>> to drm_ioctl() as the default behavior. No driver opt-in flag, because
>>> that's definitely worse than any other approach because we really need to
>>> get rid of driver specific behaviour for generic ioctls, especially
>>> anything a compositor will use directly.
>>>
>>>>> My reasons for making this work is both less trouble for userspace (did
>>>>> you test with various wayland compositors out there, not just amdgpu x86
>>>>
>>>> I didn't - will give it a try.
>>
>> Weston worked without crashes, run the egl tester cube there.
>>
>>>>
>>>>> driver?), but also testing.
>>>>>
>>>>> We still need a bunch of these checks in various places or you'll wait a
>>>>> very long time for a pending modeset or similar to complete. Being able to
>>>>> run that code easily after hotunplug has completed should help a lot with
>>>>> testing.
>>>>>
>>>>> Plus various drivers already acquired drm_dev_enter/exit and now I wonder
>>>>> whether that was properly tested or not ...
>>>>>
>>>>> I guess maybe we need a drm module option to disable this check, so that
>>>>> we can exercise the code as if the ioctl has raced with hotunplug at the
>>>>> worst possible moment.
>>>>>
>>>>> Also atomic is really tricky here: I assume your testing has just done
>>>>> normal synchronous commits, but anything that goes through atomic can be
>>>>> done nonblocking in a separate thread. Which the ioctl catch-all here wont
>>>>> capture.
>>>>
>>>> Yes, async commit was on my mind and thanks for reminding me. Indeed
>>>> I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
>>>> drm_dev_enter/exit. Note that i have a bunch of patches, all name's
>>>> starting with 'Scope....' that just methodically put all the background
>>>> work items and timers the drivers schedules in drm_dev_enter/exit scope.
>>>> This was supposed to be part of the 'Scope Display code' patch.
>>>
>>> That's too much. You still have to arrange that the flip completion event
>>> gets sent out. So it's a bit tricky.
>>>
>>> In other places the same problem applies, e.g. probe functions need to
>>> make sure they report "disconnected".
>>
>> I see, well, this is all part of KMS support which I defer for now
>> anyway. Will tackle it then.
>>
>>>
>>>>>>>> You probably need similar (and very precisely defined) rules for amdgpu.
>>>>>>>> And those must definitely exclude any shard ioctls from randomly failing
>>>>>>>> with EIO, because that just kills the box and defeats the point of trying
>>>>>>>> to gracefully handling hotunplug and making sure userspace has a chance of
>>>>>>>> survival. E.g. for atomic everything should continue, including flip
>>>>>>>> completion, but we set all outputs to "disconnected" and send out the
>>>>>>>> uevent. Maybe crtc enabling can fail too, but that can also be handled
>>>>>>>> through the async status we're using to signal DP link failures to
>>>>>>>> userspace.
>>>>>>
>>>>>> As I pointed before, because of the complexity of the topic I prefer to
>>>>>> take it step by step and solve first for secondary device use case, not
>>>>>> for primary, display attached device.
>>>>>
>>>>> Yeah makes sense. But then I think the right patch is to roll this out for
>>>>> all drivers, properly justified with existing code. Not behind a driver
>>>>> flag, because with all these different compositors the last thing we want
>>>>> is a proliferation of driver-specific behaviour. That's imo the worst
>>>>> option of all of them and needs to be avoided.
>>>>
>>>> So this kind of patch would be acceptable to you if I unconditionally
>>>> scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
>>>> I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ca0ca5bdab20a4533491c08d91072fe2a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637558908355926609%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=62f4gdl3lQH0ap58HTyv47zxALjaa5Td%2BysskR83rig%3D&amp;reserved=0
>>>> Before setting drm_dev_unplug I go through a whole process of signalling
>>>> all possible fences in the system which some one some where might be
>>>> waiting on. My concern is that in the absence of HW those fences won't
>>>> signal and so unless I signal them myself srcu_synchrionize in
>>>> drm_dev_unplug will hang waiting for any such code scoped by
>>>> drm_dev_enter/exit.
>>>
>>> Uh right. I forgot about this.
>>>
>>> Which would kinda mean the top level scope is maybe not the best idea, and
>>> perhaps we should indeed drill it down. But then the testing issue
>>> definitely gets a lot worse.
>>>
>>> So what if we'd push that drm_dev_is_unplugged check down into ioctls?
>>> Then we can make a case-by case decision whether it should be converted to
>>> drm_dev_enter/exit, needs to be pushed down further into drivers (due to
>>> fence wait issues) or other concerns?
>>>
>>> Also I guess we need to have a subsystem wide rule on whether you need to
>>> force complete all fences before you call drm_dev_unplug, or afterwards.
>>
>> I don't see how you can handle it afterwards. If a thread is stuck in
>> dma_fence_wait in non interruptible wait (any kernel thread) and with no
>> timeout there is nothing you can do to stop the wait. Any such code
>> scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
>> The only way then is to preemptively force signal all such fences before
>> calling drm_dev_unplug - as I do in the above mentioned patch.
> 
> Yeah, which is why I don't think top-level drm_dev_enter/exit is a good
> idea.
> 
>>> If we have mixed behaviour on this there will be disappointment. And since
>>> hotunplug and dma_fence completion are both userspace visible that
>>> inconsistency might have bigger impact.
>>>
>>> This is all very tricky indeed :-/
>>>
>>> btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
>>> with that: We could do the same trick we've done for DRM_UNLOCKED:
>>> - drm_dev_enter/exit is called for any ioctl that has not set the
>>>     DRM_HOTUNPLUG_SAFE flag
>>> - for drm core ioctls we push them into all ioctls and decide how to
>>>     handle/where (with the aim to have the least amount of code flow
>>>     different during hotunplug vs after hotunplug has finished, to reduce
>>>     testing scope)
>>> - then we make DRM_HOTUNPLUG_SAFE the implied default
>>>
>>> This would have us left with render ioctls, and I think the defensive
>>> assumption there is that they're all hotunplug safe. We might hang on a
>>> fence wait, but that's fixable, and it's better than blowing up on a
>>> use-after-free security bug.
>>>
>>> Thoughts?
>>
>> I don't fully see a difference between the approach described above and
>> the full drill down to each driver and even within the driver, to the HW
>> back-ends - what criteria I would use to decide if for a given IOCTL i
>> scope with drm_dev_enter/exit at the highest level while for another
>> i go all the way down ? If we would agree that signaling the fences
>> preemptively before engaging drm_dev_unplug is generically the right
>> approach maybe we can then scope drm_ioctl unconditionally with
>> drm_dev_enter/exit and then for each driver go through the same process
>> I do for amdgpu - writing driver specific function which takes care of
>> all the fences. We could then just create a drm callback which would
>> be called from drm_ioctl before drm_dev_unplug is called.
> 
> So I see the appeal of just nuking all the fences, but I'm not sure that's
> a good plan. We've done this in the old i915 gpu reset code too, and the
> issue is it's defacto inverting the locking. But also the hw is truly
> gone, so it also makes sense.
> 
> The problem is a bit roll-out, if we state that dma_fence_wait is allowed
> with a drm_dev_enter/exit, then all drivers need to force-retire their
> fences.
> 
> The other option would be that we require that dma_fence_wait is _not_
> allowed in drm_dev_enter/exit, and that therefore these areas must be
> marked up more fine-grained to avoid deadlocks. I like this more from the
> testing aspect (it makes it easier to be reasonable sure your code handles
> concurrent hotunplug), but also it's pretty easy to validate with the
> dma_fence lockdep annotations we have I think.

They key question as I see it - is it ok for a device to be unplugged
while it's driver has anywhere in it's code a dma_fence_wait
waiting for work completion from this device. The answers seems to me
is no, the HW is gone, this fence will never signal and so you will be
left with indefinitely hanged code thread with all it's unreleased
resources. If i am correct in the above statement then avoiding scoping
code with drm_dev_enter/exit because a dma_fence_wait might be there in 
the middle
just hides the problem. Also, then the only solution for each driver
wanting to support hot-unplug is to force retire all it's HW
fences once it's notified of device removal.

> 
> A third reasons for not requiring force-retiring of dma_fence before
> drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new
> fences from happening, but until you've stopped new fences it's hard to
> guarantee they're all retired. How do you solve this currently.

See amdgpu_finilize_device_fences in 
https://patchwork.ozlabs.org/project/linux-pci/patch/20210428151207.1212258-20-andrey.grodzovsky@amd.com/ 
I think the steps described there answer your
concern here.

> 
> Finally there's still hangcheck and all that, so if we go with forbidding
> dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't
> need to have additional tricky code to force-retire fences. TDR will take
> care already (albeit with maybe a slightly annoying long timeout, which
> we can shorten to "time out everything immediately" after drm_dev_unplug).

I am not aware of TDR handlers  that do it today, at least we don't,
we don't check that if device is gone let's instead of resetting the device
and resubmit jobs just force retire all the HW fences. In any case, this
can and i think should be done in pci remove callback because this is
the place that supposed to handle device extraction. I for example in 
amdgpu_finilize_device_fences just block all TDRs from taking place as 
first step in the process. If other drivers want to force retire fences
in their TDR handlers they still need to block and wait for all such
TDRs in their pci_remove handler.

> 
> What we definitely can't have is half the drivers doing it one way, and
> the other half the other way. So your driver flag to wrap the ioctl
> optionally in a drm_dev_enter/exit path is a no-go still I think.
> 
> I guess my tldr; is: I definitely see how your current approach gives
> quicker results for amdgpu right now, but long term I'm seeing more
> positives on the other one. At least I expect less special cases due to
> hotunplug with that.

As i expressed my viewpoint above - seems to me any driver in need to
support hot-unplug must force retire it's fences because of need to
unblock all dma_fence waits and so it will not be a special case.

Andrey

> 
> Cheers, Daniel
> 
>>
>> Andrey
>>
>>>
>>> It is unfortunately even more work until we've reached the goal, but I
>>> think it's safest and most flexible approach overall.
>>>
>>> Cheers, Daniel
>>>
>>>>
>>>> Andrey
>>>>
>>>>>
>>>>> Cheers, Daniel
>>>>>
>>>>>
>>>>>>
>>>>>>>>
>>>>>>>> I guess we should clarify this in the hotunplug doc?
>>>>>>
>>>>>> Agree
>>>>>>
>>>>>>>
>>>>>>> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
>>>>>>> really make no sense, and where we're rather confident that all properly
>>>>>>> implemented userspace will gracefully handle failures. Like a modeset, or
>>>>>>> opening a device, or trying to import a dma-buf or stuff like that which
>>>>>>> can already fail in normal operation for any kind of reason.
>>>>>>>
>>>>>>> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
>>>>>>> after hotunplug.
>>>>>>
>>>>>> As I pointed above, this a bit confuses me given that we already do
>>>>>> blanker rejection of IOCTLs if device is unplugged.
>>>>>
>>>>> Well I'm confused about this too :-/
>>>>>
>>>>>>> And then there's the middle ground, like doing a pageflip or buffer flush,
>>>>>>> which I guess some userspace might handle, but risky to inflict those
>>>>>>> consequences on them. atomic modeset is especially fun since depending
>>>>>>> what you're doing it can be both "failures expected" and "failures not
>>>>>>> really expected in normal operation".
>>>>>>>
>>>>>>> Also, this really should be consistent across drivers, not solved with a
>>>>>>> driver flag for every possible combination.
>>>>>>>
>>>>>>> If you look at the current hotunplug kms drivers, they have
>>>>>>> drm_dev_enter/exit sprinkled in specific hw callback functions because of
>>>>>>> the above problems. But maybe it makes sense to change things in a few
>>>>>>> cases. But then we should do it across the board.
>>>>>>
>>>>>> So as I understand your preferred approach is that I scope any back_end, HW
>>>>>> specific function with drm_dev_enter/exit because that where MMIO
>>>>>> access takes place. But besides explicit MMIO access thorough
>>>>>> register accessors in the HW back-end there is also indirect MMIO access
>>>>>> taking place throughout the code in the driver because of various VRAM
>>>>>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>>>>>> access is spread all over in the driver and even in mid-layers such as
>>>>>> TTM and not limited to HW back-end functions. It means it's much harder
>>>>>> to spot such places to surgically scope them with drm_dev_enter/exit and
>>>>>> also that any new such code introduced will immediately break hot unplug
>>>>>> because the developers can't be expected to remember making their code
>>>>>> robust to this specific use case. That why when we discussed internally
>>>>>> what approach to take to protecting code with drm_dev_enter/exit we
>>>>>> opted for using the widest available scope.
>>>>>
>>>>> The thing is, you kinda have to anyway. There's enormous amounts of
>>>>> asynchronous processing going on. E.g. nonblocking atomic commits also do
>>>>> ttm unpinning and fun stuff like that, which if you sync things wrong can
>>>>> happen way late. So the door for bad fallout is wide open :-(
>>>>>
>>>>> I'm not sure where the right tradeoff is to make sure we catch them all,
>>>>> and can make sure with testing that we've indeed caught them all.
>>>>> -Daniel
>>>>>
>>>
> 

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
> 
> 
> On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
> > On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
> > > 
> > > 
> > > On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
> > > > On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
> > > > > 
> > > > > 
> > > > > On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
> > > > > > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
> > > > > > > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
> > > > > > > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> > > > > > > > > > With this calling drm_dev_unplug will flush and block
> > > > > > > > > > all in flight IOCTLs
> > > > > > > > > > 
> > > > > > > > > > Also, add feature such that if device supports graceful unplug
> > > > > > > > > > we enclose entire IOCTL in SRCU critical section.
> > > > > > > > > > 
> > > > > > > > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > > > > > > > > 
> > > > > > > > > Nope.
> > > > > > > > > 
> > > > > > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
> > > > > > > 
> > > > > > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ca0ca5bdab20a4533491c08d91072fe2a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637558908355926609%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=SESZFWQEcQUHGGek8d1cNi9Iwo9XOmXqxg9MieRkxNU%3D&amp;reserved=0
> > > > > > > currently in code ?
> > > > > > 
> > > > > > I forgot about this one, again. Thanks for reminding.
> > > > > > 
> > > > > > > > > Especially not with an opt-in flag so that it could be shrugged of as a
> > > > > > > > > driver hack. Most of these ioctls should have absolutely no problem
> > > > > > > > > working after hotunplug.
> > > > > > > > > 
> > > > > > > > > Also, doing this defeats the point since it pretty much guarantees
> > > > > > > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract
> > > > > > > > > is that only execbuf (and even that only when userspace has indicated
> > > > > > > > > support for non-recoverable hw ctx) is allowed to fail. Anything else
> > > > > > > > > might crash userspace.
> > > > > > > 
> > > > > > > Given that as I pointed above we already fail any IOCTls with -ENODEV
> > > > > > > when device is unplugged, it seems those crashes don't happen that
> > > > > > > often ? Also, in all my testing I don't think I saw a user space crash
> > > > > > > I could attribute to this.
> > > > > > 
> > > > > > I guess it should be ok.
> > > > > 
> > > > > What should be ok ?
> > > > 
> > > > Your approach, but not your patch. If we go with this let's just lift it
> > > > to drm_ioctl() as the default behavior. No driver opt-in flag, because
> > > > that's definitely worse than any other approach because we really need to
> > > > get rid of driver specific behaviour for generic ioctls, especially
> > > > anything a compositor will use directly.
> > > > 
> > > > > > My reasons for making this work is both less trouble for userspace (did
> > > > > > you test with various wayland compositors out there, not just amdgpu x86
> > > > > 
> > > > > I didn't - will give it a try.
> > > 
> > > Weston worked without crashes, run the egl tester cube there.
> > > 
> > > > > 
> > > > > > driver?), but also testing.
> > > > > > 
> > > > > > We still need a bunch of these checks in various places or you'll wait a
> > > > > > very long time for a pending modeset or similar to complete. Being able to
> > > > > > run that code easily after hotunplug has completed should help a lot with
> > > > > > testing.
> > > > > > 
> > > > > > Plus various drivers already acquired drm_dev_enter/exit and now I wonder
> > > > > > whether that was properly tested or not ...
> > > > > > 
> > > > > > I guess maybe we need a drm module option to disable this check, so that
> > > > > > we can exercise the code as if the ioctl has raced with hotunplug at the
> > > > > > worst possible moment.
> > > > > > 
> > > > > > Also atomic is really tricky here: I assume your testing has just done
> > > > > > normal synchronous commits, but anything that goes through atomic can be
> > > > > > done nonblocking in a separate thread. Which the ioctl catch-all here wont
> > > > > > capture.
> > > > > 
> > > > > Yes, async commit was on my mind and thanks for reminding me. Indeed
> > > > > I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
> > > > > drm_dev_enter/exit. Note that i have a bunch of patches, all name's
> > > > > starting with 'Scope....' that just methodically put all the background
> > > > > work items and timers the drivers schedules in drm_dev_enter/exit scope.
> > > > > This was supposed to be part of the 'Scope Display code' patch.
> > > > 
> > > > That's too much. You still have to arrange that the flip completion event
> > > > gets sent out. So it's a bit tricky.
> > > > 
> > > > In other places the same problem applies, e.g. probe functions need to
> > > > make sure they report "disconnected".
> > > 
> > > I see, well, this is all part of KMS support which I defer for now
> > > anyway. Will tackle it then.
> > > 
> > > > 
> > > > > > > > > You probably need similar (and very precisely defined) rules for amdgpu.
> > > > > > > > > And those must definitely exclude any shard ioctls from randomly failing
> > > > > > > > > with EIO, because that just kills the box and defeats the point of trying
> > > > > > > > > to gracefully handling hotunplug and making sure userspace has a chance of
> > > > > > > > > survival. E.g. for atomic everything should continue, including flip
> > > > > > > > > completion, but we set all outputs to "disconnected" and send out the
> > > > > > > > > uevent. Maybe crtc enabling can fail too, but that can also be handled
> > > > > > > > > through the async status we're using to signal DP link failures to
> > > > > > > > > userspace.
> > > > > > > 
> > > > > > > As I pointed before, because of the complexity of the topic I prefer to
> > > > > > > take it step by step and solve first for secondary device use case, not
> > > > > > > for primary, display attached device.
> > > > > > 
> > > > > > Yeah makes sense. But then I think the right patch is to roll this out for
> > > > > > all drivers, properly justified with existing code. Not behind a driver
> > > > > > flag, because with all these different compositors the last thing we want
> > > > > > is a proliferation of driver-specific behaviour. That's imo the worst
> > > > > > option of all of them and needs to be avoided.
> > > > > 
> > > > > So this kind of patch would be acceptable to you if I unconditionally
> > > > > scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
> > > > > I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ca0ca5bdab20a4533491c08d91072fe2a%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637558908355926609%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=62f4gdl3lQH0ap58HTyv47zxALjaa5Td%2BysskR83rig%3D&amp;reserved=0
> > > > > Before setting drm_dev_unplug I go through a whole process of signalling
> > > > > all possible fences in the system which some one some where might be
> > > > > waiting on. My concern is that in the absence of HW those fences won't
> > > > > signal and so unless I signal them myself srcu_synchrionize in
> > > > > drm_dev_unplug will hang waiting for any such code scoped by
> > > > > drm_dev_enter/exit.
> > > > 
> > > > Uh right. I forgot about this.
> > > > 
> > > > Which would kinda mean the top level scope is maybe not the best idea, and
> > > > perhaps we should indeed drill it down. But then the testing issue
> > > > definitely gets a lot worse.
> > > > 
> > > > So what if we'd push that drm_dev_is_unplugged check down into ioctls?
> > > > Then we can make a case-by case decision whether it should be converted to
> > > > drm_dev_enter/exit, needs to be pushed down further into drivers (due to
> > > > fence wait issues) or other concerns?
> > > > 
> > > > Also I guess we need to have a subsystem wide rule on whether you need to
> > > > force complete all fences before you call drm_dev_unplug, or afterwards.
> > > 
> > > I don't see how you can handle it afterwards. If a thread is stuck in
> > > dma_fence_wait in non interruptible wait (any kernel thread) and with no
> > > timeout there is nothing you can do to stop the wait. Any such code
> > > scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
> > > The only way then is to preemptively force signal all such fences before
> > > calling drm_dev_unplug - as I do in the above mentioned patch.
> > 
> > Yeah, which is why I don't think top-level drm_dev_enter/exit is a good
> > idea.
> > 
> > > > If we have mixed behaviour on this there will be disappointment. And since
> > > > hotunplug and dma_fence completion are both userspace visible that
> > > > inconsistency might have bigger impact.
> > > > 
> > > > This is all very tricky indeed :-/
> > > > 
> > > > btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
> > > > with that: We could do the same trick we've done for DRM_UNLOCKED:
> > > > - drm_dev_enter/exit is called for any ioctl that has not set the
> > > >     DRM_HOTUNPLUG_SAFE flag
> > > > - for drm core ioctls we push them into all ioctls and decide how to
> > > >     handle/where (with the aim to have the least amount of code flow
> > > >     different during hotunplug vs after hotunplug has finished, to reduce
> > > >     testing scope)
> > > > - then we make DRM_HOTUNPLUG_SAFE the implied default
> > > > 
> > > > This would have us left with render ioctls, and I think the defensive
> > > > assumption there is that they're all hotunplug safe. We might hang on a
> > > > fence wait, but that's fixable, and it's better than blowing up on a
> > > > use-after-free security bug.
> > > > 
> > > > Thoughts?
> > > 
> > > I don't fully see a difference between the approach described above and
> > > the full drill down to each driver and even within the driver, to the HW
> > > back-ends - what criteria I would use to decide if for a given IOCTL i
> > > scope with drm_dev_enter/exit at the highest level while for another
> > > i go all the way down ? If we would agree that signaling the fences
> > > preemptively before engaging drm_dev_unplug is generically the right
> > > approach maybe we can then scope drm_ioctl unconditionally with
> > > drm_dev_enter/exit and then for each driver go through the same process
> > > I do for amdgpu - writing driver specific function which takes care of
> > > all the fences. We could then just create a drm callback which would
> > > be called from drm_ioctl before drm_dev_unplug is called.
> > 
> > So I see the appeal of just nuking all the fences, but I'm not sure that's
> > a good plan. We've done this in the old i915 gpu reset code too, and the
> > issue is it's defacto inverting the locking. But also the hw is truly
> > gone, so it also makes sense.
> > 
> > The problem is a bit roll-out, if we state that dma_fence_wait is allowed
> > with a drm_dev_enter/exit, then all drivers need to force-retire their
> > fences.
> > 
> > The other option would be that we require that dma_fence_wait is _not_
> > allowed in drm_dev_enter/exit, and that therefore these areas must be
> > marked up more fine-grained to avoid deadlocks. I like this more from the
> > testing aspect (it makes it easier to be reasonable sure your code handles
> > concurrent hotunplug), but also it's pretty easy to validate with the
> > dma_fence lockdep annotations we have I think.
> 
> They key question as I see it - is it ok for a device to be unplugged
> while it's driver has anywhere in it's code a dma_fence_wait
> waiting for work completion from this device. The answers seems to me
> is no, the HW is gone, this fence will never signal and so you will be
> left with indefinitely hanged code thread with all it's unreleased
> resources. If i am correct in the above statement then avoiding scoping
> code with drm_dev_enter/exit because a dma_fence_wait might be there in the
> middle
> just hides the problem. Also, then the only solution for each driver
> wanting to support hot-unplug is to force retire all it's HW
> fences once it's notified of device removal.

At a high level, yes dma_fence must always complete. I don't think we have
a disagreement here on that.

What we're discussing here is the precise sequencing and barriers, where
things get tricky. Requiring that you force-complete all dma_fence that
might be affected before you hotunplug is one solution, the other is
tuning the critical sections that drm_dev_enter/exit annotates.

This isn't about avoiding anything or hiding problems, this is about
locking/synchronization design. And for that we must agree on what is
allowed inside/outside of a critical section for all possible
combinations.

E.g. we're also "hiding" problems with calling dma_fence_wait from
shrinkers/mmu notifiers by forbidding allocations in
dma_fence_begin/end_signalling critical paths.

> > A third reasons for not requiring force-retiring of dma_fence before
> > drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new
> > fences from happening, but until you've stopped new fences it's hard to
> > guarantee they're all retired. How do you solve this currently.
> 
> See amdgpu_finilize_device_fences in https://patchwork.ozlabs.org/project/linux-pci/patch/20210428151207.1212258-20-andrey.grodzovsky@amd.com/
> I think the steps described there answer your
> concern here.

The hard problem is stopping further command submission. Not seeing how
you solve that.

But I'm definitely scared about all the scheduler/tdr interactions you
already have there, and that looks quite a bit like fallout from doing
things the wrong way round.

Also given that drm/scheduler is shared, why can't this be a drm/scheduler
helper function?

> > Finally there's still hangcheck and all that, so if we go with forbidding
> > dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't
> > need to have additional tricky code to force-retire fences. TDR will take
> > care already (albeit with maybe a slightly annoying long timeout, which
> > we can shorten to "time out everything immediately" after drm_dev_unplug).
> 
> I am not aware of TDR handlers  that do it today, at least we don't,
> we don't check that if device is gone let's instead of resetting the device
> and resubmit jobs just force retire all the HW fences. In any case, this
> can and i think should be done in pci remove callback because this is
> the place that supposed to handle device extraction. I for example in
> amdgpu_finilize_device_fences just block all TDRs from taking place as first
> step in the process. If other drivers want to force retire fences
> in their TDR handlers they still need to block and wait for all such
> TDRs in their pci_remove handler.

TDR definitely force-completes the fence that did hang. Of course it'll
take a while until they've all completed this way, but we do have
guaranteed forward progress since we've stopped all further fences from
showing up because drm_dev_unplug is called already.

And yes after drm_dev_unplug you can then force-retire the tdr stuff.

> > What we definitely can't have is half the drivers doing it one way, and
> > the other half the other way. So your driver flag to wrap the ioctl
> > optionally in a drm_dev_enter/exit path is a no-go still I think.
> > 
> > I guess my tldr; is: I definitely see how your current approach gives
> > quicker results for amdgpu right now, but long term I'm seeing more
> > positives on the other one. At least I expect less special cases due to
> > hotunplug with that.
> 
> As i expressed my viewpoint above - seems to me any driver in need to
> support hot-unplug must force retire it's fences because of need to
> unblock all dma_fence waits and so it will not be a special case.

This isn't the special case I meant. It's the very tricky
force-retire-before-you-unplugged-officially which is large scale nasty.

Also if your driver doesn't force-retire already, it's buggy. The
additional need of hotunplug is just that we're trying to force-retire a
bit faster, because we know it's all hopeless. But e.g. i915 already has a
fallback that does this automatically:
- first we reset only the engine/context, keeping everyone else running
- if that doesn't pan out, we reset the entire chip and give up an
  anything that's in-flight, which (iirc, it did so at least in the past)
  force retires everything outstanding.

I think amdgpu only has full chip reset, so your first step tries to
reissue all other tasks. But that's not necessarily how it needs to
happen.

Either way drivers must force retire everything (albeit maybe a bit at a
slow pace) if the hw ceased to work properly already. Hotunplug really
isn't anything new here.
-Daniel
> 
> Andrey
> 
> > 
> > Cheers, Daniel
> > 
> > > 
> > > Andrey
> > > 
> > > > 
> > > > It is unfortunately even more work until we've reached the goal, but I
> > > > think it's safest and most flexible approach overall.
> > > > 
> > > > Cheers, Daniel
> > > > 
> > > > > 
> > > > > Andrey
> > > > > 
> > > > > > 
> > > > > > Cheers, Daniel
> > > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > > 
> > > > > > > > > I guess we should clarify this in the hotunplug doc?
> > > > > > > 
> > > > > > > Agree
> > > > > > > 
> > > > > > > > 
> > > > > > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
> > > > > > > > really make no sense, and where we're rather confident that all properly
> > > > > > > > implemented userspace will gracefully handle failures. Like a modeset, or
> > > > > > > > opening a device, or trying to import a dma-buf or stuff like that which
> > > > > > > > can already fail in normal operation for any kind of reason.
> > > > > > > > 
> > > > > > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
> > > > > > > > after hotunplug.
> > > > > > > 
> > > > > > > As I pointed above, this a bit confuses me given that we already do
> > > > > > > blanker rejection of IOCTLs if device is unplugged.
> > > > > > 
> > > > > > Well I'm confused about this too :-/
> > > > > > 
> > > > > > > > And then there's the middle ground, like doing a pageflip or buffer flush,
> > > > > > > > which I guess some userspace might handle, but risky to inflict those
> > > > > > > > consequences on them. atomic modeset is especially fun since depending
> > > > > > > > what you're doing it can be both "failures expected" and "failures not
> > > > > > > > really expected in normal operation".
> > > > > > > > 
> > > > > > > > Also, this really should be consistent across drivers, not solved with a
> > > > > > > > driver flag for every possible combination.
> > > > > > > > 
> > > > > > > > If you look at the current hotunplug kms drivers, they have
> > > > > > > > drm_dev_enter/exit sprinkled in specific hw callback functions because of
> > > > > > > > the above problems. But maybe it makes sense to change things in a few
> > > > > > > > cases. But then we should do it across the board.
> > > > > > > 
> > > > > > > So as I understand your preferred approach is that I scope any back_end, HW
> > > > > > > specific function with drm_dev_enter/exit because that where MMIO
> > > > > > > access takes place. But besides explicit MMIO access thorough
> > > > > > > register accessors in the HW back-end there is also indirect MMIO access
> > > > > > > taking place throughout the code in the driver because of various VRAM
> > > > > > > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
> > > > > > > access is spread all over in the driver and even in mid-layers such as
> > > > > > > TTM and not limited to HW back-end functions. It means it's much harder
> > > > > > > to spot such places to surgically scope them with drm_dev_enter/exit and
> > > > > > > also that any new such code introduced will immediately break hot unplug
> > > > > > > because the developers can't be expected to remember making their code
> > > > > > > robust to this specific use case. That why when we discussed internally
> > > > > > > what approach to take to protecting code with drm_dev_enter/exit we
> > > > > > > opted for using the widest available scope.
> > > > > > 
> > > > > > The thing is, you kinda have to anyway. There's enormous amounts of
> > > > > > asynchronous processing going on. E.g. nonblocking atomic commits also do
> > > > > > ttm unpinning and fun stuff like that, which if you sync things wrong can
> > > > > > happen way late. So the door for bad fallout is wide open :-(
> > > > > > 
> > > > > > I'm not sure where the right tradeoff is to make sure we catch them all,
> > > > > > and can make sure with testing that we've indeed caught them all.
> > > > > > -Daniel
> > > > > > 
> > > > 
> > 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
> On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
>>
>>
>> On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
>>> On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
>>>>
>>>>
>>>> On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
>>>>> On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
>>>>>>
>>>>>>
>>>>>> On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
>>>>>>> On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
>>>>>>>>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
>>>>>>>>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
>>>>>>>>>>> With this calling drm_dev_unplug will flush and block
>>>>>>>>>>> all in flight IOCTLs
>>>>>>>>>>>
>>>>>>>>>>> Also, add feature such that if device supports graceful unplug
>>>>>>>>>>> we enclose entire IOCTL in SRCU critical section.
>>>>>>>>>>>
>>>>>>>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>>>>>>>
>>>>>>>>>> Nope.
>>>>>>>>>>
>>>>>>>>>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
>>>>>>>>
>>>>>>>> Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ce53ea46e66fa40a0e03f08d911381a05%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637559754928702763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=zMlHiglnn8Vm%2BVxI9Rbk8X%2BTyuokq1x1INbhbRCWK4E%3D&amp;reserved=0
>>>>>>>> currently in code ?
>>>>>>>
>>>>>>> I forgot about this one, again. Thanks for reminding.
>>>>>>>
>>>>>>>>>> Especially not with an opt-in flag so that it could be shrugged of as a
>>>>>>>>>> driver hack. Most of these ioctls should have absolutely no problem
>>>>>>>>>> working after hotunplug.
>>>>>>>>>>
>>>>>>>>>> Also, doing this defeats the point since it pretty much guarantees
>>>>>>>>>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract
>>>>>>>>>> is that only execbuf (and even that only when userspace has indicated
>>>>>>>>>> support for non-recoverable hw ctx) is allowed to fail. Anything else
>>>>>>>>>> might crash userspace.
>>>>>>>>
>>>>>>>> Given that as I pointed above we already fail any IOCTls with -ENODEV
>>>>>>>> when device is unplugged, it seems those crashes don't happen that
>>>>>>>> often ? Also, in all my testing I don't think I saw a user space crash
>>>>>>>> I could attribute to this.
>>>>>>>
>>>>>>> I guess it should be ok.
>>>>>>
>>>>>> What should be ok ?
>>>>>
>>>>> Your approach, but not your patch. If we go with this let's just lift it
>>>>> to drm_ioctl() as the default behavior. No driver opt-in flag, because
>>>>> that's definitely worse than any other approach because we really need to
>>>>> get rid of driver specific behaviour for generic ioctls, especially
>>>>> anything a compositor will use directly.
>>>>>
>>>>>>> My reasons for making this work is both less trouble for userspace (did
>>>>>>> you test with various wayland compositors out there, not just amdgpu x86
>>>>>>
>>>>>> I didn't - will give it a try.
>>>>
>>>> Weston worked without crashes, run the egl tester cube there.
>>>>
>>>>>>
>>>>>>> driver?), but also testing.
>>>>>>>
>>>>>>> We still need a bunch of these checks in various places or you'll wait a
>>>>>>> very long time for a pending modeset or similar to complete. Being able to
>>>>>>> run that code easily after hotunplug has completed should help a lot with
>>>>>>> testing.
>>>>>>>
>>>>>>> Plus various drivers already acquired drm_dev_enter/exit and now I wonder
>>>>>>> whether that was properly tested or not ...
>>>>>>>
>>>>>>> I guess maybe we need a drm module option to disable this check, so that
>>>>>>> we can exercise the code as if the ioctl has raced with hotunplug at the
>>>>>>> worst possible moment.
>>>>>>>
>>>>>>> Also atomic is really tricky here: I assume your testing has just done
>>>>>>> normal synchronous commits, but anything that goes through atomic can be
>>>>>>> done nonblocking in a separate thread. Which the ioctl catch-all here wont
>>>>>>> capture.
>>>>>>
>>>>>> Yes, async commit was on my mind and thanks for reminding me. Indeed
>>>>>> I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
>>>>>> drm_dev_enter/exit. Note that i have a bunch of patches, all name's
>>>>>> starting with 'Scope....' that just methodically put all the background
>>>>>> work items and timers the drivers schedules in drm_dev_enter/exit scope.
>>>>>> This was supposed to be part of the 'Scope Display code' patch.
>>>>>
>>>>> That's too much. You still have to arrange that the flip completion event
>>>>> gets sent out. So it's a bit tricky.
>>>>>
>>>>> In other places the same problem applies, e.g. probe functions need to
>>>>> make sure they report "disconnected".
>>>>
>>>> I see, well, this is all part of KMS support which I defer for now
>>>> anyway. Will tackle it then.
>>>>
>>>>>
>>>>>>>>>> You probably need similar (and very precisely defined) rules for amdgpu.
>>>>>>>>>> And those must definitely exclude any shard ioctls from randomly failing
>>>>>>>>>> with EIO, because that just kills the box and defeats the point of trying
>>>>>>>>>> to gracefully handling hotunplug and making sure userspace has a chance of
>>>>>>>>>> survival. E.g. for atomic everything should continue, including flip
>>>>>>>>>> completion, but we set all outputs to "disconnected" and send out the
>>>>>>>>>> uevent. Maybe crtc enabling can fail too, but that can also be handled
>>>>>>>>>> through the async status we're using to signal DP link failures to
>>>>>>>>>> userspace.
>>>>>>>>
>>>>>>>> As I pointed before, because of the complexity of the topic I prefer to
>>>>>>>> take it step by step and solve first for secondary device use case, not
>>>>>>>> for primary, display attached device.
>>>>>>>
>>>>>>> Yeah makes sense. But then I think the right patch is to roll this out for
>>>>>>> all drivers, properly justified with existing code. Not behind a driver
>>>>>>> flag, because with all these different compositors the last thing we want
>>>>>>> is a proliferation of driver-specific behaviour. That's imo the worst
>>>>>>> option of all of them and needs to be avoided.
>>>>>>
>>>>>> So this kind of patch would be acceptable to you if I unconditionally
>>>>>> scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
>>>>>> I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ce53ea46e66fa40a0e03f08d911381a05%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637559754928702763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=NcUTm%2BttKzbr2yo6PlSZRZ4e5%2BkHF%2BCZJSQyo3m7w7Q%3D&amp;reserved=0
>>>>>> Before setting drm_dev_unplug I go through a whole process of signalling
>>>>>> all possible fences in the system which some one some where might be
>>>>>> waiting on. My concern is that in the absence of HW those fences won't
>>>>>> signal and so unless I signal them myself srcu_synchrionize in
>>>>>> drm_dev_unplug will hang waiting for any such code scoped by
>>>>>> drm_dev_enter/exit.
>>>>>
>>>>> Uh right. I forgot about this.
>>>>>
>>>>> Which would kinda mean the top level scope is maybe not the best idea, and
>>>>> perhaps we should indeed drill it down. But then the testing issue
>>>>> definitely gets a lot worse.
>>>>>
>>>>> So what if we'd push that drm_dev_is_unplugged check down into ioctls?
>>>>> Then we can make a case-by case decision whether it should be converted to
>>>>> drm_dev_enter/exit, needs to be pushed down further into drivers (due to
>>>>> fence wait issues) or other concerns?
>>>>>
>>>>> Also I guess we need to have a subsystem wide rule on whether you need to
>>>>> force complete all fences before you call drm_dev_unplug, or afterwards.
>>>>
>>>> I don't see how you can handle it afterwards. If a thread is stuck in
>>>> dma_fence_wait in non interruptible wait (any kernel thread) and with no
>>>> timeout there is nothing you can do to stop the wait. Any such code
>>>> scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
>>>> The only way then is to preemptively force signal all such fences before
>>>> calling drm_dev_unplug - as I do in the above mentioned patch.
>>>
>>> Yeah, which is why I don't think top-level drm_dev_enter/exit is a good
>>> idea.
>>>
>>>>> If we have mixed behaviour on this there will be disappointment. And since
>>>>> hotunplug and dma_fence completion are both userspace visible that
>>>>> inconsistency might have bigger impact.
>>>>>
>>>>> This is all very tricky indeed :-/
>>>>>
>>>>> btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
>>>>> with that: We could do the same trick we've done for DRM_UNLOCKED:
>>>>> - drm_dev_enter/exit is called for any ioctl that has not set the
>>>>>      DRM_HOTUNPLUG_SAFE flag
>>>>> - for drm core ioctls we push them into all ioctls and decide how to
>>>>>      handle/where (with the aim to have the least amount of code flow
>>>>>      different during hotunplug vs after hotunplug has finished, to reduce
>>>>>      testing scope)
>>>>> - then we make DRM_HOTUNPLUG_SAFE the implied default
>>>>>
>>>>> This would have us left with render ioctls, and I think the defensive
>>>>> assumption there is that they're all hotunplug safe. We might hang on a
>>>>> fence wait, but that's fixable, and it's better than blowing up on a
>>>>> use-after-free security bug.
>>>>>
>>>>> Thoughts?
>>>>
>>>> I don't fully see a difference between the approach described above and
>>>> the full drill down to each driver and even within the driver, to the HW
>>>> back-ends - what criteria I would use to decide if for a given IOCTL i
>>>> scope with drm_dev_enter/exit at the highest level while for another
>>>> i go all the way down ? If we would agree that signaling the fences
>>>> preemptively before engaging drm_dev_unplug is generically the right
>>>> approach maybe we can then scope drm_ioctl unconditionally with
>>>> drm_dev_enter/exit and then for each driver go through the same process
>>>> I do for amdgpu - writing driver specific function which takes care of
>>>> all the fences. We could then just create a drm callback which would
>>>> be called from drm_ioctl before drm_dev_unplug is called.
>>>
>>> So I see the appeal of just nuking all the fences, but I'm not sure that's
>>> a good plan. We've done this in the old i915 gpu reset code too, and the
>>> issue is it's defacto inverting the locking. But also the hw is truly
>>> gone, so it also makes sense.
>>>
>>> The problem is a bit roll-out, if we state that dma_fence_wait is allowed
>>> with a drm_dev_enter/exit, then all drivers need to force-retire their
>>> fences.
>>>
>>> The other option would be that we require that dma_fence_wait is _not_
>>> allowed in drm_dev_enter/exit, and that therefore these areas must be
>>> marked up more fine-grained to avoid deadlocks. I like this more from the
>>> testing aspect (it makes it easier to be reasonable sure your code handles
>>> concurrent hotunplug), but also it's pretty easy to validate with the
>>> dma_fence lockdep annotations we have I think.
>>
>> They key question as I see it - is it ok for a device to be unplugged
>> while it's driver has anywhere in it's code a dma_fence_wait
>> waiting for work completion from this device. The answers seems to me
>> is no, the HW is gone, this fence will never signal and so you will be
>> left with indefinitely hanged code thread with all it's unreleased
>> resources. If i am correct in the above statement then avoiding scoping
>> code with drm_dev_enter/exit because a dma_fence_wait might be there in the
>> middle
>> just hides the problem. Also, then the only solution for each driver
>> wanting to support hot-unplug is to force retire all it's HW
>> fences once it's notified of device removal.
> 
> At a high level, yes dma_fence must always complete. I don't think we have
> a disagreement here on that.
> 
> What we're discussing here is the precise sequencing and barriers, where
> things get tricky. Requiring that you force-complete all dma_fence that
> might be affected before you hotunplug is one solution, the other is
> tuning the critical sections that drm_dev_enter/exit annotates.
> 
> This isn't about avoiding anything or hiding problems, this is about
> locking/synchronization design. And for that we must agree on what is
> allowed inside/outside of a critical section for all possible
> combinations.
> 
> E.g. we're also "hiding" problems with calling dma_fence_wait from
> shrinkers/mmu notifiers by forbidding allocations in
> dma_fence_begin/end_signalling critical paths.
> 
>>> A third reasons for not requiring force-retiring of dma_fence before
>>> drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new
>>> fences from happening, but until you've stopped new fences it's hard to
>>> guarantee they're all retired. How do you solve this currently.
>>
>> See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ozlabs.org%2Fproject%2Flinux-pci%2Fpatch%2F20210428151207.1212258-20-andrey.grodzovsky%40amd.com%2F&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ce53ea46e66fa40a0e03f08d911381a05%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637559754928702763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=pinKT1LMic%2FCEfAlr%2FPAhyBAaDEqpMeeJC%2BPdqUaiL8%3D&amp;reserved=0
>> I think the steps described there answer your
>> concern here.
> 
> The hard problem is stopping further command submission. Not seeing how
> you solve that.

By stopping GPU SW scheduler before force completion of HW fences, see
amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the
comment above it.

> 
> But I'm definitely scared about all the scheduler/tdr interactions you
> already have there, and that looks quite a bit like fallout from doing
> things the wrong way round.
> 
> Also given that drm/scheduler is shared, why can't this be a drm/scheduler
> helper function?

I was thinking about it, what stopped me is that HW fences signaling is
done from driver specific HW fence array. But we could do it generic by
instead iterating sched->pending_list and signaling
s_job->s_fence->parent instead.
You also need to retire scheduler's scheduled fences once you stopped
the schedulers as they are waited on as dependencies for other jobs
submissions (i do take care of it).

> 
>>> Finally there's still hangcheck and all that, so if we go with forbidding
>>> dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't
>>> need to have additional tricky code to force-retire fences. TDR will take
>>> care already (albeit with maybe a slightly annoying long timeout, which
>>> we can shorten to "time out everything immediately" after drm_dev_unplug).
>>
>> I am not aware of TDR handlers  that do it today, at least we don't,
>> we don't check that if device is gone let's instead of resetting the device
>> and resubmit jobs just force retire all the HW fences. In any case, this
>> can and i think should be done in pci remove callback because this is
>> the place that supposed to handle device extraction. I for example in
>> amdgpu_finilize_device_fences just block all TDRs from taking place as first
>> step in the process. If other drivers want to force retire fences
>> in their TDR handlers they still need to block and wait for all such
>> TDRs in their pci_remove handler.
> 
> TDR definitely force-completes the fence that did hang. Of course it'll
> take a while until they've all completed this way, but we do have
> guaranteed forward progress since we've stopped all further fences from
> showing up because drm_dev_unplug is called already.
> 
> And yes after drm_dev_unplug you can then force-retire the tdr stuff.
> 
>>> What we definitely can't have is half the drivers doing it one way, and
>>> the other half the other way. So your driver flag to wrap the ioctl
>>> optionally in a drm_dev_enter/exit path is a no-go still I think.
>>>
>>> I guess my tldr; is: I definitely see how your current approach gives
>>> quicker results for amdgpu right now, but long term I'm seeing more
>>> positives on the other one. At least I expect less special cases due to
>>> hotunplug with that.
>>
>> As i expressed my viewpoint above - seems to me any driver in need to
>> support hot-unplug must force retire it's fences because of need to
>> unblock all dma_fence waits and so it will not be a special case.
> 
> This isn't the special case I meant. It's the very tricky
> force-retire-before-you-unplugged-officially which is large scale nasty.
> 
> Also if your driver doesn't force-retire already, it's buggy. The
> additional need of hotunplug is just that we're trying to force-retire a
> bit faster, because we know it's all hopeless. But e.g. i915 already has a
> fallback that does this automatically:
> - first we reset only the engine/context, keeping everyone else running
> - if that doesn't pan out, we reset the entire chip and give up an
>    anything that's in-flight, which (iirc, it did so at least in the past)
>    force retires everything outstanding.
> 
> I think amdgpu only has full chip reset, so your first step tries to
> reissue all other tasks. But that's not necessarily how it needs to
> happen.
> 
> Either way drivers must force retire everything (albeit maybe a bit at a
> slow pace) if the hw ceased to work properly already. Hotunplug really
> isn't anything new here.
> -Daniel

Let's then agree on the way forward -

You raised before the following suggestion -

"
btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
with that: We could do the same trick we've done for DRM_UNLOCKED:
- drm_dev_enter/exit is called for any ioctl that has not set the
   DRM_HOTUNPLUG_SAFE flag
- for drm core ioctls we push them into all ioctls and decide how to
   handle/where (with the aim to have the least amount of code flow
   different during hotunplug vs after hotunplug has finished, to reduce
   testing scope)
- then we make DRM_HOTUNPLUG_SAFE the implied default
"
My problem here is that I have no good understating, criteria
for how to decide per each ioctl on the right scope of drm_dev_enter/
exit. It depends on whether each next function call can lead somewhere
down the call stack to dma_fence_wait and/or whether it can lead
to registers access. Seems to me very hard to cover and error prone.

Another options which we discussed internally before and is basically
same as current drivers i guess is simply to scope with drm_dev_enter/
exit all the back-end HW specific callbacks. Those are most of the
places MMIO access takes place and by definition no dma_fence_wait
can be there as it's HW specific code. This leaves MMIO
access through pointers (memcpy, and various pointer de-references)
which will need to be protected on case by case, but given that I unmap
all MMIO anyway as last step of PCI remove callback, all of them will
be found by try and error eventually.
I feel more comfortable with this approach as I have a clear
understating of how to deal with it.

P.S Please respond on the question for you on the other thread at
'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released'
about suggestion by Christian of partial up-streaming of this code up to
and before the patches dealing with scoping of drm_dev_enter/exit scoping.

Andrey






>>
>> Andrey
>>
>>>
>>> Cheers, Daniel
>>>
>>>>
>>>> Andrey
>>>>
>>>>>
>>>>> It is unfortunately even more work until we've reached the goal, but I
>>>>> think it's safest and most flexible approach overall.
>>>>>
>>>>> Cheers, Daniel
>>>>>
>>>>>>
>>>>>> Andrey
>>>>>>
>>>>>>>
>>>>>>> Cheers, Daniel
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I guess we should clarify this in the hotunplug doc?
>>>>>>>>
>>>>>>>> Agree
>>>>>>>>
>>>>>>>>>
>>>>>>>>> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
>>>>>>>>> really make no sense, and where we're rather confident that all properly
>>>>>>>>> implemented userspace will gracefully handle failures. Like a modeset, or
>>>>>>>>> opening a device, or trying to import a dma-buf or stuff like that which
>>>>>>>>> can already fail in normal operation for any kind of reason.
>>>>>>>>>
>>>>>>>>> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
>>>>>>>>> after hotunplug.
>>>>>>>>
>>>>>>>> As I pointed above, this a bit confuses me given that we already do
>>>>>>>> blanker rejection of IOCTLs if device is unplugged.
>>>>>>>
>>>>>>> Well I'm confused about this too :-/
>>>>>>>
>>>>>>>>> And then there's the middle ground, like doing a pageflip or buffer flush,
>>>>>>>>> which I guess some userspace might handle, but risky to inflict those
>>>>>>>>> consequences on them. atomic modeset is especially fun since depending
>>>>>>>>> what you're doing it can be both "failures expected" and "failures not
>>>>>>>>> really expected in normal operation".
>>>>>>>>>
>>>>>>>>> Also, this really should be consistent across drivers, not solved with a
>>>>>>>>> driver flag for every possible combination.
>>>>>>>>>
>>>>>>>>> If you look at the current hotunplug kms drivers, they have
>>>>>>>>> drm_dev_enter/exit sprinkled in specific hw callback functions because of
>>>>>>>>> the above problems. But maybe it makes sense to change things in a few
>>>>>>>>> cases. But then we should do it across the board.
>>>>>>>>
>>>>>>>> So as I understand your preferred approach is that I scope any back_end, HW
>>>>>>>> specific function with drm_dev_enter/exit because that where MMIO
>>>>>>>> access takes place. But besides explicit MMIO access thorough
>>>>>>>> register accessors in the HW back-end there is also indirect MMIO access
>>>>>>>> taking place throughout the code in the driver because of various VRAM
>>>>>>>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>>>>>>>> access is spread all over in the driver and even in mid-layers such as
>>>>>>>> TTM and not limited to HW back-end functions. It means it's much harder
>>>>>>>> to spot such places to surgically scope them with drm_dev_enter/exit and
>>>>>>>> also that any new such code introduced will immediately break hot unplug
>>>>>>>> because the developers can't be expected to remember making their code
>>>>>>>> robust to this specific use case. That why when we discussed internally
>>>>>>>> what approach to take to protecting code with drm_dev_enter/exit we
>>>>>>>> opted for using the widest available scope.
>>>>>>>
>>>>>>> The thing is, you kinda have to anyway. There's enormous amounts of
>>>>>>> asynchronous processing going on. E.g. nonblocking atomic commits also do
>>>>>>> ttm unpinning and fun stuff like that, which if you sync things wrong can
>>>>>>> happen way late. So the door for bad fallout is wide open :-(
>>>>>>>
>>>>>>> I'm not sure where the right tradeoff is to make sure we catch them all,
>>>>>>> and can make sure with testing that we've indeed caught them all.
>>>>>>> -Daniel
>>>>>>>
>>>>>
>>>
> 

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Fri, May 07, 2021 at 11:39:49AM -0400, Andrey Grodzovsky wrote:
> 
> 
> On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
> > On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
> > > 
> > > 
> > > On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
> > > > On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
> > > > > 
> > > > > 
> > > > > On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
> > > > > > On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
> > > > > > > > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
> > > > > > > > > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
> > > > > > > > > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> > > > > > > > > > > > With this calling drm_dev_unplug will flush and block
> > > > > > > > > > > > all in flight IOCTLs
> > > > > > > > > > > > 
> > > > > > > > > > > > Also, add feature such that if device supports graceful unplug
> > > > > > > > > > > > we enclose entire IOCTL in SRCU critical section.
> > > > > > > > > > > > 
> > > > > > > > > > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > > > > > > > > > > 
> > > > > > > > > > > Nope.
> > > > > > > > > > > 
> > > > > > > > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
> > > > > > > > > 
> > > > > > > > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ce53ea46e66fa40a0e03f08d911381a05%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637559754928702763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=zMlHiglnn8Vm%2BVxI9Rbk8X%2BTyuokq1x1INbhbRCWK4E%3D&amp;reserved=0
> > > > > > > > > currently in code ?
> > > > > > > > 
> > > > > > > > I forgot about this one, again. Thanks for reminding.
> > > > > > > > 
> > > > > > > > > > > Especially not with an opt-in flag so that it could be shrugged of as a
> > > > > > > > > > > driver hack. Most of these ioctls should have absolutely no problem
> > > > > > > > > > > working after hotunplug.
> > > > > > > > > > > 
> > > > > > > > > > > Also, doing this defeats the point since it pretty much guarantees
> > > > > > > > > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract
> > > > > > > > > > > is that only execbuf (and even that only when userspace has indicated
> > > > > > > > > > > support for non-recoverable hw ctx) is allowed to fail. Anything else
> > > > > > > > > > > might crash userspace.
> > > > > > > > > 
> > > > > > > > > Given that as I pointed above we already fail any IOCTls with -ENODEV
> > > > > > > > > when device is unplugged, it seems those crashes don't happen that
> > > > > > > > > often ? Also, in all my testing I don't think I saw a user space crash
> > > > > > > > > I could attribute to this.
> > > > > > > > 
> > > > > > > > I guess it should be ok.
> > > > > > > 
> > > > > > > What should be ok ?
> > > > > > 
> > > > > > Your approach, but not your patch. If we go with this let's just lift it
> > > > > > to drm_ioctl() as the default behavior. No driver opt-in flag, because
> > > > > > that's definitely worse than any other approach because we really need to
> > > > > > get rid of driver specific behaviour for generic ioctls, especially
> > > > > > anything a compositor will use directly.
> > > > > > 
> > > > > > > > My reasons for making this work is both less trouble for userspace (did
> > > > > > > > you test with various wayland compositors out there, not just amdgpu x86
> > > > > > > 
> > > > > > > I didn't - will give it a try.
> > > > > 
> > > > > Weston worked without crashes, run the egl tester cube there.
> > > > > 
> > > > > > > 
> > > > > > > > driver?), but also testing.
> > > > > > > > 
> > > > > > > > We still need a bunch of these checks in various places or you'll wait a
> > > > > > > > very long time for a pending modeset or similar to complete. Being able to
> > > > > > > > run that code easily after hotunplug has completed should help a lot with
> > > > > > > > testing.
> > > > > > > > 
> > > > > > > > Plus various drivers already acquired drm_dev_enter/exit and now I wonder
> > > > > > > > whether that was properly tested or not ...
> > > > > > > > 
> > > > > > > > I guess maybe we need a drm module option to disable this check, so that
> > > > > > > > we can exercise the code as if the ioctl has raced with hotunplug at the
> > > > > > > > worst possible moment.
> > > > > > > > 
> > > > > > > > Also atomic is really tricky here: I assume your testing has just done
> > > > > > > > normal synchronous commits, but anything that goes through atomic can be
> > > > > > > > done nonblocking in a separate thread. Which the ioctl catch-all here wont
> > > > > > > > capture.
> > > > > > > 
> > > > > > > Yes, async commit was on my mind and thanks for reminding me. Indeed
> > > > > > > I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
> > > > > > > drm_dev_enter/exit. Note that i have a bunch of patches, all name's
> > > > > > > starting with 'Scope....' that just methodically put all the background
> > > > > > > work items and timers the drivers schedules in drm_dev_enter/exit scope.
> > > > > > > This was supposed to be part of the 'Scope Display code' patch.
> > > > > > 
> > > > > > That's too much. You still have to arrange that the flip completion event
> > > > > > gets sent out. So it's a bit tricky.
> > > > > > 
> > > > > > In other places the same problem applies, e.g. probe functions need to
> > > > > > make sure they report "disconnected".
> > > > > 
> > > > > I see, well, this is all part of KMS support which I defer for now
> > > > > anyway. Will tackle it then.
> > > > > 
> > > > > > 
> > > > > > > > > > > You probably need similar (and very precisely defined) rules for amdgpu.
> > > > > > > > > > > And those must definitely exclude any shard ioctls from randomly failing
> > > > > > > > > > > with EIO, because that just kills the box and defeats the point of trying
> > > > > > > > > > > to gracefully handling hotunplug and making sure userspace has a chance of
> > > > > > > > > > > survival. E.g. for atomic everything should continue, including flip
> > > > > > > > > > > completion, but we set all outputs to "disconnected" and send out the
> > > > > > > > > > > uevent. Maybe crtc enabling can fail too, but that can also be handled
> > > > > > > > > > > through the async status we're using to signal DP link failures to
> > > > > > > > > > > userspace.
> > > > > > > > > 
> > > > > > > > > As I pointed before, because of the complexity of the topic I prefer to
> > > > > > > > > take it step by step and solve first for secondary device use case, not
> > > > > > > > > for primary, display attached device.
> > > > > > > > 
> > > > > > > > Yeah makes sense. But then I think the right patch is to roll this out for
> > > > > > > > all drivers, properly justified with existing code. Not behind a driver
> > > > > > > > flag, because with all these different compositors the last thing we want
> > > > > > > > is a proliferation of driver-specific behaviour. That's imo the worst
> > > > > > > > option of all of them and needs to be avoided.
> > > > > > > 
> > > > > > > So this kind of patch would be acceptable to you if I unconditionally
> > > > > > > scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
> > > > > > > I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ce53ea46e66fa40a0e03f08d911381a05%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637559754928702763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=NcUTm%2BttKzbr2yo6PlSZRZ4e5%2BkHF%2BCZJSQyo3m7w7Q%3D&amp;reserved=0
> > > > > > > Before setting drm_dev_unplug I go through a whole process of signalling
> > > > > > > all possible fences in the system which some one some where might be
> > > > > > > waiting on. My concern is that in the absence of HW those fences won't
> > > > > > > signal and so unless I signal them myself srcu_synchrionize in
> > > > > > > drm_dev_unplug will hang waiting for any such code scoped by
> > > > > > > drm_dev_enter/exit.
> > > > > > 
> > > > > > Uh right. I forgot about this.
> > > > > > 
> > > > > > Which would kinda mean the top level scope is maybe not the best idea, and
> > > > > > perhaps we should indeed drill it down. But then the testing issue
> > > > > > definitely gets a lot worse.
> > > > > > 
> > > > > > So what if we'd push that drm_dev_is_unplugged check down into ioctls?
> > > > > > Then we can make a case-by case decision whether it should be converted to
> > > > > > drm_dev_enter/exit, needs to be pushed down further into drivers (due to
> > > > > > fence wait issues) or other concerns?
> > > > > > 
> > > > > > Also I guess we need to have a subsystem wide rule on whether you need to
> > > > > > force complete all fences before you call drm_dev_unplug, or afterwards.
> > > > > 
> > > > > I don't see how you can handle it afterwards. If a thread is stuck in
> > > > > dma_fence_wait in non interruptible wait (any kernel thread) and with no
> > > > > timeout there is nothing you can do to stop the wait. Any such code
> > > > > scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
> > > > > The only way then is to preemptively force signal all such fences before
> > > > > calling drm_dev_unplug - as I do in the above mentioned patch.
> > > > 
> > > > Yeah, which is why I don't think top-level drm_dev_enter/exit is a good
> > > > idea.
> > > > 
> > > > > > If we have mixed behaviour on this there will be disappointment. And since
> > > > > > hotunplug and dma_fence completion are both userspace visible that
> > > > > > inconsistency might have bigger impact.
> > > > > > 
> > > > > > This is all very tricky indeed :-/
> > > > > > 
> > > > > > btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
> > > > > > with that: We could do the same trick we've done for DRM_UNLOCKED:
> > > > > > - drm_dev_enter/exit is called for any ioctl that has not set the
> > > > > >      DRM_HOTUNPLUG_SAFE flag
> > > > > > - for drm core ioctls we push them into all ioctls and decide how to
> > > > > >      handle/where (with the aim to have the least amount of code flow
> > > > > >      different during hotunplug vs after hotunplug has finished, to reduce
> > > > > >      testing scope)
> > > > > > - then we make DRM_HOTUNPLUG_SAFE the implied default
> > > > > > 
> > > > > > This would have us left with render ioctls, and I think the defensive
> > > > > > assumption there is that they're all hotunplug safe. We might hang on a
> > > > > > fence wait, but that's fixable, and it's better than blowing up on a
> > > > > > use-after-free security bug.
> > > > > > 
> > > > > > Thoughts?
> > > > > 
> > > > > I don't fully see a difference between the approach described above and
> > > > > the full drill down to each driver and even within the driver, to the HW
> > > > > back-ends - what criteria I would use to decide if for a given IOCTL i
> > > > > scope with drm_dev_enter/exit at the highest level while for another
> > > > > i go all the way down ? If we would agree that signaling the fences
> > > > > preemptively before engaging drm_dev_unplug is generically the right
> > > > > approach maybe we can then scope drm_ioctl unconditionally with
> > > > > drm_dev_enter/exit and then for each driver go through the same process
> > > > > I do for amdgpu - writing driver specific function which takes care of
> > > > > all the fences. We could then just create a drm callback which would
> > > > > be called from drm_ioctl before drm_dev_unplug is called.
> > > > 
> > > > So I see the appeal of just nuking all the fences, but I'm not sure that's
> > > > a good plan. We've done this in the old i915 gpu reset code too, and the
> > > > issue is it's defacto inverting the locking. But also the hw is truly
> > > > gone, so it also makes sense.
> > > > 
> > > > The problem is a bit roll-out, if we state that dma_fence_wait is allowed
> > > > with a drm_dev_enter/exit, then all drivers need to force-retire their
> > > > fences.
> > > > 
> > > > The other option would be that we require that dma_fence_wait is _not_
> > > > allowed in drm_dev_enter/exit, and that therefore these areas must be
> > > > marked up more fine-grained to avoid deadlocks. I like this more from the
> > > > testing aspect (it makes it easier to be reasonable sure your code handles
> > > > concurrent hotunplug), but also it's pretty easy to validate with the
> > > > dma_fence lockdep annotations we have I think.
> > > 
> > > They key question as I see it - is it ok for a device to be unplugged
> > > while it's driver has anywhere in it's code a dma_fence_wait
> > > waiting for work completion from this device. The answers seems to me
> > > is no, the HW is gone, this fence will never signal and so you will be
> > > left with indefinitely hanged code thread with all it's unreleased
> > > resources. If i am correct in the above statement then avoiding scoping
> > > code with drm_dev_enter/exit because a dma_fence_wait might be there in the
> > > middle
> > > just hides the problem. Also, then the only solution for each driver
> > > wanting to support hot-unplug is to force retire all it's HW
> > > fences once it's notified of device removal.
> > 
> > At a high level, yes dma_fence must always complete. I don't think we have
> > a disagreement here on that.
> > 
> > What we're discussing here is the precise sequencing and barriers, where
> > things get tricky. Requiring that you force-complete all dma_fence that
> > might be affected before you hotunplug is one solution, the other is
> > tuning the critical sections that drm_dev_enter/exit annotates.
> > 
> > This isn't about avoiding anything or hiding problems, this is about
> > locking/synchronization design. And for that we must agree on what is
> > allowed inside/outside of a critical section for all possible
> > combinations.
> > 
> > E.g. we're also "hiding" problems with calling dma_fence_wait from
> > shrinkers/mmu notifiers by forbidding allocations in
> > dma_fence_begin/end_signalling critical paths.
> > 
> > > > A third reasons for not requiring force-retiring of dma_fence before
> > > > drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new
> > > > fences from happening, but until you've stopped new fences it's hard to
> > > > guarantee they're all retired. How do you solve this currently.
> > > 
> > > See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ozlabs.org%2Fproject%2Flinux-pci%2Fpatch%2F20210428151207.1212258-20-andrey.grodzovsky%40amd.com%2F&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7Ce53ea46e66fa40a0e03f08d911381a05%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637559754928702763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=pinKT1LMic%2FCEfAlr%2FPAhyBAaDEqpMeeJC%2BPdqUaiL8%3D&amp;reserved=0
> > > I think the steps described there answer your
> > > concern here.
> > 
> > The hard problem is stopping further command submission. Not seeing how
> > you solve that.
> 
> By stopping GPU SW scheduler before force completion of HW fences, see
> amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the
> comment above it.
> 
> > 
> > But I'm definitely scared about all the scheduler/tdr interactions you
> > already have there, and that looks quite a bit like fallout from doing
> > things the wrong way round.
> > 
> > Also given that drm/scheduler is shared, why can't this be a drm/scheduler
> > helper function?
> 
> I was thinking about it, what stopped me is that HW fences signaling is
> done from driver specific HW fence array. But we could do it generic by
> instead iterating sched->pending_list and signaling
> s_job->s_fence->parent instead.
> You also need to retire scheduler's scheduled fences once you stopped
> the schedulers as they are waited on as dependencies for other jobs
> submissions (i do take care of it).
> 
> > 
> > > > Finally there's still hangcheck and all that, so if we go with forbidding
> > > > dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't
> > > > need to have additional tricky code to force-retire fences. TDR will take
> > > > care already (albeit with maybe a slightly annoying long timeout, which
> > > > we can shorten to "time out everything immediately" after drm_dev_unplug).
> > > 
> > > I am not aware of TDR handlers  that do it today, at least we don't,
> > > we don't check that if device is gone let's instead of resetting the device
> > > and resubmit jobs just force retire all the HW fences. In any case, this
> > > can and i think should be done in pci remove callback because this is
> > > the place that supposed to handle device extraction. I for example in
> > > amdgpu_finilize_device_fences just block all TDRs from taking place as first
> > > step in the process. If other drivers want to force retire fences
> > > in their TDR handlers they still need to block and wait for all such
> > > TDRs in their pci_remove handler.
> > 
> > TDR definitely force-completes the fence that did hang. Of course it'll
> > take a while until they've all completed this way, but we do have
> > guaranteed forward progress since we've stopped all further fences from
> > showing up because drm_dev_unplug is called already.
> > 
> > And yes after drm_dev_unplug you can then force-retire the tdr stuff.
> > 
> > > > What we definitely can't have is half the drivers doing it one way, and
> > > > the other half the other way. So your driver flag to wrap the ioctl
> > > > optionally in a drm_dev_enter/exit path is a no-go still I think.
> > > > 
> > > > I guess my tldr; is: I definitely see how your current approach gives
> > > > quicker results for amdgpu right now, but long term I'm seeing more
> > > > positives on the other one. At least I expect less special cases due to
> > > > hotunplug with that.
> > > 
> > > As i expressed my viewpoint above - seems to me any driver in need to
> > > support hot-unplug must force retire it's fences because of need to
> > > unblock all dma_fence waits and so it will not be a special case.
> > 
> > This isn't the special case I meant. It's the very tricky
> > force-retire-before-you-unplugged-officially which is large scale nasty.
> > 
> > Also if your driver doesn't force-retire already, it's buggy. The
> > additional need of hotunplug is just that we're trying to force-retire a
> > bit faster, because we know it's all hopeless. But e.g. i915 already has a
> > fallback that does this automatically:
> > - first we reset only the engine/context, keeping everyone else running
> > - if that doesn't pan out, we reset the entire chip and give up an
> >    anything that's in-flight, which (iirc, it did so at least in the past)
> >    force retires everything outstanding.
> > 
> > I think amdgpu only has full chip reset, so your first step tries to
> > reissue all other tasks. But that's not necessarily how it needs to
> > happen.
> > 
> > Either way drivers must force retire everything (albeit maybe a bit at a
> > slow pace) if the hw ceased to work properly already. Hotunplug really
> > isn't anything new here.
> > -Daniel
> 
> Let's then agree on the way forward -
> 
> You raised before the following suggestion -
> 
> "
> btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
> with that: We could do the same trick we've done for DRM_UNLOCKED:
> - drm_dev_enter/exit is called for any ioctl that has not set the
>   DRM_HOTUNPLUG_SAFE flag
> - for drm core ioctls we push them into all ioctls and decide how to
>   handle/where (with the aim to have the least amount of code flow
>   different during hotunplug vs after hotunplug has finished, to reduce
>   testing scope)
> - then we make DRM_HOTUNPLUG_SAFE the implied default
> "
> My problem here is that I have no good understating, criteria
> for how to decide per each ioctl on the right scope of drm_dev_enter/
> exit. It depends on whether each next function call can lead somewhere
> down the call stack to dma_fence_wait and/or whether it can lead
> to registers access. Seems to me very hard to cover and error prone.

Tbh, neither do I. This requires a lot of work to analyze.

> Another options which we discussed internally before and is basically
> same as current drivers i guess is simply to scope with drm_dev_enter/
> exit all the back-end HW specific callbacks. Those are most of the
> places MMIO access takes place and by definition no dma_fence_wait
> can be there as it's HW specific code. This leaves MMIO
> access through pointers (memcpy, and various pointer de-references)
> which will need to be protected on case by case, but given that I unmap
> all MMIO anyway as last step of PCI remove callback, all of them will
> be found by try and error eventually.
> I feel more comfortable with this approach as I have a clear
> understating of how to deal with it.

Hm ... I'm maybe failing to see the difference, but at least on the kms
side "put drm_dev_enter/exit into driver callbacks" feels like the right
place for them.

Render (and I guess kfd for amd as the special case) ioctl are different,
especially all the driver specific ones.

So one thing that cross my mind maybe as step 0 is to annotate the rules
for drm_dev_enter/exit using lockdep. With lockdep we can both check
whether a lock is held, but also whether it's not held (but the latter is
only possible with CONFIG_PROVE_LOCKING enabled). I think it would be good
to annotate all the major locks in the kernel against drm_dev_enter/exit:

- dma_fence_wait could check that the drm_dev_enter/exit srcu is _not_
  held.

- because srcu nest _very_ freely there's kinda no real restrictions for
  putting drm_dev_enter/exit within a lock critical section. Might still
  be good to explicitly call out in docs where it's all ok:

  - interrupt handlers (I hope that's the case, otherwise I screwed up)
  - shrinkers/mmu_notifier callbacks

- anything else that's not allowed in within drm_dev_enter/exit. I'm e.g.
  wondering whether we should disallow drm_modeset_lock() or maybe
  dma_resv_lock(), or whether that's too restrictive. It could help quite
  a bit in finding places where the drm_dev_enter/exit section is too
  wide.

- another one is the inverse, but I guess you have that already with
  putting a drm_dev_is_held() into mmio helpers and all that to make sure
  we really have them all caught.

Above is just examples, I think the more we nail down these rules one way
or the other, the better for consistency across drivers. And without
consistency everyone will be forced to write their own mmap helpers
instead of one in ttm, or scheduler cleanup helpers instead of one in
drm/scheduler.

> P.S Please respond on the question for you on the other thread at
> 'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released'
> about suggestion by Christian of partial up-streaming of this code up to
> and before the patches dealing with scoping of drm_dev_enter/exit scoping.

Ok will try, I'm a bit burried unfortunately so thanks for reminder when I
miss something.
-Daniel

> 
> Andrey
> 
> 
> 
> 
> 
> 
> > > 
> > > Andrey
> > > 
> > > > 
> > > > Cheers, Daniel
> > > > 
> > > > > 
> > > > > Andrey
> > > > > 
> > > > > > 
> > > > > > It is unfortunately even more work until we've reached the goal, but I
> > > > > > think it's safest and most flexible approach overall.
> > > > > > 
> > > > > > Cheers, Daniel
> > > > > > 
> > > > > > > 
> > > > > > > Andrey
> > > > > > > 
> > > > > > > > 
> > > > > > > > Cheers, Daniel
> > > > > > > > 
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > I guess we should clarify this in the hotunplug doc?
> > > > > > > > > 
> > > > > > > > > Agree
> > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
> > > > > > > > > > really make no sense, and where we're rather confident that all properly
> > > > > > > > > > implemented userspace will gracefully handle failures. Like a modeset, or
> > > > > > > > > > opening a device, or trying to import a dma-buf or stuff like that which
> > > > > > > > > > can already fail in normal operation for any kind of reason.
> > > > > > > > > > 
> > > > > > > > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
> > > > > > > > > > after hotunplug.
> > > > > > > > > 
> > > > > > > > > As I pointed above, this a bit confuses me given that we already do
> > > > > > > > > blanker rejection of IOCTLs if device is unplugged.
> > > > > > > > 
> > > > > > > > Well I'm confused about this too :-/
> > > > > > > > 
> > > > > > > > > > And then there's the middle ground, like doing a pageflip or buffer flush,
> > > > > > > > > > which I guess some userspace might handle, but risky to inflict those
> > > > > > > > > > consequences on them. atomic modeset is especially fun since depending
> > > > > > > > > > what you're doing it can be both "failures expected" and "failures not
> > > > > > > > > > really expected in normal operation".
> > > > > > > > > > 
> > > > > > > > > > Also, this really should be consistent across drivers, not solved with a
> > > > > > > > > > driver flag for every possible combination.
> > > > > > > > > > 
> > > > > > > > > > If you look at the current hotunplug kms drivers, they have
> > > > > > > > > > drm_dev_enter/exit sprinkled in specific hw callback functions because of
> > > > > > > > > > the above problems. But maybe it makes sense to change things in a few
> > > > > > > > > > cases. But then we should do it across the board.
> > > > > > > > > 
> > > > > > > > > So as I understand your preferred approach is that I scope any back_end, HW
> > > > > > > > > specific function with drm_dev_enter/exit because that where MMIO
> > > > > > > > > access takes place. But besides explicit MMIO access thorough
> > > > > > > > > register accessors in the HW back-end there is also indirect MMIO access
> > > > > > > > > taking place throughout the code in the driver because of various VRAM
> > > > > > > > > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
> > > > > > > > > access is spread all over in the driver and even in mid-layers such as
> > > > > > > > > TTM and not limited to HW back-end functions. It means it's much harder
> > > > > > > > > to spot such places to surgically scope them with drm_dev_enter/exit and
> > > > > > > > > also that any new such code introduced will immediately break hot unplug
> > > > > > > > > because the developers can't be expected to remember making their code
> > > > > > > > > robust to this specific use case. That why when we discussed internally
> > > > > > > > > what approach to take to protecting code with drm_dev_enter/exit we
> > > > > > > > > opted for using the widest available scope.
> > > > > > > > 
> > > > > > > > The thing is, you kinda have to anyway. There's enormous amounts of
> > > > > > > > asynchronous processing going on. E.g. nonblocking atomic commits also do
> > > > > > > > ttm unpinning and fun stuff like that, which if you sync things wrong can
> > > > > > > > happen way late. So the door for bad fallout is wide open :-(
> > > > > > > > 
> > > > > > > > I'm not sure where the right tradeoff is to make sure we catch them all,
> > > > > > > > and can make sure with testing that we've indeed caught them all.
> > > > > > > > -Daniel
> > > > > > > > 
> > > > > > 
> > > > 
> > 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Daniel Vetter]

On Fri, Apr 30, 2021 at 12:10:57PM -0400, Andrey Grodzovsky wrote:
> 
> 
> On 2021-04-30 2:47 a.m., Christian König wrote:
> > 
> > 
> > Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
> > > 
> > > 
> > > On 2021-04-29 3:18 a.m., Christian König wrote:
> > > > I need to take another look at this part when I don't have a
> > > > massive headache any more.
> > > > 
> > > > Maybe split the patch set up into different parts, something like:
> > > > 1. Adding general infrastructure.
> > > > 2. Making sure all memory is unpolated.
> > > > 3. Job and fence handling
> > > 
> > > I am not sure you mean this patch here, maybe another one ?
> > > Also note you already RBed it.
> > 
> > No what I meant was to send out the patches before this one as #1 and #2.
> > 
> > That is the easier stuff which can easily go into the drm-misc-next or
> > amd-staging-drm-next branch.
> > 
> > The scheduler stuff certainly need to go into drm-misc-next.
> > 
> > Christian.
> 
> Got you. I am fine with it. What we have here is a working hot-unplug
> code but, one with potential use after free MMIO ranges frpom the zombie
> device. The followup patches after this patch are all about preventing
> this and so the patch-set up until this patch including, is functional
> on it's own. While it's necessary to solve the above issue, it's has
> complications as can be seen from the discussion with Daniel on later
> patch in this series. Still, in my opinion it's better to rollout some
> initial support to hot-unplug without use after free protection then
> having no support for hot-unplug at all. It will also make the merge
> work easier as I need to constantly rebase the patches on top latest
> kernel and solve new regressions.
> 
> Daniel - given the arguments above can you sound your opinion on this
> approach ?

I'm all for incrementally landing this, because it's really hard and
tricky. We might need to go back to some of the decisions, or clarify
things more, or more headaches and pondering how to fix all the parts
that works best to make sure there's no nasty races right across hotunplug
if you're unlucky enough.

But yeah better aim for something and then readjust than bikeshed forever
out of tree.

Cheers, Daniel

> 
> Andrey
> > 
> > > 
> > > Andrey
> > > 
> > > > 
> > > > Christian.
> > > > 
> > > > Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
> > > > > Problem: If scheduler is already stopped by the time sched_entity
> > > > > is released and entity's job_queue not empty I encountred
> > > > > a hang in drm_sched_entity_flush. This is because
> > > > > drm_sched_entity_is_idle
> > > > > never becomes false.
> > > > > 
> > > > > Fix: In drm_sched_fini detach all sched_entities from the
> > > > > scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
> > > > > Also wakeup all those processes stuck in sched_entity flushing
> > > > > as the scheduler main thread which wakes them up is stopped by now.
> > > > > 
> > > > > v2:
> > > > > Reverse order of drm_sched_rq_remove_entity and marking
> > > > > s_entity as stopped to prevent reinserion back to rq due
> > > > > to race.
> > > > > 
> > > > > v3:
> > > > > Drop drm_sched_rq_remove_entity, only modify entity->stopped
> > > > > and check for it in drm_sched_entity_is_idle
> > > > > 
> > > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > > > > Reviewed-by: Christian König <christian.koenig@amd.com>
> > > > > ---
> > > > >   drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
> > > > >   drivers/gpu/drm/scheduler/sched_main.c   | 24
> > > > > ++++++++++++++++++++++++
> > > > >   2 files changed, 26 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/drivers/gpu/drm/scheduler/sched_entity.c
> > > > > b/drivers/gpu/drm/scheduler/sched_entity.c
> > > > > index f0790e9471d1..cb58f692dad9 100644
> > > > > --- a/drivers/gpu/drm/scheduler/sched_entity.c
> > > > > +++ b/drivers/gpu/drm/scheduler/sched_entity.c
> > > > > @@ -116,7 +116,8 @@ static bool
> > > > > drm_sched_entity_is_idle(struct drm_sched_entity *entity)
> > > > >       rmb(); /* for list_empty to work without lock */
> > > > >       if (list_empty(&entity->list) ||
> > > > > -        spsc_queue_count(&entity->job_queue) == 0)
> > > > > +        spsc_queue_count(&entity->job_queue) == 0 ||
> > > > > +        entity->stopped)
> > > > >           return true;
> > > > >       return false;
> > > > > diff --git a/drivers/gpu/drm/scheduler/sched_main.c
> > > > > b/drivers/gpu/drm/scheduler/sched_main.c
> > > > > index 908b0b56032d..ba087354d0a8 100644
> > > > > --- a/drivers/gpu/drm/scheduler/sched_main.c
> > > > > +++ b/drivers/gpu/drm/scheduler/sched_main.c
> > > > > @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
> > > > >    */
> > > > >   void drm_sched_fini(struct drm_gpu_scheduler *sched)
> > > > >   {
> > > > > +    struct drm_sched_entity *s_entity;
> > > > > +    int i;
> > > > > +
> > > > >       if (sched->thread)
> > > > >           kthread_stop(sched->thread);
> > > > > +    for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >=
> > > > > DRM_SCHED_PRIORITY_MIN; i--) {
> > > > > +        struct drm_sched_rq *rq = &sched->sched_rq[i];
> > > > > +
> > > > > +        if (!rq)
> > > > > +            continue;
> > > > > +
> > > > > +        spin_lock(&rq->lock);
> > > > > +        list_for_each_entry(s_entity, &rq->entities, list)
> > > > > +            /*
> > > > > +             * Prevents reinsertion and marks job_queue as idle,
> > > > > +             * it will removed from rq in drm_sched_entity_fini
> > > > > +             * eventually
> > > > > +             */
> > > > > +            s_entity->stopped = true;
> > > > > +        spin_unlock(&rq->lock);
> > > > > +
> > > > > +    }
> > > > > +
> > > > > +    /* Wakeup everyone stuck in drm_sched_entity_flush for
> > > > > this scheduler */
> > > > > +    wake_up_all(&sched->job_scheduled);
> > > > > +
> > > > >       /* Confirm no work left behind accessing device structures */
> > > > >       cancel_delayed_work_sync(&sched->work_tdr);
> > > > 
> > 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released

[Andrey Grodzovsky]

On 2021-05-07 12:29 p.m., Daniel Vetter wrote:
> On Fri, Apr 30, 2021 at 12:10:57PM -0400, Andrey Grodzovsky wrote:
>>
>>
>> On 2021-04-30 2:47 a.m., Christian König wrote:
>>>
>>>
>>> Am 29.04.21 um 19:06 schrieb Andrey Grodzovsky:
>>>>
>>>>
>>>> On 2021-04-29 3:18 a.m., Christian König wrote:
>>>>> I need to take another look at this part when I don't have a
>>>>> massive headache any more.
>>>>>
>>>>> Maybe split the patch set up into different parts, something like:
>>>>> 1. Adding general infrastructure.
>>>>> 2. Making sure all memory is unpolated.
>>>>> 3. Job and fence handling
>>>>
>>>> I am not sure you mean this patch here, maybe another one ?
>>>> Also note you already RBed it.
>>>
>>> No what I meant was to send out the patches before this one as #1 and #2.
>>>
>>> That is the easier stuff which can easily go into the drm-misc-next or
>>> amd-staging-drm-next branch.
>>>
>>> The scheduler stuff certainly need to go into drm-misc-next.
>>>
>>> Christian.
>>
>> Got you. I am fine with it. What we have here is a working hot-unplug
>> code but, one with potential use after free MMIO ranges frpom the zombie
>> device. The followup patches after this patch are all about preventing
>> this and so the patch-set up until this patch including, is functional
>> on it's own. While it's necessary to solve the above issue, it's has
>> complications as can be seen from the discussion with Daniel on later
>> patch in this series. Still, in my opinion it's better to rollout some
>> initial support to hot-unplug without use after free protection then
>> having no support for hot-unplug at all. It will also make the merge
>> work easier as I need to constantly rebase the patches on top latest
>> kernel and solve new regressions.
>>
>> Daniel - given the arguments above can you sound your opinion on this
>> approach ?
> 
> I'm all for incrementally landing this, because it's really hard and
> tricky. We might need to go back to some of the decisions, or clarify
> things more, or more headaches and pondering how to fix all the parts
> that works best to make sure there's no nasty races right across hotunplug
> if you're unlucky enough.
> 
> But yeah better aim for something and then readjust than bikeshed forever
> out of tree.
> 
> Cheers, Daniel

Thanks, I will send out V6 limited in scope up to here and fixing
any relevant comments.

Andrey

> 
>>
>> Andrey
>>>
>>>>
>>>> Andrey
>>>>
>>>>>
>>>>> Christian.
>>>>>
>>>>> Am 28.04.21 um 17:11 schrieb Andrey Grodzovsky:
>>>>>> Problem: If scheduler is already stopped by the time sched_entity
>>>>>> is released and entity's job_queue not empty I encountred
>>>>>> a hang in drm_sched_entity_flush. This is because
>>>>>> drm_sched_entity_is_idle
>>>>>> never becomes false.
>>>>>>
>>>>>> Fix: In drm_sched_fini detach all sched_entities from the
>>>>>> scheduler's run queues. This will satisfy drm_sched_entity_is_idle.
>>>>>> Also wakeup all those processes stuck in sched_entity flushing
>>>>>> as the scheduler main thread which wakes them up is stopped by now.
>>>>>>
>>>>>> v2:
>>>>>> Reverse order of drm_sched_rq_remove_entity and marking
>>>>>> s_entity as stopped to prevent reinserion back to rq due
>>>>>> to race.
>>>>>>
>>>>>> v3:
>>>>>> Drop drm_sched_rq_remove_entity, only modify entity->stopped
>>>>>> and check for it in drm_sched_entity_is_idle
>>>>>>
>>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>>> Reviewed-by: Christian König <christian.koenig@amd.com>
>>>>>> ---
>>>>>>    drivers/gpu/drm/scheduler/sched_entity.c |  3 ++-
>>>>>>    drivers/gpu/drm/scheduler/sched_main.c   | 24
>>>>>> ++++++++++++++++++++++++
>>>>>>    2 files changed, 26 insertions(+), 1 deletion(-)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/scheduler/sched_entity.c
>>>>>> b/drivers/gpu/drm/scheduler/sched_entity.c
>>>>>> index f0790e9471d1..cb58f692dad9 100644
>>>>>> --- a/drivers/gpu/drm/scheduler/sched_entity.c
>>>>>> +++ b/drivers/gpu/drm/scheduler/sched_entity.c
>>>>>> @@ -116,7 +116,8 @@ static bool
>>>>>> drm_sched_entity_is_idle(struct drm_sched_entity *entity)
>>>>>>        rmb(); /* for list_empty to work without lock */
>>>>>>        if (list_empty(&entity->list) ||
>>>>>> -        spsc_queue_count(&entity->job_queue) == 0)
>>>>>> +        spsc_queue_count(&entity->job_queue) == 0 ||
>>>>>> +        entity->stopped)
>>>>>>            return true;
>>>>>>        return false;
>>>>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c
>>>>>> b/drivers/gpu/drm/scheduler/sched_main.c
>>>>>> index 908b0b56032d..ba087354d0a8 100644
>>>>>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>>>>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>>>>>> @@ -897,9 +897,33 @@ EXPORT_SYMBOL(drm_sched_init);
>>>>>>     */
>>>>>>    void drm_sched_fini(struct drm_gpu_scheduler *sched)
>>>>>>    {
>>>>>> +    struct drm_sched_entity *s_entity;
>>>>>> +    int i;
>>>>>> +
>>>>>>        if (sched->thread)
>>>>>>            kthread_stop(sched->thread);
>>>>>> +    for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >=
>>>>>> DRM_SCHED_PRIORITY_MIN; i--) {
>>>>>> +        struct drm_sched_rq *rq = &sched->sched_rq[i];
>>>>>> +
>>>>>> +        if (!rq)
>>>>>> +            continue;
>>>>>> +
>>>>>> +        spin_lock(&rq->lock);
>>>>>> +        list_for_each_entry(s_entity, &rq->entities, list)
>>>>>> +            /*
>>>>>> +             * Prevents reinsertion and marks job_queue as idle,
>>>>>> +             * it will removed from rq in drm_sched_entity_fini
>>>>>> +             * eventually
>>>>>> +             */
>>>>>> +            s_entity->stopped = true;
>>>>>> +        spin_unlock(&rq->lock);
>>>>>> +
>>>>>> +    }
>>>>>> +
>>>>>> +    /* Wakeup everyone stuck in drm_sched_entity_flush for
>>>>>> this scheduler */
>>>>>> +    wake_up_all(&sched->job_scheduled);
>>>>>> +
>>>>>>        /* Confirm no work left behind accessing device structures */
>>>>>>        cancel_delayed_work_sync(&sched->work_tdr);
>>>>>
>>>
> 

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Andrey Grodzovsky]

On 2021-05-07 12:24 p.m., Daniel Vetter wrote:
> On Fri, May 07, 2021 at 11:39:49AM -0400, Andrey Grodzovsky wrote:
>>
>>
>> On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
>>> On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
>>>>
>>>>
>>>> On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
>>>>> On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
>>>>>>
>>>>>>
>>>>>> On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
>>>>>>> On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
>>>>>>>>> On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
>>>>>>>>>>> On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
>>>>>>>>>>>> On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
>>>>>>>>>>>>> With this calling drm_dev_unplug will flush and block
>>>>>>>>>>>>> all in flight IOCTLs
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, add feature such that if device supports graceful unplug
>>>>>>>>>>>>> we enclose entire IOCTL in SRCU critical section.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
>>>>>>>>>>>>
>>>>>>>>>>>> Nope.
>>>>>>>>>>>>
>>>>>>>>>>>> The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
>>>>>>>>>>
>>>>>>>>>> Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C66e4988eb341427e8b0108d91174a232%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637560014906903277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=rdb3xesAUYYTeqU2WdoZ%2BWLzOuuRdOxBBQNTMMB%2BKB4%3D&amp;reserved=0
>>>>>>>>>> currently in code ?
>>>>>>>>>
>>>>>>>>> I forgot about this one, again. Thanks for reminding.
>>>>>>>>>
>>>>>>>>>>>> Especially not with an opt-in flag so that it could be shrugged of as a
>>>>>>>>>>>> driver hack. Most of these ioctls should have absolutely no problem
>>>>>>>>>>>> working after hotunplug.
>>>>>>>>>>>>
>>>>>>>>>>>> Also, doing this defeats the point since it pretty much guarantees
>>>>>>>>>>>> userspace will die in assert()s and stuff. E.g. on i915 the rough contract
>>>>>>>>>>>> is that only execbuf (and even that only when userspace has indicated
>>>>>>>>>>>> support for non-recoverable hw ctx) is allowed to fail. Anything else
>>>>>>>>>>>> might crash userspace.
>>>>>>>>>>
>>>>>>>>>> Given that as I pointed above we already fail any IOCTls with -ENODEV
>>>>>>>>>> when device is unplugged, it seems those crashes don't happen that
>>>>>>>>>> often ? Also, in all my testing I don't think I saw a user space crash
>>>>>>>>>> I could attribute to this.
>>>>>>>>>
>>>>>>>>> I guess it should be ok.
>>>>>>>>
>>>>>>>> What should be ok ?
>>>>>>>
>>>>>>> Your approach, but not your patch. If we go with this let's just lift it
>>>>>>> to drm_ioctl() as the default behavior. No driver opt-in flag, because
>>>>>>> that's definitely worse than any other approach because we really need to
>>>>>>> get rid of driver specific behaviour for generic ioctls, especially
>>>>>>> anything a compositor will use directly.
>>>>>>>
>>>>>>>>> My reasons for making this work is both less trouble for userspace (did
>>>>>>>>> you test with various wayland compositors out there, not just amdgpu x86
>>>>>>>>
>>>>>>>> I didn't - will give it a try.
>>>>>>
>>>>>> Weston worked without crashes, run the egl tester cube there.
>>>>>>
>>>>>>>>
>>>>>>>>> driver?), but also testing.
>>>>>>>>>
>>>>>>>>> We still need a bunch of these checks in various places or you'll wait a
>>>>>>>>> very long time for a pending modeset or similar to complete. Being able to
>>>>>>>>> run that code easily after hotunplug has completed should help a lot with
>>>>>>>>> testing.
>>>>>>>>>
>>>>>>>>> Plus various drivers already acquired drm_dev_enter/exit and now I wonder
>>>>>>>>> whether that was properly tested or not ...
>>>>>>>>>
>>>>>>>>> I guess maybe we need a drm module option to disable this check, so that
>>>>>>>>> we can exercise the code as if the ioctl has raced with hotunplug at the
>>>>>>>>> worst possible moment.
>>>>>>>>>
>>>>>>>>> Also atomic is really tricky here: I assume your testing has just done
>>>>>>>>> normal synchronous commits, but anything that goes through atomic can be
>>>>>>>>> done nonblocking in a separate thread. Which the ioctl catch-all here wont
>>>>>>>>> capture.
>>>>>>>>
>>>>>>>> Yes, async commit was on my mind and thanks for reminding me. Indeed
>>>>>>>> I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
>>>>>>>> drm_dev_enter/exit. Note that i have a bunch of patches, all name's
>>>>>>>> starting with 'Scope....' that just methodically put all the background
>>>>>>>> work items and timers the drivers schedules in drm_dev_enter/exit scope.
>>>>>>>> This was supposed to be part of the 'Scope Display code' patch.
>>>>>>>
>>>>>>> That's too much. You still have to arrange that the flip completion event
>>>>>>> gets sent out. So it's a bit tricky.
>>>>>>>
>>>>>>> In other places the same problem applies, e.g. probe functions need to
>>>>>>> make sure they report "disconnected".
>>>>>>
>>>>>> I see, well, this is all part of KMS support which I defer for now
>>>>>> anyway. Will tackle it then.
>>>>>>
>>>>>>>
>>>>>>>>>>>> You probably need similar (and very precisely defined) rules for amdgpu.
>>>>>>>>>>>> And those must definitely exclude any shard ioctls from randomly failing
>>>>>>>>>>>> with EIO, because that just kills the box and defeats the point of trying
>>>>>>>>>>>> to gracefully handling hotunplug and making sure userspace has a chance of
>>>>>>>>>>>> survival. E.g. for atomic everything should continue, including flip
>>>>>>>>>>>> completion, but we set all outputs to "disconnected" and send out the
>>>>>>>>>>>> uevent. Maybe crtc enabling can fail too, but that can also be handled
>>>>>>>>>>>> through the async status we're using to signal DP link failures to
>>>>>>>>>>>> userspace.
>>>>>>>>>>
>>>>>>>>>> As I pointed before, because of the complexity of the topic I prefer to
>>>>>>>>>> take it step by step and solve first for secondary device use case, not
>>>>>>>>>> for primary, display attached device.
>>>>>>>>>
>>>>>>>>> Yeah makes sense. But then I think the right patch is to roll this out for
>>>>>>>>> all drivers, properly justified with existing code. Not behind a driver
>>>>>>>>> flag, because with all these different compositors the last thing we want
>>>>>>>>> is a proliferation of driver-specific behaviour. That's imo the worst
>>>>>>>>> option of all of them and needs to be avoided.
>>>>>>>>
>>>>>>>> So this kind of patch would be acceptable to you if I unconditionally
>>>>>>>> scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
>>>>>>>> I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C66e4988eb341427e8b0108d91174a232%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637560014906903277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=e0PWmu%2FYZlPJMajOmR6rxRp%2Fj0w%2FsfdJPnJ6dwDqVag%3D&amp;reserved=0
>>>>>>>> Before setting drm_dev_unplug I go through a whole process of signalling
>>>>>>>> all possible fences in the system which some one some where might be
>>>>>>>> waiting on. My concern is that in the absence of HW those fences won't
>>>>>>>> signal and so unless I signal them myself srcu_synchrionize in
>>>>>>>> drm_dev_unplug will hang waiting for any such code scoped by
>>>>>>>> drm_dev_enter/exit.
>>>>>>>
>>>>>>> Uh right. I forgot about this.
>>>>>>>
>>>>>>> Which would kinda mean the top level scope is maybe not the best idea, and
>>>>>>> perhaps we should indeed drill it down. But then the testing issue
>>>>>>> definitely gets a lot worse.
>>>>>>>
>>>>>>> So what if we'd push that drm_dev_is_unplugged check down into ioctls?
>>>>>>> Then we can make a case-by case decision whether it should be converted to
>>>>>>> drm_dev_enter/exit, needs to be pushed down further into drivers (due to
>>>>>>> fence wait issues) or other concerns?
>>>>>>>
>>>>>>> Also I guess we need to have a subsystem wide rule on whether you need to
>>>>>>> force complete all fences before you call drm_dev_unplug, or afterwards.
>>>>>>
>>>>>> I don't see how you can handle it afterwards. If a thread is stuck in
>>>>>> dma_fence_wait in non interruptible wait (any kernel thread) and with no
>>>>>> timeout there is nothing you can do to stop the wait. Any such code
>>>>>> scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
>>>>>> The only way then is to preemptively force signal all such fences before
>>>>>> calling drm_dev_unplug - as I do in the above mentioned patch.
>>>>>
>>>>> Yeah, which is why I don't think top-level drm_dev_enter/exit is a good
>>>>> idea.
>>>>>
>>>>>>> If we have mixed behaviour on this there will be disappointment. And since
>>>>>>> hotunplug and dma_fence completion are both userspace visible that
>>>>>>> inconsistency might have bigger impact.
>>>>>>>
>>>>>>> This is all very tricky indeed :-/
>>>>>>>
>>>>>>> btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
>>>>>>> with that: We could do the same trick we've done for DRM_UNLOCKED:
>>>>>>> - drm_dev_enter/exit is called for any ioctl that has not set the
>>>>>>>       DRM_HOTUNPLUG_SAFE flag
>>>>>>> - for drm core ioctls we push them into all ioctls and decide how to
>>>>>>>       handle/where (with the aim to have the least amount of code flow
>>>>>>>       different during hotunplug vs after hotunplug has finished, to reduce
>>>>>>>       testing scope)
>>>>>>> - then we make DRM_HOTUNPLUG_SAFE the implied default
>>>>>>>
>>>>>>> This would have us left with render ioctls, and I think the defensive
>>>>>>> assumption there is that they're all hotunplug safe. We might hang on a
>>>>>>> fence wait, but that's fixable, and it's better than blowing up on a
>>>>>>> use-after-free security bug.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>
>>>>>> I don't fully see a difference between the approach described above and
>>>>>> the full drill down to each driver and even within the driver, to the HW
>>>>>> back-ends - what criteria I would use to decide if for a given IOCTL i
>>>>>> scope with drm_dev_enter/exit at the highest level while for another
>>>>>> i go all the way down ? If we would agree that signaling the fences
>>>>>> preemptively before engaging drm_dev_unplug is generically the right
>>>>>> approach maybe we can then scope drm_ioctl unconditionally with
>>>>>> drm_dev_enter/exit and then for each driver go through the same process
>>>>>> I do for amdgpu - writing driver specific function which takes care of
>>>>>> all the fences. We could then just create a drm callback which would
>>>>>> be called from drm_ioctl before drm_dev_unplug is called.
>>>>>
>>>>> So I see the appeal of just nuking all the fences, but I'm not sure that's
>>>>> a good plan. We've done this in the old i915 gpu reset code too, and the
>>>>> issue is it's defacto inverting the locking. But also the hw is truly
>>>>> gone, so it also makes sense.
>>>>>
>>>>> The problem is a bit roll-out, if we state that dma_fence_wait is allowed
>>>>> with a drm_dev_enter/exit, then all drivers need to force-retire their
>>>>> fences.
>>>>>
>>>>> The other option would be that we require that dma_fence_wait is _not_
>>>>> allowed in drm_dev_enter/exit, and that therefore these areas must be
>>>>> marked up more fine-grained to avoid deadlocks. I like this more from the
>>>>> testing aspect (it makes it easier to be reasonable sure your code handles
>>>>> concurrent hotunplug), but also it's pretty easy to validate with the
>>>>> dma_fence lockdep annotations we have I think.
>>>>
>>>> They key question as I see it - is it ok for a device to be unplugged
>>>> while it's driver has anywhere in it's code a dma_fence_wait
>>>> waiting for work completion from this device. The answers seems to me
>>>> is no, the HW is gone, this fence will never signal and so you will be
>>>> left with indefinitely hanged code thread with all it's unreleased
>>>> resources. If i am correct in the above statement then avoiding scoping
>>>> code with drm_dev_enter/exit because a dma_fence_wait might be there in the
>>>> middle
>>>> just hides the problem. Also, then the only solution for each driver
>>>> wanting to support hot-unplug is to force retire all it's HW
>>>> fences once it's notified of device removal.
>>>
>>> At a high level, yes dma_fence must always complete. I don't think we have
>>> a disagreement here on that.
>>>
>>> What we're discussing here is the precise sequencing and barriers, where
>>> things get tricky. Requiring that you force-complete all dma_fence that
>>> might be affected before you hotunplug is one solution, the other is
>>> tuning the critical sections that drm_dev_enter/exit annotates.
>>>
>>> This isn't about avoiding anything or hiding problems, this is about
>>> locking/synchronization design. And for that we must agree on what is
>>> allowed inside/outside of a critical section for all possible
>>> combinations.
>>>
>>> E.g. we're also "hiding" problems with calling dma_fence_wait from
>>> shrinkers/mmu notifiers by forbidding allocations in
>>> dma_fence_begin/end_signalling critical paths.
>>>
>>>>> A third reasons for not requiring force-retiring of dma_fence before
>>>>> drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new
>>>>> fences from happening, but until you've stopped new fences it's hard to
>>>>> guarantee they're all retired. How do you solve this currently.
>>>>
>>>> See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ozlabs.org%2Fproject%2Flinux-pci%2Fpatch%2F20210428151207.1212258-20-andrey.grodzovsky%40amd.com%2F&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C66e4988eb341427e8b0108d91174a232%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637560014906903277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=VEpwZuwoT0lBPJriOYCrZwEFcb0zAnWWpenFaPHsUQI%3D&amp;reserved=0
>>>> I think the steps described there answer your
>>>> concern here.
>>>
>>> The hard problem is stopping further command submission. Not seeing how
>>> you solve that.
>>
>> By stopping GPU SW scheduler before force completion of HW fences, see
>> amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the
>> comment above it.
>>
>>>
>>> But I'm definitely scared about all the scheduler/tdr interactions you
>>> already have there, and that looks quite a bit like fallout from doing
>>> things the wrong way round.
>>>
>>> Also given that drm/scheduler is shared, why can't this be a drm/scheduler
>>> helper function?
>>
>> I was thinking about it, what stopped me is that HW fences signaling is
>> done from driver specific HW fence array. But we could do it generic by
>> instead iterating sched->pending_list and signaling
>> s_job->s_fence->parent instead.
>> You also need to retire scheduler's scheduled fences once you stopped
>> the schedulers as they are waited on as dependencies for other jobs
>> submissions (i do take care of it).
>>
>>>
>>>>> Finally there's still hangcheck and all that, so if we go with forbidding
>>>>> dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't
>>>>> need to have additional tricky code to force-retire fences. TDR will take
>>>>> care already (albeit with maybe a slightly annoying long timeout, which
>>>>> we can shorten to "time out everything immediately" after drm_dev_unplug).
>>>>
>>>> I am not aware of TDR handlers  that do it today, at least we don't,
>>>> we don't check that if device is gone let's instead of resetting the device
>>>> and resubmit jobs just force retire all the HW fences. In any case, this
>>>> can and i think should be done in pci remove callback because this is
>>>> the place that supposed to handle device extraction. I for example in
>>>> amdgpu_finilize_device_fences just block all TDRs from taking place as first
>>>> step in the process. If other drivers want to force retire fences
>>>> in their TDR handlers they still need to block and wait for all such
>>>> TDRs in their pci_remove handler.
>>>
>>> TDR definitely force-completes the fence that did hang. Of course it'll
>>> take a while until they've all completed this way, but we do have
>>> guaranteed forward progress since we've stopped all further fences from
>>> showing up because drm_dev_unplug is called already.
>>>
>>> And yes after drm_dev_unplug you can then force-retire the tdr stuff.
>>>
>>>>> What we definitely can't have is half the drivers doing it one way, and
>>>>> the other half the other way. So your driver flag to wrap the ioctl
>>>>> optionally in a drm_dev_enter/exit path is a no-go still I think.
>>>>>
>>>>> I guess my tldr; is: I definitely see how your current approach gives
>>>>> quicker results for amdgpu right now, but long term I'm seeing more
>>>>> positives on the other one. At least I expect less special cases due to
>>>>> hotunplug with that.
>>>>
>>>> As i expressed my viewpoint above - seems to me any driver in need to
>>>> support hot-unplug must force retire it's fences because of need to
>>>> unblock all dma_fence waits and so it will not be a special case.
>>>
>>> This isn't the special case I meant. It's the very tricky
>>> force-retire-before-you-unplugged-officially which is large scale nasty.
>>>
>>> Also if your driver doesn't force-retire already, it's buggy. The
>>> additional need of hotunplug is just that we're trying to force-retire a
>>> bit faster, because we know it's all hopeless. But e.g. i915 already has a
>>> fallback that does this automatically:
>>> - first we reset only the engine/context, keeping everyone else running
>>> - if that doesn't pan out, we reset the entire chip and give up an
>>>     anything that's in-flight, which (iirc, it did so at least in the past)
>>>     force retires everything outstanding.
>>>
>>> I think amdgpu only has full chip reset, so your first step tries to
>>> reissue all other tasks. But that's not necessarily how it needs to
>>> happen.
>>>
>>> Either way drivers must force retire everything (albeit maybe a bit at a
>>> slow pace) if the hw ceased to work properly already. Hotunplug really
>>> isn't anything new here.
>>> -Daniel
>>
>> Let's then agree on the way forward -
>>
>> You raised before the following suggestion -
>>
>> "
>> btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
>> with that: We could do the same trick we've done for DRM_UNLOCKED:
>> - drm_dev_enter/exit is called for any ioctl that has not set the
>>    DRM_HOTUNPLUG_SAFE flag
>> - for drm core ioctls we push them into all ioctls and decide how to
>>    handle/where (with the aim to have the least amount of code flow
>>    different during hotunplug vs after hotunplug has finished, to reduce
>>    testing scope)
>> - then we make DRM_HOTUNPLUG_SAFE the implied default
>> "
>> My problem here is that I have no good understating, criteria
>> for how to decide per each ioctl on the right scope of drm_dev_enter/
>> exit. It depends on whether each next function call can lead somewhere
>> down the call stack to dma_fence_wait and/or whether it can lead
>> to registers access. Seems to me very hard to cover and error prone.
> 
> Tbh, neither do I. This requires a lot of work to analyze.
> 
>> Another options which we discussed internally before and is basically
>> same as current drivers i guess is simply to scope with drm_dev_enter/
>> exit all the back-end HW specific callbacks. Those are most of the
>> places MMIO access takes place and by definition no dma_fence_wait
>> can be there as it's HW specific code. This leaves MMIO
>> access through pointers (memcpy, and various pointer de-references)
>> which will need to be protected on case by case, but given that I unmap
>> all MMIO anyway as last step of PCI remove callback, all of them will
>> be found by try and error eventually.
>> I feel more comfortable with this approach as I have a clear
>> understating of how to deal with it.
> 
> Hm ... I'm maybe failing to see the difference, but at least on the kms
> side "put drm_dev_enter/exit into driver callbacks" feels like the right
> place for them.
> 
> Render (and I guess kfd for amd as the special case) ioctl are different,
> especially all the driver specific ones.
> 
> So one thing that cross my mind maybe as step 0 is to annotate the rules
> for drm_dev_enter/exit using lockdep. With lockdep we can both check
> whether a lock is held, but also whether it's not held (but the latter is
> only possible with CONFIG_PROVE_LOCKING enabled). I think it would be good
> to annotate all the major locks in the kernel against drm_dev_enter/exit:
> 
> - dma_fence_wait could check that the drm_dev_enter/exit srcu is _not_
>    held.

We can't insert this directly inside dma_fence_wait as a DRM agnostic
layer, are you porpoising to wrap all dma_fence_waits in DRM and drivers
code with thin wrapper calling lockdep_assert_held(drm_unplug_srcu) ?
What about dma_resv objs and dma_resv_wait_timeout_rcu style of waits ?


> 
> - because srcu nest _very_ freely there's kinda no real restrictions for
>    putting drm_dev_enter/exit within a lock critical section. Might still
>    be good to explicitly call out in docs where it's all ok:
> 
>    - interrupt handlers (I hope that's the case, otherwise I screwed up)
>    - shrinkers/mmu_notifier callbacks

Is there a problem scooping these with drm_dev_enter/exit ? Might they
hang indefinitely once a device is gone ?

> 
> - anything else that's not allowed in within drm_dev_enter/exit. I'm e.g.
>    wondering whether we should disallow drm_modeset_lock() or maybe
>    dma_resv_lock(), or whether that's too restrictive. It could help quite
>    a bit in finding places where the drm_dev_enter/exit section is too
>    wide.
> 
> - another one is the inverse, but I guess you have that already with
>    putting a drm_dev_is_held() into mmio helpers and all that to make sure
>    we really have them all caught.

Not sure what drm_dev_is_held means here ? Also what do you mean by MMIO
helpers ? Maybe you meant the page fault helpers like ttm_bo_vm_fault ?

Andrey

> 
> Above is just examples, I think the more we nail down these rules one way
> or the other, the better for consistency across drivers. And without
> consistency everyone will be forced to write their own mmap helpers
> instead of one in ttm, or scheduler cleanup helpers instead of one in
> drm/scheduler.
> 
>> P.S Please respond on the question for you on the other thread at
>> 'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released'
>> about suggestion by Christian of partial up-streaming of this code up to
>> and before the patches dealing with scoping of drm_dev_enter/exit scoping.
> 
> Ok will try, I'm a bit burried unfortunately so thanks for reminder when I
> miss something.
> -Daniel
> 
>>
>> Andrey
>>
>>
>>
>>
>>
>>
>>>>
>>>> Andrey
>>>>
>>>>>
>>>>> Cheers, Daniel
>>>>>
>>>>>>
>>>>>> Andrey
>>>>>>
>>>>>>>
>>>>>>> It is unfortunately even more work until we've reached the goal, but I
>>>>>>> think it's safest and most flexible approach overall.
>>>>>>>
>>>>>>> Cheers, Daniel
>>>>>>>
>>>>>>>>
>>>>>>>> Andrey
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Cheers, Daniel
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I guess we should clarify this in the hotunplug doc?
>>>>>>>>>>
>>>>>>>>>> Agree
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
>>>>>>>>>>> really make no sense, and where we're rather confident that all properly
>>>>>>>>>>> implemented userspace will gracefully handle failures. Like a modeset, or
>>>>>>>>>>> opening a device, or trying to import a dma-buf or stuff like that which
>>>>>>>>>>> can already fail in normal operation for any kind of reason.
>>>>>>>>>>>
>>>>>>>>>>> But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
>>>>>>>>>>> after hotunplug.
>>>>>>>>>>
>>>>>>>>>> As I pointed above, this a bit confuses me given that we already do
>>>>>>>>>> blanker rejection of IOCTLs if device is unplugged.
>>>>>>>>>
>>>>>>>>> Well I'm confused about this too :-/
>>>>>>>>>
>>>>>>>>>>> And then there's the middle ground, like doing a pageflip or buffer flush,
>>>>>>>>>>> which I guess some userspace might handle, but risky to inflict those
>>>>>>>>>>> consequences on them. atomic modeset is especially fun since depending
>>>>>>>>>>> what you're doing it can be both "failures expected" and "failures not
>>>>>>>>>>> really expected in normal operation".
>>>>>>>>>>>
>>>>>>>>>>> Also, this really should be consistent across drivers, not solved with a
>>>>>>>>>>> driver flag for every possible combination.
>>>>>>>>>>>
>>>>>>>>>>> If you look at the current hotunplug kms drivers, they have
>>>>>>>>>>> drm_dev_enter/exit sprinkled in specific hw callback functions because of
>>>>>>>>>>> the above problems. But maybe it makes sense to change things in a few
>>>>>>>>>>> cases. But then we should do it across the board.
>>>>>>>>>>
>>>>>>>>>> So as I understand your preferred approach is that I scope any back_end, HW
>>>>>>>>>> specific function with drm_dev_enter/exit because that where MMIO
>>>>>>>>>> access takes place. But besides explicit MMIO access thorough
>>>>>>>>>> register accessors in the HW back-end there is also indirect MMIO access
>>>>>>>>>> taking place throughout the code in the driver because of various VRAM
>>>>>>>>>> BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
>>>>>>>>>> access is spread all over in the driver and even in mid-layers such as
>>>>>>>>>> TTM and not limited to HW back-end functions. It means it's much harder
>>>>>>>>>> to spot such places to surgically scope them with drm_dev_enter/exit and
>>>>>>>>>> also that any new such code introduced will immediately break hot unplug
>>>>>>>>>> because the developers can't be expected to remember making their code
>>>>>>>>>> robust to this specific use case. That why when we discussed internally
>>>>>>>>>> what approach to take to protecting code with drm_dev_enter/exit we
>>>>>>>>>> opted for using the widest available scope.
>>>>>>>>>
>>>>>>>>> The thing is, you kinda have to anyway. There's enormous amounts of
>>>>>>>>> asynchronous processing going on. E.g. nonblocking atomic commits also do
>>>>>>>>> ttm unpinning and fun stuff like that, which if you sync things wrong can
>>>>>>>>> happen way late. So the door for bad fallout is wide open :-(
>>>>>>>>>
>>>>>>>>> I'm not sure where the right tradeoff is to make sure we catch them all,
>>>>>>>>> and can make sure with testing that we've indeed caught them all.
>>>>>>>>> -Daniel
>>>>>>>>>
>>>>>>>
>>>>>
>>>
> 

Re: [PATCH v5 20/27] drm: Scope all DRM IOCTLs with drm_dev_enter/exit

[Daniel Vetter]

On Fri, May 07, 2021 at 02:00:14PM -0400, Andrey Grodzovsky wrote:
> 
> 
> On 2021-05-07 12:24 p.m., Daniel Vetter wrote:
> > On Fri, May 07, 2021 at 11:39:49AM -0400, Andrey Grodzovsky wrote:
> > > 
> > > 
> > > On 2021-05-07 5:11 a.m., Daniel Vetter wrote:
> > > > On Thu, May 06, 2021 at 12:25:06PM -0400, Andrey Grodzovsky wrote:
> > > > > 
> > > > > 
> > > > > On 2021-05-06 5:40 a.m., Daniel Vetter wrote:
> > > > > > On Fri, Apr 30, 2021 at 01:27:37PM -0400, Andrey Grodzovsky wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > On 2021-04-30 6:25 a.m., Daniel Vetter wrote:
> > > > > > > > On Thu, Apr 29, 2021 at 04:34:55PM -0400, Andrey Grodzovsky wrote:
> > > > > > > > > 
> > > > > > > > > 
> > > > > > > > > On 2021-04-29 3:05 p.m., Daniel Vetter wrote:
> > > > > > > > > > On Thu, Apr 29, 2021 at 12:04:33PM -0400, Andrey Grodzovsky wrote:
> > > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > On 2021-04-29 7:32 a.m., Daniel Vetter wrote:
> > > > > > > > > > > > On Thu, Apr 29, 2021 at 01:23:19PM +0200, Daniel Vetter wrote:
> > > > > > > > > > > > > On Wed, Apr 28, 2021 at 11:12:00AM -0400, Andrey Grodzovsky wrote:
> > > > > > > > > > > > > > With this calling drm_dev_unplug will flush and block
> > > > > > > > > > > > > > all in flight IOCTLs
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Also, add feature such that if device supports graceful unplug
> > > > > > > > > > > > > > we enclose entire IOCTL in SRCU critical section.
> > > > > > > > > > > > > > 
> > > > > > > > > > > > > > Signed-off-by: Andrey Grodzovsky <andrey.grodzovsky@amd.com>
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Nope.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > The idea of drm_dev_enter/exit is to mark up hw access. Not entire ioctl.
> > > > > > > > > > > 
> > > > > > > > > > > Then I am confused why we have https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Felixir.bootlin.com%2Flinux%2Fv5.12%2Fsource%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_ioctl.c%23L826&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C66e4988eb341427e8b0108d91174a232%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637560014906903277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=rdb3xesAUYYTeqU2WdoZ%2BWLzOuuRdOxBBQNTMMB%2BKB4%3D&amp;reserved=0
> > > > > > > > > > > currently in code ?
> > > > > > > > > > 
> > > > > > > > > > I forgot about this one, again. Thanks for reminding.
> > > > > > > > > > 
> > > > > > > > > > > > > Especially not with an opt-in flag so that it could be shrugged of as a
> > > > > > > > > > > > > driver hack. Most of these ioctls should have absolutely no problem
> > > > > > > > > > > > > working after hotunplug.
> > > > > > > > > > > > > 
> > > > > > > > > > > > > Also, doing this defeats the point since it pretty much guarantees
> > > > > > > > > > > > > userspace will die in assert()s and stuff. E.g. on i915 the rough contract
> > > > > > > > > > > > > is that only execbuf (and even that only when userspace has indicated
> > > > > > > > > > > > > support for non-recoverable hw ctx) is allowed to fail. Anything else
> > > > > > > > > > > > > might crash userspace.
> > > > > > > > > > > 
> > > > > > > > > > > Given that as I pointed above we already fail any IOCTls with -ENODEV
> > > > > > > > > > > when device is unplugged, it seems those crashes don't happen that
> > > > > > > > > > > often ? Also, in all my testing I don't think I saw a user space crash
> > > > > > > > > > > I could attribute to this.
> > > > > > > > > > 
> > > > > > > > > > I guess it should be ok.
> > > > > > > > > 
> > > > > > > > > What should be ok ?
> > > > > > > > 
> > > > > > > > Your approach, but not your patch. If we go with this let's just lift it
> > > > > > > > to drm_ioctl() as the default behavior. No driver opt-in flag, because
> > > > > > > > that's definitely worse than any other approach because we really need to
> > > > > > > > get rid of driver specific behaviour for generic ioctls, especially
> > > > > > > > anything a compositor will use directly.
> > > > > > > > 
> > > > > > > > > > My reasons for making this work is both less trouble for userspace (did
> > > > > > > > > > you test with various wayland compositors out there, not just amdgpu x86
> > > > > > > > > 
> > > > > > > > > I didn't - will give it a try.
> > > > > > > 
> > > > > > > Weston worked without crashes, run the egl tester cube there.
> > > > > > > 
> > > > > > > > > 
> > > > > > > > > > driver?), but also testing.
> > > > > > > > > > 
> > > > > > > > > > We still need a bunch of these checks in various places or you'll wait a
> > > > > > > > > > very long time for a pending modeset or similar to complete. Being able to
> > > > > > > > > > run that code easily after hotunplug has completed should help a lot with
> > > > > > > > > > testing.
> > > > > > > > > > 
> > > > > > > > > > Plus various drivers already acquired drm_dev_enter/exit and now I wonder
> > > > > > > > > > whether that was properly tested or not ...
> > > > > > > > > > 
> > > > > > > > > > I guess maybe we need a drm module option to disable this check, so that
> > > > > > > > > > we can exercise the code as if the ioctl has raced with hotunplug at the
> > > > > > > > > > worst possible moment.
> > > > > > > > > > 
> > > > > > > > > > Also atomic is really tricky here: I assume your testing has just done
> > > > > > > > > > normal synchronous commits, but anything that goes through atomic can be
> > > > > > > > > > done nonblocking in a separate thread. Which the ioctl catch-all here wont
> > > > > > > > > > capture.
> > > > > > > > > 
> > > > > > > > > Yes, async commit was on my mind and thanks for reminding me. Indeed
> > > > > > > > > I forgot this but i planned to scope the entire amdgpu_dm_atomic_tail in
> > > > > > > > > drm_dev_enter/exit. Note that i have a bunch of patches, all name's
> > > > > > > > > starting with 'Scope....' that just methodically put all the background
> > > > > > > > > work items and timers the drivers schedules in drm_dev_enter/exit scope.
> > > > > > > > > This was supposed to be part of the 'Scope Display code' patch.
> > > > > > > > 
> > > > > > > > That's too much. You still have to arrange that the flip completion event
> > > > > > > > gets sent out. So it's a bit tricky.
> > > > > > > > 
> > > > > > > > In other places the same problem applies, e.g. probe functions need to
> > > > > > > > make sure they report "disconnected".
> > > > > > > 
> > > > > > > I see, well, this is all part of KMS support which I defer for now
> > > > > > > anyway. Will tackle it then.
> > > > > > > 
> > > > > > > > 
> > > > > > > > > > > > > You probably need similar (and very precisely defined) rules for amdgpu.
> > > > > > > > > > > > > And those must definitely exclude any shard ioctls from randomly failing
> > > > > > > > > > > > > with EIO, because that just kills the box and defeats the point of trying
> > > > > > > > > > > > > to gracefully handling hotunplug and making sure userspace has a chance of
> > > > > > > > > > > > > survival. E.g. for atomic everything should continue, including flip
> > > > > > > > > > > > > completion, but we set all outputs to "disconnected" and send out the
> > > > > > > > > > > > > uevent. Maybe crtc enabling can fail too, but that can also be handled
> > > > > > > > > > > > > through the async status we're using to signal DP link failures to
> > > > > > > > > > > > > userspace.
> > > > > > > > > > > 
> > > > > > > > > > > As I pointed before, because of the complexity of the topic I prefer to
> > > > > > > > > > > take it step by step and solve first for secondary device use case, not
> > > > > > > > > > > for primary, display attached device.
> > > > > > > > > > 
> > > > > > > > > > Yeah makes sense. But then I think the right patch is to roll this out for
> > > > > > > > > > all drivers, properly justified with existing code. Not behind a driver
> > > > > > > > > > flag, because with all these different compositors the last thing we want
> > > > > > > > > > is a proliferation of driver-specific behaviour. That's imo the worst
> > > > > > > > > > option of all of them and needs to be avoided.
> > > > > > > > > 
> > > > > > > > > So this kind of patch would be acceptable to you if I unconditionally
> > > > > > > > > scope the drm_ioctl with drm_dev_enter/exit without the driver flag ?
> > > > > > > > > I am worried to break other drivers with this, see patch https://nam11.safelinks.protection.outlook.com/?url=https:%2F%2Fcgit.freedesktop.org%2F~agrodzov%2Flinux%2Fcommit%2F%3Fh%3Ddrm-misc-next%26id%3Df0c593f35b22ca5bf60ed9e7ce2bf2b80e6c68c6&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C66e4988eb341427e8b0108d91174a232%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637560014906903277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=e0PWmu%2FYZlPJMajOmR6rxRp%2Fj0w%2FsfdJPnJ6dwDqVag%3D&amp;reserved=0
> > > > > > > > > Before setting drm_dev_unplug I go through a whole process of signalling
> > > > > > > > > all possible fences in the system which some one some where might be
> > > > > > > > > waiting on. My concern is that in the absence of HW those fences won't
> > > > > > > > > signal and so unless I signal them myself srcu_synchrionize in
> > > > > > > > > drm_dev_unplug will hang waiting for any such code scoped by
> > > > > > > > > drm_dev_enter/exit.
> > > > > > > > 
> > > > > > > > Uh right. I forgot about this.
> > > > > > > > 
> > > > > > > > Which would kinda mean the top level scope is maybe not the best idea, and
> > > > > > > > perhaps we should indeed drill it down. But then the testing issue
> > > > > > > > definitely gets a lot worse.
> > > > > > > > 
> > > > > > > > So what if we'd push that drm_dev_is_unplugged check down into ioctls?
> > > > > > > > Then we can make a case-by case decision whether it should be converted to
> > > > > > > > drm_dev_enter/exit, needs to be pushed down further into drivers (due to
> > > > > > > > fence wait issues) or other concerns?
> > > > > > > > 
> > > > > > > > Also I guess we need to have a subsystem wide rule on whether you need to
> > > > > > > > force complete all fences before you call drm_dev_unplug, or afterwards.
> > > > > > > 
> > > > > > > I don't see how you can handle it afterwards. If a thread is stuck in
> > > > > > > dma_fence_wait in non interruptible wait (any kernel thread) and with no
> > > > > > > timeout there is nothing you can do to stop the wait. Any such code
> > > > > > > scopped with drm_dev_enter/exit will cause a hang in drm_dev_unplug.
> > > > > > > The only way then is to preemptively force signal all such fences before
> > > > > > > calling drm_dev_unplug - as I do in the above mentioned patch.
> > > > > > 
> > > > > > Yeah, which is why I don't think top-level drm_dev_enter/exit is a good
> > > > > > idea.
> > > > > > 
> > > > > > > > If we have mixed behaviour on this there will be disappointment. And since
> > > > > > > > hotunplug and dma_fence completion are both userspace visible that
> > > > > > > > inconsistency might have bigger impact.
> > > > > > > > 
> > > > > > > > This is all very tricky indeed :-/
> > > > > > > > 
> > > > > > > > btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
> > > > > > > > with that: We could do the same trick we've done for DRM_UNLOCKED:
> > > > > > > > - drm_dev_enter/exit is called for any ioctl that has not set the
> > > > > > > >       DRM_HOTUNPLUG_SAFE flag
> > > > > > > > - for drm core ioctls we push them into all ioctls and decide how to
> > > > > > > >       handle/where (with the aim to have the least amount of code flow
> > > > > > > >       different during hotunplug vs after hotunplug has finished, to reduce
> > > > > > > >       testing scope)
> > > > > > > > - then we make DRM_HOTUNPLUG_SAFE the implied default
> > > > > > > > 
> > > > > > > > This would have us left with render ioctls, and I think the defensive
> > > > > > > > assumption there is that they're all hotunplug safe. We might hang on a
> > > > > > > > fence wait, but that's fixable, and it's better than blowing up on a
> > > > > > > > use-after-free security bug.
> > > > > > > > 
> > > > > > > > Thoughts?
> > > > > > > 
> > > > > > > I don't fully see a difference between the approach described above and
> > > > > > > the full drill down to each driver and even within the driver, to the HW
> > > > > > > back-ends - what criteria I would use to decide if for a given IOCTL i
> > > > > > > scope with drm_dev_enter/exit at the highest level while for another
> > > > > > > i go all the way down ? If we would agree that signaling the fences
> > > > > > > preemptively before engaging drm_dev_unplug is generically the right
> > > > > > > approach maybe we can then scope drm_ioctl unconditionally with
> > > > > > > drm_dev_enter/exit and then for each driver go through the same process
> > > > > > > I do for amdgpu - writing driver specific function which takes care of
> > > > > > > all the fences. We could then just create a drm callback which would
> > > > > > > be called from drm_ioctl before drm_dev_unplug is called.
> > > > > > 
> > > > > > So I see the appeal of just nuking all the fences, but I'm not sure that's
> > > > > > a good plan. We've done this in the old i915 gpu reset code too, and the
> > > > > > issue is it's defacto inverting the locking. But also the hw is truly
> > > > > > gone, so it also makes sense.
> > > > > > 
> > > > > > The problem is a bit roll-out, if we state that dma_fence_wait is allowed
> > > > > > with a drm_dev_enter/exit, then all drivers need to force-retire their
> > > > > > fences.
> > > > > > 
> > > > > > The other option would be that we require that dma_fence_wait is _not_
> > > > > > allowed in drm_dev_enter/exit, and that therefore these areas must be
> > > > > > marked up more fine-grained to avoid deadlocks. I like this more from the
> > > > > > testing aspect (it makes it easier to be reasonable sure your code handles
> > > > > > concurrent hotunplug), but also it's pretty easy to validate with the
> > > > > > dma_fence lockdep annotations we have I think.
> > > > > 
> > > > > They key question as I see it - is it ok for a device to be unplugged
> > > > > while it's driver has anywhere in it's code a dma_fence_wait
> > > > > waiting for work completion from this device. The answers seems to me
> > > > > is no, the HW is gone, this fence will never signal and so you will be
> > > > > left with indefinitely hanged code thread with all it's unreleased
> > > > > resources. If i am correct in the above statement then avoiding scoping
> > > > > code with drm_dev_enter/exit because a dma_fence_wait might be there in the
> > > > > middle
> > > > > just hides the problem. Also, then the only solution for each driver
> > > > > wanting to support hot-unplug is to force retire all it's HW
> > > > > fences once it's notified of device removal.
> > > > 
> > > > At a high level, yes dma_fence must always complete. I don't think we have
> > > > a disagreement here on that.
> > > > 
> > > > What we're discussing here is the precise sequencing and barriers, where
> > > > things get tricky. Requiring that you force-complete all dma_fence that
> > > > might be affected before you hotunplug is one solution, the other is
> > > > tuning the critical sections that drm_dev_enter/exit annotates.
> > > > 
> > > > This isn't about avoiding anything or hiding problems, this is about
> > > > locking/synchronization design. And for that we must agree on what is
> > > > allowed inside/outside of a critical section for all possible
> > > > combinations.
> > > > 
> > > > E.g. we're also "hiding" problems with calling dma_fence_wait from
> > > > shrinkers/mmu notifiers by forbidding allocations in
> > > > dma_fence_begin/end_signalling critical paths.
> > > > 
> > > > > > A third reasons for not requiring force-retiring of dma_fence before
> > > > > > drm_dev_unplug is the races: Before drm_dev_unplug you haven't stopped new
> > > > > > fences from happening, but until you've stopped new fences it's hard to
> > > > > > guarantee they're all retired. How do you solve this currently.
> > > > > 
> > > > > See amdgpu_finilize_device_fences in https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.ozlabs.org%2Fproject%2Flinux-pci%2Fpatch%2F20210428151207.1212258-20-andrey.grodzovsky%40amd.com%2F&amp;data=04%7C01%7Candrey.grodzovsky%40amd.com%7C66e4988eb341427e8b0108d91174a232%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637560014906903277%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=VEpwZuwoT0lBPJriOYCrZwEFcb0zAnWWpenFaPHsUQI%3D&amp;reserved=0
> > > > > I think the steps described there answer your
> > > > > concern here.
> > > > 
> > > > The hard problem is stopping further command submission. Not seeing how
> > > > you solve that.
> > > 
> > > By stopping GPU SW scheduler before force completion of HW fences, see
> > > amdgpu_finilize_device_fences->amdgpu_fence_driver_fini_hw and the
> > > comment above it.
> > > 
> > > > 
> > > > But I'm definitely scared about all the scheduler/tdr interactions you
> > > > already have there, and that looks quite a bit like fallout from doing
> > > > things the wrong way round.
> > > > 
> > > > Also given that drm/scheduler is shared, why can't this be a drm/scheduler
> > > > helper function?
> > > 
> > > I was thinking about it, what stopped me is that HW fences signaling is
> > > done from driver specific HW fence array. But we could do it generic by
> > > instead iterating sched->pending_list and signaling
> > > s_job->s_fence->parent instead.
> > > You also need to retire scheduler's scheduled fences once you stopped
> > > the schedulers as they are waited on as dependencies for other jobs
> > > submissions (i do take care of it).
> > > 
> > > > 
> > > > > > Finally there's still hangcheck and all that, so if we go with forbidding
> > > > > > dma_fence_wait from within drm_dev_enter/exit sections, then drivers don't
> > > > > > need to have additional tricky code to force-retire fences. TDR will take
> > > > > > care already (albeit with maybe a slightly annoying long timeout, which
> > > > > > we can shorten to "time out everything immediately" after drm_dev_unplug).
> > > > > 
> > > > > I am not aware of TDR handlers  that do it today, at least we don't,
> > > > > we don't check that if device is gone let's instead of resetting the device
> > > > > and resubmit jobs just force retire all the HW fences. In any case, this
> > > > > can and i think should be done in pci remove callback because this is
> > > > > the place that supposed to handle device extraction. I for example in
> > > > > amdgpu_finilize_device_fences just block all TDRs from taking place as first
> > > > > step in the process. If other drivers want to force retire fences
> > > > > in their TDR handlers they still need to block and wait for all such
> > > > > TDRs in their pci_remove handler.
> > > > 
> > > > TDR definitely force-completes the fence that did hang. Of course it'll
> > > > take a while until they've all completed this way, but we do have
> > > > guaranteed forward progress since we've stopped all further fences from
> > > > showing up because drm_dev_unplug is called already.
> > > > 
> > > > And yes after drm_dev_unplug you can then force-retire the tdr stuff.
> > > > 
> > > > > > What we definitely can't have is half the drivers doing it one way, and
> > > > > > the other half the other way. So your driver flag to wrap the ioctl
> > > > > > optionally in a drm_dev_enter/exit path is a no-go still I think.
> > > > > > 
> > > > > > I guess my tldr; is: I definitely see how your current approach gives
> > > > > > quicker results for amdgpu right now, but long term I'm seeing more
> > > > > > positives on the other one. At least I expect less special cases due to
> > > > > > hotunplug with that.
> > > > > 
> > > > > As i expressed my viewpoint above - seems to me any driver in need to
> > > > > support hot-unplug must force retire it's fences because of need to
> > > > > unblock all dma_fence waits and so it will not be a special case.
> > > > 
> > > > This isn't the special case I meant. It's the very tricky
> > > > force-retire-before-you-unplugged-officially which is large scale nasty.
> > > > 
> > > > Also if your driver doesn't force-retire already, it's buggy. The
> > > > additional need of hotunplug is just that we're trying to force-retire a
> > > > bit faster, because we know it's all hopeless. But e.g. i915 already has a
> > > > fallback that does this automatically:
> > > > - first we reset only the engine/context, keeping everyone else running
> > > > - if that doesn't pan out, we reset the entire chip and give up an
> > > >     anything that's in-flight, which (iirc, it did so at least in the past)
> > > >     force retires everything outstanding.
> > > > 
> > > > I think amdgpu only has full chip reset, so your first step tries to
> > > > reissue all other tasks. But that's not necessarily how it needs to
> > > > happen.
> > > > 
> > > > Either way drivers must force retire everything (albeit maybe a bit at a
> > > > slow pace) if the hw ceased to work properly already. Hotunplug really
> > > > isn't anything new here.
> > > > -Daniel
> > > 
> > > Let's then agree on the way forward -
> > > 
> > > You raised before the following suggestion -
> > > 
> > > "
> > > btw for the "gradual pushing drm_dev_enter into ioctl" approach, if we go
> > > with that: We could do the same trick we've done for DRM_UNLOCKED:
> > > - drm_dev_enter/exit is called for any ioctl that has not set the
> > >    DRM_HOTUNPLUG_SAFE flag
> > > - for drm core ioctls we push them into all ioctls and decide how to
> > >    handle/where (with the aim to have the least amount of code flow
> > >    different during hotunplug vs after hotunplug has finished, to reduce
> > >    testing scope)
> > > - then we make DRM_HOTUNPLUG_SAFE the implied default
> > > "
> > > My problem here is that I have no good understating, criteria
> > > for how to decide per each ioctl on the right scope of drm_dev_enter/
> > > exit. It depends on whether each next function call can lead somewhere
> > > down the call stack to dma_fence_wait and/or whether it can lead
> > > to registers access. Seems to me very hard to cover and error prone.
> > 
> > Tbh, neither do I. This requires a lot of work to analyze.
> > 
> > > Another options which we discussed internally before and is basically
> > > same as current drivers i guess is simply to scope with drm_dev_enter/
> > > exit all the back-end HW specific callbacks. Those are most of the
> > > places MMIO access takes place and by definition no dma_fence_wait
> > > can be there as it's HW specific code. This leaves MMIO
> > > access through pointers (memcpy, and various pointer de-references)
> > > which will need to be protected on case by case, but given that I unmap
> > > all MMIO anyway as last step of PCI remove callback, all of them will
> > > be found by try and error eventually.
> > > I feel more comfortable with this approach as I have a clear
> > > understating of how to deal with it.
> > 
> > Hm ... I'm maybe failing to see the difference, but at least on the kms
> > side "put drm_dev_enter/exit into driver callbacks" feels like the right
> > place for them.
> > 
> > Render (and I guess kfd for amd as the special case) ioctl are different,
> > especially all the driver specific ones.
> > 
> > So one thing that cross my mind maybe as step 0 is to annotate the rules
> > for drm_dev_enter/exit using lockdep. With lockdep we can both check
> > whether a lock is held, but also whether it's not held (but the latter is
> > only possible with CONFIG_PROVE_LOCKING enabled). I think it would be good
> > to annotate all the major locks in the kernel against drm_dev_enter/exit:
> > 
> > - dma_fence_wait could check that the drm_dev_enter/exit srcu is _not_
> >    held.
> 
> We can't insert this directly inside dma_fence_wait as a DRM agnostic
> layer, are you porpoising to wrap all dma_fence_waits in DRM and drivers
> code with thin wrapper calling lockdep_assert_held(drm_unplug_srcu) ?
> What about dma_resv objs and dma_resv_wait_timeout_rcu style of waits ?

Hm ... yeah this doesn't work. Least because we also don't have a generic
way to get from dma_fence to anything resembling a drm_device or any other
device. Which is really annoying.

The dma_resv waits boil down to dma_fence waits underneath, which is why I
wanted to annotate dma_fence_wait ...

The problem is also that because srcu sections nest completely freely I
don't think we can teach lockdep that it's not allowed to hold the srcu
read side lock when we call dma_fence_wait. The latter has it's own
lockdep_key, so maybe annotating things that way is possible. Not sure.
Lockdep has some big limitations for read side locks.

Maybe you can figure out something, my current bandwidth for screaming at
clever lockdep annotations is a bit thin :-(

> > - because srcu nest _very_ freely there's kinda no real restrictions for
> >    putting drm_dev_enter/exit within a lock critical section. Might still
> >    be good to explicitly call out in docs where it's all ok:
> > 
> >    - interrupt handlers (I hope that's the case, otherwise I screwed up)
> >    - shrinkers/mmu_notifier callbacks
> 
> Is there a problem scooping these with drm_dev_enter/exit ? Might they
> hang indefinitely once a device is gone ?

Well your irq handler must be robust, since drm_dev_enter/exit only helps
so much.

The shrinker still needs to work somewhat so it can help free memory,
while the hotunplug code

> > - anything else that's not allowed in within drm_dev_enter/exit. I'm e.g.
> >    wondering whether we should disallow drm_modeset_lock() or maybe
> >    dma_resv_lock(), or whether that's too restrictive. It could help quite
> >    a bit in finding places where the drm_dev_enter/exit section is too
> >    wide.
> > 
> > - another one is the inverse, but I guess you have that already with
> >    putting a drm_dev_is_held() into mmio helpers and all that to make sure
> >    we really have them all caught.
> 
> Not sure what drm_dev_is_held means here ? Also what do you mean by MMIO
> helpers ? Maybe you meant the page fault helpers like ttm_bo_vm_fault ?

Checking that we're in the srcu read side critical section is what I meant
with drm_dev_is_held. Maybe drm_dev_is_entered is clearer, but also sounds
a bit silly.
-Daniel

> 
> Andrey
> 
> > 
> > Above is just examples, I think the more we nail down these rules one way
> > or the other, the better for consistency across drivers. And without
> > consistency everyone will be forced to write their own mmap helpers
> > instead of one in ttm, or scheduler cleanup helpers instead of one in
> > drm/scheduler.
> > 
> > > P.S Please respond on the question for you on the other thread at
> > > 'PATCH v5 15/27] drm/scheduler: Fix hang when sched_entity released'
> > > about suggestion by Christian of partial up-streaming of this code up to
> > > and before the patches dealing with scoping of drm_dev_enter/exit scoping.
> > 
> > Ok will try, I'm a bit burried unfortunately so thanks for reminder when I
> > miss something.
> > -Daniel
> > 
> > > 
> > > Andrey
> > > 
> > > 
> > > 
> > > 
> > > 
> > > 
> > > > > 
> > > > > Andrey
> > > > > 
> > > > > > 
> > > > > > Cheers, Daniel
> > > > > > 
> > > > > > > 
> > > > > > > Andrey
> > > > > > > 
> > > > > > > > 
> > > > > > > > It is unfortunately even more work until we've reached the goal, but I
> > > > > > > > think it's safest and most flexible approach overall.
> > > > > > > > 
> > > > > > > > Cheers, Daniel
> > > > > > > > 
> > > > > > > > > 
> > > > > > > > > Andrey
> > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Cheers, Daniel
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > > 
> > > > > > > > > > > > > 
> > > > > > > > > > > > > I guess we should clarify this in the hotunplug doc?
> > > > > > > > > > > 
> > > > > > > > > > > Agree
> > > > > > > > > > > 
> > > > > > > > > > > > 
> > > > > > > > > > > > To clarify: I'm not against throwing an ENODEV at userspace for ioctl that
> > > > > > > > > > > > really make no sense, and where we're rather confident that all properly
> > > > > > > > > > > > implemented userspace will gracefully handle failures. Like a modeset, or
> > > > > > > > > > > > opening a device, or trying to import a dma-buf or stuff like that which
> > > > > > > > > > > > can already fail in normal operation for any kind of reason.
> > > > > > > > > > > > 
> > > > > > > > > > > > But stuff that never fails, like GETRESOURCES ioctl, really shouldn't fail
> > > > > > > > > > > > after hotunplug.
> > > > > > > > > > > 
> > > > > > > > > > > As I pointed above, this a bit confuses me given that we already do
> > > > > > > > > > > blanker rejection of IOCTLs if device is unplugged.
> > > > > > > > > > 
> > > > > > > > > > Well I'm confused about this too :-/
> > > > > > > > > > 
> > > > > > > > > > > > And then there's the middle ground, like doing a pageflip or buffer flush,
> > > > > > > > > > > > which I guess some userspace might handle, but risky to inflict those
> > > > > > > > > > > > consequences on them. atomic modeset is especially fun since depending
> > > > > > > > > > > > what you're doing it can be both "failures expected" and "failures not
> > > > > > > > > > > > really expected in normal operation".
> > > > > > > > > > > > 
> > > > > > > > > > > > Also, this really should be consistent across drivers, not solved with a
> > > > > > > > > > > > driver flag for every possible combination.
> > > > > > > > > > > > 
> > > > > > > > > > > > If you look at the current hotunplug kms drivers, they have
> > > > > > > > > > > > drm_dev_enter/exit sprinkled in specific hw callback functions because of
> > > > > > > > > > > > the above problems. But maybe it makes sense to change things in a few
> > > > > > > > > > > > cases. But then we should do it across the board.
> > > > > > > > > > > 
> > > > > > > > > > > So as I understand your preferred approach is that I scope any back_end, HW
> > > > > > > > > > > specific function with drm_dev_enter/exit because that where MMIO
> > > > > > > > > > > access takes place. But besides explicit MMIO access thorough
> > > > > > > > > > > register accessors in the HW back-end there is also indirect MMIO access
> > > > > > > > > > > taking place throughout the code in the driver because of various VRAM
> > > > > > > > > > > BOs which provide CPU access to VRAM through the VRAM BAR. This kind of
> > > > > > > > > > > access is spread all over in the driver and even in mid-layers such as
> > > > > > > > > > > TTM and not limited to HW back-end functions. It means it's much harder
> > > > > > > > > > > to spot such places to surgically scope them with drm_dev_enter/exit and
> > > > > > > > > > > also that any new such code introduced will immediately break hot unplug
> > > > > > > > > > > because the developers can't be expected to remember making their code
> > > > > > > > > > > robust to this specific use case. That why when we discussed internally
> > > > > > > > > > > what approach to take to protecting code with drm_dev_enter/exit we
> > > > > > > > > > > opted for using the widest available scope.
> > > > > > > > > > 
> > > > > > > > > > The thing is, you kinda have to anyway. There's enormous amounts of
> > > > > > > > > > asynchronous processing going on. E.g. nonblocking atomic commits also do
> > > > > > > > > > ttm unpinning and fun stuff like that, which if you sync things wrong can
> > > > > > > > > > happen way late. So the door for bad fallout is wide open :-(
> > > > > > > > > > 
> > > > > > > > > > I'm not sure where the right tradeoff is to make sure we catch them all,
> > > > > > > > > > and can make sure with testing that we've indeed caught them all.
> > > > > > > > > > -Daniel
> > > > > > > > > > 
> > > > > > > > 
> > > > > > 
> > > > 
> > 

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch