Re: [PATCH v5 00/24] AMDKFD Kernel Driver

[Oded Gabbay <oded.gabbay@amd.com>]

On 11/10/2014 04:34 AM, Dave Airlie wrote:
> On 9 November 2014 04:37, Oded Gabbay <oded.gabbay@amd.com> wrote:
>> Hi,
>> Here is the v5 patch set of amdkfd.
>>
>> This version is released several days ahead of the release of AMD's HSA Runtime
>> library as Open Source. Coupled with the modification that Thomas Stellard
>> did for the r600 LLVM back-end, AMD will be effectively releasing a _complete_
>> userspace Open Source stack/solution for running HSA applications using
>> kernels written in OpenCL C99 on top of amdkfd.
>>
>
> Daniel pointed out too me a major problem with the approach of having
> two drivers
> touching on hw at all, they are facing similiar issues with their virt stuff,
>
> How does amdkfd interfact with runtime pm on the radeon driver? I'd expect
> some calls to the runtime get/put functions in some places.
>
> Dave.
>
Hi Dave,
Per Jerome's request from the first time he saw the driver, we removed all the 
"register bashing" code from amdkfd and moved them to the interface file between 
amdkfd and radeon, which is part of the radeon driver.
See "[PATCH v5 05/24] drm/radeon: Add radeon <--> amdkfd interface" for the 
implementation of that interface file.

amdkfd communicates with the H/W either through that file, or through packets 
submitted via the HIQ (a single kernel queue that is created/destroyed and 
exclusively used by amdkfd). By using the HIQ to submit the runlist to the GPU, 
we actually don't call to most of the interface file's functions. Those 
functions are mostly used in bring-up of new H/W with the scheduling mode of: 
"No CP scheduling"

Regarding power management, we have two hooks from the radeon driver, that call 
our driver (if it exists. if it doesn't exists, those call simply return without 
doing anything). I think this is better approach than amdkfd interface directly 
with the runtime pm, in order to prevent races between us and radeon. Do you agree ?

That code is also in Patch 5/24, at cik.c:

@@ -8471,6 +8476,10 @@ static int cik_startup(struct radeon_device *rdev)
  	if (r)
  		return r;

+	r = radeon_kfd_resume(rdev);
+	if (r)
+		return r;
+
  	return 0;
  }

@@ -8519,6 +8528,7 @@ int cik_resume(struct radeon_device *rdev)
   */
  int cik_suspend(struct radeon_device *rdev)
  {
+	radeon_kfd_suspend(rdev);
  	radeon_pm_suspend(rdev);
  	dce6_audio_fini(rdev);
  	radeon_vm_manager_fini(rdev);

- Oded