From: "Zhou, Peng Ju" <PengJu.Zhou@amd.com>
To: "Kuehling, Felix" <Felix.Kuehling@amd.com>,
"amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>
Subject: RE: [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG
Date: Thu, 15 Apr 2021 07:25:38 +0000 [thread overview]
Message-ID: <DM6PR12MB4371D57D363B2E70042D62F2F84D9@DM6PR12MB4371.namprd12.prod.outlook.com> (raw)
In-Reply-To: <DM6PR12MB4371E4C410C5940878820261F8739@DM6PR12MB4371.namprd12.prod.outlook.com>
[AMD Official Use Only - Internal Distribution Only]
Hi Felix
Thanks for your proposal about " Given the number of call-sites being modified in this patch series"
We discussed internally, and have following concern:
1. expose our ranges to opensource.
2. lost the orthogonality in our software stack.
So we want to keep our original workaround.
Do you agree?
----------------------------------------------------------------------
BW
Pengju Zhou
-----Original Message-----
From: Zhou, Peng Ju
Sent: Friday, April 9, 2021 11:36 AM
To: Kuehling, Felix <Felix.Kuehling@amd.com>; amd-gfx@lists.freedesktop.org
Subject: RE: [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG
[AMD Official Use Only - Internal Distribution Only]
Hi Felix
That is a great idea, I will try it.
----------------------------------------------------------------------
BW
Pengju Zhou
-----Original Message-----
From: Kuehling, Felix <Felix.Kuehling@amd.com>
Sent: Thursday, April 8, 2021 10:58 PM
To: Zhou, Peng Ju <PengJu.Zhou@amd.com>; amd-gfx@lists.freedesktop.org
Subject: Re: [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG
Given the number of call-sites being modified in this patch series, would it be easier (and more maintainable) to change the behaviour or the regular register macros and add NO_RLC versions for the exceptions, similar to NO_KIQ?
Regards,
Felix
Am 2021-04-08 um 6:21 a.m. schrieb Peng Ju Zhou:
> From: pengzhou <PengJu.Zhou@amd.com>
>
> In SRIOV environment, KMD should access MMHUB registers with RLCG if
> MMHUB indirect access bit enabled.
>
> Change MMHUB register access from MMIO to RLCG.
>
> Signed-off-by: pengzhou <PengJu.Zhou@amd.com>
> ---
> drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c | 12 ++++++--
> drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c | 39 +++++++++++++------------
> 2 files changed, 29 insertions(+), 22 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> index 2bfd620576f2..42818c40d08c 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
> @@ -38,6 +38,7 @@
> #include "soc15.h"
> #include "soc15d.h"
> #include "soc15_common.h"
> +#include "gc/gc_10_1_0_offset.h"
>
> #include "nbio_v2_3.h"
>
> @@ -253,7 +254,10 @@ static void gmc_v10_0_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid,
> DRM_ERROR("Timeout waiting for sem acquire in VM flush!\n");
> }
>
> - WREG32_NO_KIQ(hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req);
> + if (vmhub == AMDGPU_MMHUB_0)
> + WREG32_RLC_NO_KIQ((hub->vm_inv_eng0_req + eng), inv_req);
> + else
> + WREG32_NO_KIQ(hub->vm_inv_eng0_req + eng, inv_req);
>
> /*
> * Issue a dummy read to wait for the ACK register to be cleared @@
> -280,8 +284,10 @@ static void gmc_v10_0_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid,
> * add semaphore release after invalidation,
> * write with 0 means semaphore release
> */
> - WREG32_NO_KIQ(hub->vm_inv_eng0_sem +
> - hub->eng_distance * eng, 0);
> + if (vmhub == AMDGPU_MMHUB_0)
> + WREG32_RLC_NO_KIQ(hub->vm_inv_eng0_sem + eng, 0);
> + else
> + WREG32_NO_KIQ(hub->vm_inv_eng0_sem + eng, 0);
>
> spin_unlock(&adev->gmc.invalidate_lock);
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> index da7edd1ed6b2..e8ecdf383192 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
> @@ -29,6 +29,7 @@
> #include "mmhub/mmhub_2_0_0_default.h"
> #include "navi10_enum.h"
>
> +#include "gc/gc_10_1_0_offset.h"
> #include "soc15_common.h"
>
> #define mmMM_ATC_L2_MISC_CG_Sienna_Cichlid 0x064d
> @@ -165,11 +166,11 @@ static void mmhub_v2_0_setup_vm_pt_regs(struct
> amdgpu_device *adev, uint32_t vmi {
> struct amdgpu_vmhub *hub = &adev->vmhub[AMDGPU_MMHUB_0];
>
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32,
> hub->ctx_addr_distance * vmid,
> lower_32_bits(page_table_base));
>
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32,
> hub->ctx_addr_distance * vmid,
> upper_32_bits(page_table_base)); } @@ -180,14 +181,14 @@
> static void mmhub_v2_0_init_gart_aperture_regs(struct amdgpu_device
> *adev)
>
> mmhub_v2_0_setup_vm_pt_regs(adev, 0, pt_base);
>
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
> + WREG32_SOC15_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
> (u32)(adev->gmc.gart_start >> 12));
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
> + WREG32_SOC15_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
> (u32)(adev->gmc.gart_start >> 44));
>
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
> (u32)(adev->gmc.gart_end >> 12));
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
> (u32)(adev->gmc.gart_end >> 44)); }
>
> @@ -197,9 +198,9 @@ static void mmhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
> uint32_t tmp;
>
> /* Program the AGP BAR */
> - WREG32_SOC15(MMHUB, 0, mmMMMC_VM_AGP_BASE, 0);
> - WREG32_SOC15(MMHUB, 0, mmMMMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
> - WREG32_SOC15(MMHUB, 0, mmMMMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMMC_VM_AGP_BASE, 0);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMMC_VM_AGP_TOP, adev->gmc.agp_end >>
> +24);
>
> if (!amdgpu_sriov_vf(adev)) {
> /* Program the system aperture low logical page number. */ @@
> -304,12 +305,12 @@ static void mmhub_v2_0_enable_system_domain(struct
> amdgpu_device *adev) {
> uint32_t tmp;
>
> - tmp = RREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_CNTL);
> + tmp = RREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_CNTL);
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT0_CNTL,
> RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
> - WREG32_SOC15(MMHUB, 0, mmMMVM_CONTEXT0_CNTL, tmp);
> + WREG32_SOC15_RLC(MMHUB, 0, mmMMVM_CONTEXT0_CNTL, tmp);
> }
>
> static void mmhub_v2_0_disable_identity_aperture(struct amdgpu_device
> *adev) @@ -371,16 +372,16 @@ static void mmhub_v2_0_setup_vmid_config(struct amdgpu_device *adev)
> tmp = REG_SET_FIELD(tmp, MMVM_CONTEXT1_CNTL,
> RETRY_PERMISSION_OR_INVALID_PAGE_FAULT,
> !adev->gmc.noretry);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_CNTL,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0, mmMMVM_CONTEXT1_CNTL,
> i * hub->ctx_distance, tmp);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_LO32,
> i * hub->ctx_addr_distance, 0);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_START_ADDR_HI32,
> i * hub->ctx_addr_distance, 0);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_LO32,
> i * hub->ctx_addr_distance,
> lower_32_bits(adev->vm_manager.max_pfn - 1));
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_CONTEXT1_PAGE_TABLE_END_ADDR_HI32,
> i * hub->ctx_addr_distance,
> upper_32_bits(adev->vm_manager.max_pfn - 1));
> }
> @@ -392,9 +393,9 @@ static void mmhub_v2_0_program_invalidation(struct amdgpu_device *adev)
> unsigned i;
>
> for (i = 0; i < 18; ++i) {
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_LO32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_LO32,
> i * hub->eng_addr_distance, 0xffffffff);
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_HI32,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0,
> +mmMMVM_INVALIDATE_ENG0_ADDR_RANGE_HI32,
> i * hub->eng_addr_distance, 0x1f);
> }
> }
> @@ -423,7 +424,7 @@ static void mmhub_v2_0_gart_disable(struct
> amdgpu_device *adev)
>
> /* Disable all tables */
> for (i = 0; i < AMDGPU_NUM_VMID; i++)
> - WREG32_SOC15_OFFSET(MMHUB, 0, mmMMVM_CONTEXT0_CNTL,
> + WREG32_SOC15_OFFSET_RLC(MMHUB, 0, mmMMVM_CONTEXT0_CNTL,
> i * hub->ctx_distance, 0);
>
> /* Setup TLB control */
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-04-15 7:25 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-08 10:21 [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG Peng Ju Zhou
2021-04-08 10:21 ` [PATCH 2/8] drm/amdgpu: Change GC(KFD/GFX) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 3/8] drm/amdgpu: Change GC(SDMA) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 4/8] drm/amdgpu: Change GC(GFX/GFXHUB) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 5/8] drm/amdgpu: Change GC(GMC/GFXHUB/..) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 6/8] drm/amdgpu: Change GC(GFXHUB) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 7/8] drm/amdgpu: Change GC(GFX) " Peng Ju Zhou
2021-04-08 10:22 ` [PATCH 8/8] drm/amdgpu: Use PSP to program IH_RB_CNTL* registers Peng Ju Zhou
2021-04-08 14:58 ` [PATCH 1/8] drm/amdgpu: change MMHUB register access from MMIO to RLCG Felix Kuehling
2021-04-09 3:35 ` Zhou, Peng Ju
2021-04-15 7:25 ` Zhou, Peng Ju [this message]
2021-04-15 15:47 ` Felix Kuehling
2021-04-16 3:18 ` Zhou, Peng Ju
-- strict thread matches above, loose matches on Subject: below --
2021-04-08 5:32 Peng Ju Zhou
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DM6PR12MB4371D57D363B2E70042D62F2F84D9@DM6PR12MB4371.namprd12.prod.outlook.com \
--to=pengju.zhou@amd.com \
--cc=Felix.Kuehling@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).