[REGRESSION] rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"

[REGRESSION] rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"

[Alexey Klimov]

#regzbot introduced: 1cfb4d612127
#regzbot title: rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"

Hi all,

I've been playing with RX7600 and it was observed that amdgpu stopped working between kernel 6.2 and 6.5.
Then I narrowed it down to 6.4 <-> 6.5-rc1 and finally bisect pointed at 1cfb4d6121276a829aa94d0e32a7f5e1830ebc21
And I manually checked if it boots/works on the previous commit and the mentioned one.

I guess the log also reveals warning in error path. Please see below.

I didn't check any further. This is simple debian testing system with the following cmdline options:
root@avadebian:~# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-6.6-rc7+ ignore_loglevel root=/dev/nvme1n1p2 ro nr_cpus=32

So far simple revert (patch is below) returns things back to normal-ish: there are huge graphics artifacts on Xorg/X11 under 6.1 to upstream kernel. Wayland-based sway works great without issues. Not sure where should I report this.

Please let me know if I can help debugging, testing or provide some other logs regarding 1cfb4d612127? Any cmdline options to collect more info?

Thanks,
Alexey



From 214372d5cedcf8757dd80d5f4d058377a3d92c52 Mon Sep 17 00:00:00 2001
From: Alexey Klimov <alexey.klimov@linaro.org>
Date: Thu, 26 Oct 2023 17:01:02 +0100
Subject: [PATCH] drm/amdgpu: Revert "drm/amdgpu: put MQDs in VRAM"

This reverts commit 1cfb4d6121276a829aa94d0e32a7f5e1830ebc21.

amdgpu driver fails during initialisation with RX7600/gfx11 on
ADLINK Ampere Altra Developer Platform (AVA developer platform)
with mentioned commit:

[   12.559893] [drm] Display Core v3.2.247 initialized on DCN 3.2.1
[   12.565906] [drm] DP-HDMI FRL PCON supported
[   12.572192] [drm] DMUB hardware initialized: version=0x07000C00
[   12.582541] snd_hda_intel 000d:03:00.1: bound 000d:03:00.0 (ops amdgpu_dm_audio_component_bind_ops [amdgpu])
[   12.625357] [drm] kiq ring mec 3 pipe 1 q 0
[   12.857087] amdgpu 000d:03:00.0: [drm:amdgpu_ring_test_helper [amdgpu]] *ERROR* ring comp_1.0.0 test failed (-110)
[   12.867930] [drm:amdgpu_device_init [amdgpu]] *ERROR* hw_init of IP block <gfx_v11_0> failed -110
[   12.877289] amdgpu 000d:03:00.0: amdgpu: amdgpu_device_ip_init failed
[   12.883723] amdgpu 000d:03:00.0: amdgpu: Fatal error during GPU init
[   12.890070] amdgpu 000d:03:00.0: amdgpu: amdgpu: finishing device.
[   12.896586] [drm] DSC precompute is not needed.
[   12.901142] ------------[ cut here ]------------
[   12.905747] WARNING: CPU: 0 PID: 212 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   12.916841] Modules linked in: hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) amdgpu(E+) nf_nat(E) nf_conntrack(E) snd_hda_codec_hdmi(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) iptable_filter(E) amdxcp(E) drm_exec(E) gpu_sched(E) snd_hda_intel(E) aes_ce_blk(E) snd_intel_dspcfg(E) drm_buddy(E) aes_ce_cipher(E) snd_hda_codec(E) xhci_pci(E) video(E) crct10dif_ce(E) polyval_ce(E) snd_hda_core(E) xhci_hcd(E) drm_suballoc_helper(E) snd_hwdep(E) polyval_generic(E) drm_ttm_helper(E) snd_pcm(E) ghash_ce(E) ast(E) ttm(E) gf128mul(E) snd_timer(E) ipmi_ssif(E) drm_display_helper(E) drm_shmem_helper(E) sha2_ce(E) sha256_arm64(E) ipmi_devintf(E) usbcore(E) snd(E) drm_kms_helper(E) igb(E) sha1_ce(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) soundcore(E) usb_common(E) i2c_algo_bit(E) cppc_cpufreq(E) i2c_designware_platform(E) arm_dsu_pmu(E) arm_cmn(E) xgene_hwmon(E) i2c_designware_core(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E) efivarfs(E)
[   12.916916]  ip_tables(E) x_tables(E) autofs4(E)
[   13.011111] CPU: 0 PID: 212 Comm: kworker/0:2 Tainted: G            E      6.6.0-rc7+ #23
[   13.019277] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   13.033084] Workqueue: events work_for_cpu_fn
[   13.037434] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   13.044384] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.049652] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   13.056220] sp : ffff80008012bc10
[   13.059522] x29: ffff80008012bc20 x28: 0000000000000000 x27: 0000000000000000
[   13.066647] x26: 0000000000000000 x25: ffff07ff98580010 x24: ffff07ff98580000
[   13.073772] x23: ffff07ff985a78f0 x22: ffff07ff98580010 x21: ffff07ff985904c8
[   13.080896] x20: ffff07ff985900e8 x19: ffff07ff98598580 x18: 0000000000000006
[   13.088020] x17: 0000000000000020 x16: ffffbb510d0d7140 x15: fffffffffffffefb
[   13.095145] x14: 0000000000000000 x13: 2e64656465656e20 x12: ffff07ff8c7fd9e0
[   13.102268] x11: 00000000000003e8 x10: ffff07ff8c7fd9e0 x9 : ffffbb50ac3345e0
[   13.109392] x8 : ffffbb50abf18000 x7 : 0000000000000000 x6 : 000000007a456104
[   13.116516] x5 : 0000000000000000 x4 : ffff07ff98580000 x3 : 0000000000000000
[   13.123641] x2 : 0000000000000000 x1 : ffff07ff985a78f0 x0 : ffff07ffc5fd4000
[   13.130765] Call trace:
[   13.133200]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.138121]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   13.143732]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   13.149517]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   13.155301]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   13.160564]  local_pci_probe+0x48/0xb8
[   13.164305]  work_for_cpu_fn+0x24/0x40
[   13.168043]  process_one_work+0x170/0x3d0
[   13.172042]  worker_thread+0x2bc/0x3e0
[   13.175781]  kthread+0x118/0x128
[   13.178999]  ret_from_fork+0x10/0x20
[   13.182564] ---[ end trace 0000000000000000 ]---
...
[   16.984679] amdgpu: probe of 000d:03:00.0 failed with error -110

Cc: Luben Tuikov <luben.tuikov@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Fixes: 1cfb4d612127 drm/amdgpu: put MQDs in VRAM
Signed-off-by: Alexey Klimov <alexey.klimov@linaro.org>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c | 9 ++-------
 drivers/gpu/drm/amd/amdgpu/mes_v10_1.c  | 1 -
 drivers/gpu/drm/amd/amdgpu/mes_v11_0.c  | 1 -
 3 files changed, 2 insertions(+), 9 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
index 2382921710ec..1f2d8be0fc44 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
@@ -382,11 +382,6 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device *adev,
 	int r, i, j;
 	struct amdgpu_kiq *kiq = &adev->gfx.kiq[xcc_id];
 	struct amdgpu_ring *ring = &kiq->ring;
-	u32 domain = AMDGPU_GEM_DOMAIN_GTT;
-
-	/* Only enable on gfx10 and 11 for now to avoid changing behavior on older chips */
-	if (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(10, 0, 0))
-		domain |= AMDGPU_GEM_DOMAIN_VRAM;
 
 	/* create MQD for KIQ */
 	if (!adev->enable_mes_kiq && !ring->mqd_obj) {
@@ -421,7 +416,7 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device *adev,
 			ring = &adev->gfx.gfx_ring[i];
 			if (!ring->mqd_obj) {
 				r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
-							    domain, &ring->mqd_obj,
+							    AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
 							    &ring->mqd_gpu_addr, &ring->mqd_ptr);
 				if (r) {
 					dev_warn(adev->dev, "failed to create ring mqd bo (%d)", r);
@@ -445,7 +440,7 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device *adev,
 		ring = &adev->gfx.compute_ring[j];
 		if (!ring->mqd_obj) {
 			r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
-						    domain, &ring->mqd_obj,
+						    AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
 						    &ring->mqd_gpu_addr, &ring->mqd_ptr);
 			if (r) {
 				dev_warn(adev->dev, "failed to create ring mqd bo (%d)", r);
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
index eb06d749876f..080e7eb3f98d 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
@@ -898,7 +898,6 @@ static int mes_v10_1_mqd_sw_init(struct amdgpu_device *adev,
 		return 0;
 
 	r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
-				    AMDGPU_GEM_DOMAIN_VRAM |
 				    AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
 				    &ring->mqd_gpu_addr, &ring->mqd_ptr);
 	if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
index 6827d547042e..0608710306b8 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
@@ -1004,7 +1004,6 @@ static int mes_v11_0_mqd_sw_init(struct amdgpu_device *adev,
 		return 0;
 
 	r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
-				    AMDGPU_GEM_DOMAIN_VRAM |
 				    AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
 				    &ring->mqd_gpu_addr, &ring->mqd_ptr);
 	if (r) {
-- 
2.42.0

Re: [REGRESSION] rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"

[Alex Deucher]

[-- Attachment #1: Type: text/plain, Size: 10959 bytes --]

On Thu, Oct 26, 2023 at 1:33 PM Alexey Klimov <alexey.klimov@linaro.org> wrote:
>
> #regzbot introduced: 1cfb4d612127
> #regzbot title: rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"
>
> Hi all,
>
> I've been playing with RX7600 and it was observed that amdgpu stopped working between kernel 6.2 and 6.5.
> Then I narrowed it down to 6.4 <-> 6.5-rc1 and finally bisect pointed at 1cfb4d6121276a829aa94d0e32a7f5e1830ebc21
> And I manually checked if it boots/works on the previous commit and the mentioned one.
>
> I guess the log also reveals warning in error path. Please see below.
>
> I didn't check any further. This is simple debian testing system with the following cmdline options:
> root@avadebian:~# cat /proc/cmdline
> BOOT_IMAGE=/boot/vmlinuz-6.6-rc7+ ignore_loglevel root=/dev/nvme1n1p2 ro nr_cpus=32
>
> So far simple revert (patch is below) returns things back to normal-ish: there are huge graphics artifacts on Xorg/X11 under 6.1 to upstream kernel. Wayland-based sway works great without issues. Not sure where should I report this.
>
> Please let me know if I can help debugging, testing or provide some other logs regarding 1cfb4d612127? Any cmdline options to collect more info?

Please make sure you have this patch as well:
e602157ec089240861cd641ee2c7c64eeaec09bf ("drm/amdgpu: fix S3 issue if
MQD in VRAM")
Please open a ticket here so we can track this:
https://gitlab.freedesktop.org/drm/amd/-/issues/
I think I see the problem.  Please see if attached patch 1 fixes the
issue.  If this fixes it, that would also explain the issues you are
seeing with Xorg.  It would appear there are limitations around MMIO
access on your platform and unfortunately most graphics APIs require
unaligned access to MMIO space with the CPU.  We can fix the kernel
side pretty easily, but userspace will be a problem.

More below.

>
> Thanks,
> Alexey
>
>
>
> From 214372d5cedcf8757dd80d5f4d058377a3d92c52 Mon Sep 17 00:00:00 2001
> From: Alexey Klimov <alexey.klimov@linaro.org>
> Date: Thu, 26 Oct 2023 17:01:02 +0100
> Subject: [PATCH] drm/amdgpu: Revert "drm/amdgpu: put MQDs in VRAM"
>
> This reverts commit 1cfb4d6121276a829aa94d0e32a7f5e1830ebc21.
>
> amdgpu driver fails during initialisation with RX7600/gfx11 on
> ADLINK Ampere Altra Developer Platform (AVA developer platform)
> with mentioned commit:
>
> [   12.559893] [drm] Display Core v3.2.247 initialized on DCN 3.2.1
> [   12.565906] [drm] DP-HDMI FRL PCON supported
> [   12.572192] [drm] DMUB hardware initialized: version=0x07000C00
> [   12.582541] snd_hda_intel 000d:03:00.1: bound 000d:03:00.0 (ops amdgpu_dm_audio_component_bind_ops [amdgpu])
> [   12.625357] [drm] kiq ring mec 3 pipe 1 q 0
> [   12.857087] amdgpu 000d:03:00.0: [drm:amdgpu_ring_test_helper [amdgpu]] *ERROR* ring comp_1.0.0 test failed (-110)
> [   12.867930] [drm:amdgpu_device_init [amdgpu]] *ERROR* hw_init of IP block <gfx_v11_0> failed -110
> [   12.877289] amdgpu 000d:03:00.0: amdgpu: amdgpu_device_ip_init failed
> [   12.883723] amdgpu 000d:03:00.0: amdgpu: Fatal error during GPU init
> [   12.890070] amdgpu 000d:03:00.0: amdgpu: amdgpu: finishing device.
> [   12.896586] [drm] DSC precompute is not needed.
> [   12.901142] ------------[ cut here ]------------
> [   12.905747] WARNING: CPU: 0 PID: 212 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
> [   12.916841] Modules linked in: hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) amdgpu(E+) nf_nat(E) nf_conntrack(E) snd_hda_codec_hdmi(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) iptable_filter(E) amdxcp(E) drm_exec(E) gpu_sched(E) snd_hda_intel(E) aes_ce_blk(E) snd_intel_dspcfg(E) drm_buddy(E) aes_ce_cipher(E) snd_hda_codec(E) xhci_pci(E) video(E) crct10dif_ce(E) polyval_ce(E) snd_hda_core(E) xhci_hcd(E) drm_suballoc_helper(E) snd_hwdep(E) polyval_generic(E) drm_ttm_helper(E) snd_pcm(E) ghash_ce(E) ast(E) ttm(E) gf128mul(E) snd_timer(E) ipmi_ssif(E) drm_display_helper(E) drm_shmem_helper(E) sha2_ce(E) sha256_arm64(E) ipmi_devintf(E) usbcore(E) snd(E) drm_kms_helper(E) igb(E) sha1_ce(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) soundcore(E) usb_common(E) i2c_algo_bit(E) cppc_cpufreq(E) i2c_designware_platform(E) arm_dsu_pmu(E) arm_cmn(E) xgene_hwmon(E) i2c_designware_core(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E) efivarfs(E)
> [   12.916916]  ip_tables(E) x_tables(E) autofs4(E)
> [   13.011111] CPU: 0 PID: 212 Comm: kworker/0:2 Tainted: G            E      6.6.0-rc7+ #23
> [   13.019277] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
> [   13.033084] Workqueue: events work_for_cpu_fn
> [   13.037434] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [   13.044384] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
> [   13.049652] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
> [   13.056220] sp : ffff80008012bc10
> [   13.059522] x29: ffff80008012bc20 x28: 0000000000000000 x27: 0000000000000000
> [   13.066647] x26: 0000000000000000 x25: ffff07ff98580010 x24: ffff07ff98580000
> [   13.073772] x23: ffff07ff985a78f0 x22: ffff07ff98580010 x21: ffff07ff985904c8
> [   13.080896] x20: ffff07ff985900e8 x19: ffff07ff98598580 x18: 0000000000000006
> [   13.088020] x17: 0000000000000020 x16: ffffbb510d0d7140 x15: fffffffffffffefb
> [   13.095145] x14: 0000000000000000 x13: 2e64656465656e20 x12: ffff07ff8c7fd9e0
> [   13.102268] x11: 00000000000003e8 x10: ffff07ff8c7fd9e0 x9 : ffffbb50ac3345e0
> [   13.109392] x8 : ffffbb50abf18000 x7 : 0000000000000000 x6 : 000000007a456104
> [   13.116516] x5 : 0000000000000000 x4 : ffff07ff98580000 x3 : 0000000000000000
> [   13.123641] x2 : 0000000000000000 x1 : ffff07ff985a78f0 x0 : ffff07ffc5fd4000
> [   13.130765] Call trace:
> [   13.133200]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
> [   13.138121]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
> [   13.143732]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
> [   13.149517]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
> [   13.155301]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
> [   13.160564]  local_pci_probe+0x48/0xb8
> [   13.164305]  work_for_cpu_fn+0x24/0x40
> [   13.168043]  process_one_work+0x170/0x3d0
> [   13.172042]  worker_thread+0x2bc/0x3e0
> [   13.175781]  kthread+0x118/0x128
> [   13.178999]  ret_from_fork+0x10/0x20
> [   13.182564] ---[ end trace 0000000000000000 ]---
> ...
> [   16.984679] amdgpu: probe of 000d:03:00.0 failed with error -110
>
> Cc: Luben Tuikov <luben.tuikov@amd.com>
> Cc: Alex Deucher <alexander.deucher@amd.com>
> Fixes: 1cfb4d612127 drm/amdgpu: put MQDs in VRAM
> Signed-off-by: Alexey Klimov <alexey.klimov@linaro.org>
> ---
>  drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c | 9 ++-------
>  drivers/gpu/drm/amd/amdgpu/mes_v10_1.c  | 1 -
>  drivers/gpu/drm/amd/amdgpu/mes_v11_0.c  | 1 -
>  3 files changed, 2 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
> index 2382921710ec..1f2d8be0fc44 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
> @@ -382,11 +382,6 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device *adev,
>         int r, i, j;
>         struct amdgpu_kiq *kiq = &adev->gfx.kiq[xcc_id];
>         struct amdgpu_ring *ring = &kiq->ring;
> -       u32 domain = AMDGPU_GEM_DOMAIN_GTT;
> -
> -       /* Only enable on gfx10 and 11 for now to avoid changing behavior on older chips */
> -       if (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(10, 0, 0))
> -               domain |= AMDGPU_GEM_DOMAIN_VRAM;

Just removing the addition of the AMDGPU_GEM_DOMAIN_VRAM domain here
will revert the behavior.  Since this is an important optimization and
we aren't seeing any issues on x86, I'd prefer to just limit your arch
to GTT if we can't resolve it some other way.

Try patch 1 and if that doesn't work we can fall back to some variant
of patch 2.

Alex

>
>         /* create MQD for KIQ */
>         if (!adev->enable_mes_kiq && !ring->mqd_obj) {
> @@ -421,7 +416,7 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device *adev,
>                         ring = &adev->gfx.gfx_ring[i];
>                         if (!ring->mqd_obj) {
>                                 r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
> -                                                           domain, &ring->mqd_obj,
> +                                                           AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
>                                                             &ring->mqd_gpu_addr, &ring->mqd_ptr);
>                                 if (r) {
>                                         dev_warn(adev->dev, "failed to create ring mqd bo (%d)", r);
> @@ -445,7 +440,7 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device *adev,
>                 ring = &adev->gfx.compute_ring[j];
>                 if (!ring->mqd_obj) {
>                         r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
> -                                                   domain, &ring->mqd_obj,
> +                                                   AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
>                                                     &ring->mqd_gpu_addr, &ring->mqd_ptr);
>                         if (r) {
>                                 dev_warn(adev->dev, "failed to create ring mqd bo (%d)", r);
> diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
> index eb06d749876f..080e7eb3f98d 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
> @@ -898,7 +898,6 @@ static int mes_v10_1_mqd_sw_init(struct amdgpu_device *adev,
>                 return 0;
>
>         r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
> -                                   AMDGPU_GEM_DOMAIN_VRAM |
>                                     AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
>                                     &ring->mqd_gpu_addr, &ring->mqd_ptr);
>         if (r) {
> diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> index 6827d547042e..0608710306b8 100644
> --- a/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> +++ b/drivers/gpu/drm/amd/amdgpu/mes_v11_0.c
> @@ -1004,7 +1004,6 @@ static int mes_v11_0_mqd_sw_init(struct amdgpu_device *adev,
>                 return 0;
>
>         r = amdgpu_bo_create_kernel(adev, mqd_size, PAGE_SIZE,
> -                                   AMDGPU_GEM_DOMAIN_VRAM |
>                                     AMDGPU_GEM_DOMAIN_GTT, &ring->mqd_obj,
>                                     &ring->mqd_gpu_addr, &ring->mqd_ptr);
>         if (r) {
> --
> 2.42.0
>

[-- Attachment #2: 0001-drm-amdgpu-gfx10-11-use-memcpy_to-fromio-for-MQDs.patch --]
[-- Type: text/x-patch, Size: 4970 bytes --]

From 53219f8cbcf50521850c542163132678b2114265 Mon Sep 17 00:00:00 2001
From: Alex Deucher <alexander.deucher@amd.com>
Date: Thu, 26 Oct 2023 14:47:57 -0400
Subject: [PATCH 1/2] drm/amdgpu/gfx10,11: use memcpy_to/fromio for MQDs

Since they were moved to VRAM, we need to use the IO
variants of memcpy.

Fixes: 1cfb4d612127 ("drm/amdgpu: put MQDs in VRAM")
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c | 12 ++++++------
 drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c | 12 ++++++------
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
index 9032d7a24d7c..306252cd67fd 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
@@ -6457,11 +6457,11 @@ static int gfx_v10_0_gfx_init_queue(struct amdgpu_ring *ring)
 		nv_grbm_select(adev, 0, 0, 0, 0);
 		mutex_unlock(&adev->srbm_mutex);
 		if (adev->gfx.me.mqd_backup[mqd_idx])
-			memcpy(adev->gfx.me.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
+			memcpy_fromio(adev->gfx.me.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
 	} else {
 		/* restore mqd with the backup copy */
 		if (adev->gfx.me.mqd_backup[mqd_idx])
-			memcpy(mqd, adev->gfx.me.mqd_backup[mqd_idx], sizeof(*mqd));
+			memcpy_toio(mqd, adev->gfx.me.mqd_backup[mqd_idx], sizeof(*mqd));
 		/* reset the ring */
 		ring->wptr = 0;
 		*ring->wptr_cpu_addr = 0;
@@ -6735,7 +6735,7 @@ static int gfx_v10_0_kiq_init_queue(struct amdgpu_ring *ring)
 	if (amdgpu_in_reset(adev)) { /* for GPU_RESET case */
 		/* reset MQD to a clean status */
 		if (adev->gfx.kiq[0].mqd_backup)
-			memcpy(mqd, adev->gfx.kiq[0].mqd_backup, sizeof(*mqd));
+			memcpy_toio(mqd, adev->gfx.kiq[0].mqd_backup, sizeof(*mqd));
 
 		/* reset ring buffer */
 		ring->wptr = 0;
@@ -6758,7 +6758,7 @@ static int gfx_v10_0_kiq_init_queue(struct amdgpu_ring *ring)
 		mutex_unlock(&adev->srbm_mutex);
 
 		if (adev->gfx.kiq[0].mqd_backup)
-			memcpy(adev->gfx.kiq[0].mqd_backup, mqd, sizeof(*mqd));
+			memcpy_fromio(adev->gfx.kiq[0].mqd_backup, mqd, sizeof(*mqd));
 	}
 
 	return 0;
@@ -6779,11 +6779,11 @@ static int gfx_v10_0_kcq_init_queue(struct amdgpu_ring *ring)
 		mutex_unlock(&adev->srbm_mutex);
 
 		if (adev->gfx.mec.mqd_backup[mqd_idx])
-			memcpy(adev->gfx.mec.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
+			memcpy_fromio(adev->gfx.mec.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
 	} else {
 		/* restore MQD to a clean status */
 		if (adev->gfx.mec.mqd_backup[mqd_idx])
-			memcpy(mqd, adev->gfx.mec.mqd_backup[mqd_idx], sizeof(*mqd));
+			memcpy_toio(mqd, adev->gfx.mec.mqd_backup[mqd_idx], sizeof(*mqd));
 		/* reset ring buffer */
 		ring->wptr = 0;
 		atomic64_set((atomic64_t *)ring->wptr_cpu_addr, 0);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
index 762d7a19f1be..43d066bc5245 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
@@ -3684,11 +3684,11 @@ static int gfx_v11_0_gfx_init_queue(struct amdgpu_ring *ring)
 		soc21_grbm_select(adev, 0, 0, 0, 0);
 		mutex_unlock(&adev->srbm_mutex);
 		if (adev->gfx.me.mqd_backup[mqd_idx])
-			memcpy(adev->gfx.me.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
+			memcpy_fromio(adev->gfx.me.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
 	} else {
 		/* restore mqd with the backup copy */
 		if (adev->gfx.me.mqd_backup[mqd_idx])
-			memcpy(mqd, adev->gfx.me.mqd_backup[mqd_idx], sizeof(*mqd));
+			memcpy_toio(mqd, adev->gfx.me.mqd_backup[mqd_idx], sizeof(*mqd));
 		/* reset the ring */
 		ring->wptr = 0;
 		*ring->wptr_cpu_addr = 0;
@@ -3977,7 +3977,7 @@ static int gfx_v11_0_kiq_init_queue(struct amdgpu_ring *ring)
 	if (amdgpu_in_reset(adev)) { /* for GPU_RESET case */
 		/* reset MQD to a clean status */
 		if (adev->gfx.kiq[0].mqd_backup)
-			memcpy(mqd, adev->gfx.kiq[0].mqd_backup, sizeof(*mqd));
+			memcpy_toio(mqd, adev->gfx.kiq[0].mqd_backup, sizeof(*mqd));
 
 		/* reset ring buffer */
 		ring->wptr = 0;
@@ -4000,7 +4000,7 @@ static int gfx_v11_0_kiq_init_queue(struct amdgpu_ring *ring)
 		mutex_unlock(&adev->srbm_mutex);
 
 		if (adev->gfx.kiq[0].mqd_backup)
-			memcpy(adev->gfx.kiq[0].mqd_backup, mqd, sizeof(*mqd));
+			memcpy_fromio(adev->gfx.kiq[0].mqd_backup, mqd, sizeof(*mqd));
 	}
 
 	return 0;
@@ -4021,11 +4021,11 @@ static int gfx_v11_0_kcq_init_queue(struct amdgpu_ring *ring)
 		mutex_unlock(&adev->srbm_mutex);
 
 		if (adev->gfx.mec.mqd_backup[mqd_idx])
-			memcpy(adev->gfx.mec.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
+			memcpy_fromio(adev->gfx.mec.mqd_backup[mqd_idx], mqd, sizeof(*mqd));
 	} else {
 		/* restore MQD to a clean status */
 		if (adev->gfx.mec.mqd_backup[mqd_idx])
-			memcpy(mqd, adev->gfx.mec.mqd_backup[mqd_idx], sizeof(*mqd));
+			memcpy_toio(mqd, adev->gfx.mec.mqd_backup[mqd_idx], sizeof(*mqd));
 		/* reset ring buffer */
 		ring->wptr = 0;
 		atomic64_set((atomic64_t *)ring->wptr_cpu_addr, 0);
-- 
2.41.0


[-- Attachment #3: 0002-drm-amdgpu-don-t-put-MQDs-in-VRAM-on-ARM-ARM64.patch --]
[-- Type: text/x-patch, Size: 1176 bytes --]

From 5ae791c79de5cb055f381f9c27a1aec3b7f00162 Mon Sep 17 00:00:00 2001
From: Alex Deucher <alexander.deucher@amd.com>
Date: Thu, 26 Oct 2023 14:37:31 -0400
Subject: [PATCH 2/2] drm/amdgpu: don't put MQDs in VRAM on ARM | ARM64

Seems to be some coherency issue?

Fixes: 1cfb4d612127 ("drm/amdgpu: put MQDs in VRAM")
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
index 2382921710ec..ef4cb921781d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
@@ -384,9 +384,11 @@ int amdgpu_gfx_mqd_sw_init(struct amdgpu_device *adev,
 	struct amdgpu_ring *ring = &kiq->ring;
 	u32 domain = AMDGPU_GEM_DOMAIN_GTT;
 
+#if !defined(CONFIG_ARM) && !defined(CONFIG_ARM64)
 	/* Only enable on gfx10 and 11 for now to avoid changing behavior on older chips */
 	if (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(10, 0, 0))
 		domain |= AMDGPU_GEM_DOMAIN_VRAM;
+#endif
 
 	/* create MQD for KIQ */
 	if (!adev->enable_mes_kiq && !ring->mqd_obj) {
-- 
2.41.0

Re: [REGRESSION] rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"

[Alexey Klimov]

[-- Attachment #1: Type: text/plain, Size: 4966 bytes --]

Hi Alex,

On Thu, 26 Oct 2023 at 19:53, Alex Deucher <alexdeucher@gmail.com> wrote:
>
> On Thu, Oct 26, 2023 at 1:33 PM Alexey Klimov <alexey.klimov@linaro.org> wrote:
> >
> > #regzbot introduced: 1cfb4d612127
> > #regzbot title: rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"
> >
> > Hi all,
> >
> > I've been playing with RX7600 and it was observed that amdgpu stopped working between kernel 6.2 and 6.5.
> > Then I narrowed it down to 6.4 <-> 6.5-rc1 and finally bisect pointed at 1cfb4d6121276a829aa94d0e32a7f5e1830ebc21
> > And I manually checked if it boots/works on the previous commit and the mentioned one.
> >
> > I guess the log also reveals warning in error path. Please see below.
> >
> > I didn't check any further. This is simple debian testing system with the following cmdline options:
> > root@avadebian:~# cat /proc/cmdline
> > BOOT_IMAGE=/boot/vmlinuz-6.6-rc7+ ignore_loglevel root=/dev/nvme1n1p2 ro nr_cpus=32
> >
> > So far simple revert (patch is below) returns things back to normal-ish: there are huge graphics artifacts on Xorg/X11 under 6.1 to upstream kernel. Wayland-based sway works great without issues. Not sure where should I report this.
> >
> > Please let me know if I can help debugging, testing or provide some other logs regarding 1cfb4d612127? Any cmdline options to collect more info?
>
> Please make sure you have this patch as well:
> e602157ec089240861cd641ee2c7c64eeaec09bf ("drm/amdgpu: fix S3 issue if
> MQD in VRAM")
> Please open a ticket here so we can track this:
> https://gitlab.freedesktop.org/drm/amd/-/issues/

The patch was there during testing and I will open a ticket there.

> I think I see the problem.  Please see if attached patch 1 fixes the
> issue.  If this fixes it, that would also explain the issues you are
> seeing with Xorg.  It would appear there are limitations around MMIO
> access on your platform and unfortunately most graphics APIs require
> unaligned access to MMIO space with the CPU.  We can fix the kernel
> side pretty easily, but userspace will be a problem.

Does it mean that we don't have unaligned access to PCIe MMIO space on
this Adlink Ampere AVA arm64 platform?
Do you know if it is related to the thing that PCIe BARs are mapped as
a device memory and not a normal memory? (and they should be mapped as
normal memory)

[..]

> Just removing the addition of the AMDGPU_GEM_DOMAIN_VRAM domain here
> will revert the behavior.  Since this is an important optimization and
> we aren't seeing any issues on x86, I'd prefer to just limit your arch
> to GTT if we can't resolve it some other way.
>
> Try patch 1 and if that doesn't work we can fall back to some variant
> of patch 2.

The patch 1 alone doesn't fix the issue. Both patches 1 & 2 do work
and amdgpu initializes. Still issues with Xorg and wayland works okay.

Apart from that I observed "amdgpu: [gfxhub] page fault" one time:

[   12.432567] amdgpu 000d:03:00.0: [drm:jpeg_v4_0_hw_init [amdgpu]]
JPEG decode initialized successfully.
[   12.442516] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault
(src_id:0 ring:72 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.454080] amdgpu 000d:03:00.0: amdgpu:   in page starting at
address 0x00000000044b0000 from client 10
[   12.457317] usb 1-4.4: new high-speed USB device number 4 using xhci_hcd
[   12.463548] amdgpu 000d:03:00.0: amdgpu:
GCVM_L2_PROTECTION_FAULT_STATUS:0x00000890
[   12.463551] amdgpu 000d:03:00.0: amdgpu: Faulty UTCL2 client ID: CPF (0x4)
[   12.484914] amdgpu 000d:03:00.0: amdgpu: MORE_FAULTS: 0x0
[   12.490474] amdgpu 000d:03:00.0: amdgpu: WALKER_ERROR: 0x0
[   12.496121] amdgpu 000d:03:00.0: amdgpu: PERMISSION_FAULTS: 0x9
[   12.502202] amdgpu 000d:03:00.0: amdgpu: MAPPING_ERROR: 0x0
[   12.507934] amdgpu 000d:03:00.0: amdgpu: RW: 0x0
[   12.512716] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault
(src_id:0 ring:221 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.524355] amdgpu 000d:03:00.0: amdgpu:   in page starting at
address 0x00000000044b1000 from client 10
[   12.533821] amdgpu 000d:03:00.0: amdgpu:
GCVM_L2_PROTECTION_FAULT_STATUS:0x000009BB
[   12.541464] amdgpu 000d:03:00.0: amdgpu: Faulty UTCL2 client ID: CPF (0x4)
[   12.548499] amdgpu 000d:03:00.0: amdgpu: MORE_FAULTS: 0x1
[   12.554059] amdgpu 000d:03:00.0: amdgpu: WALKER_ERROR: 0x5
[   12.558700] [drm:mes_v11_0_submit_pkt_and_poll_completion.constprop.0
[amdgpu]] *ERROR* MES failed to response msg=0

I am not sure if it related to clean 6.6 kernel or to additional patches 1 & 2.
I did around 20 boots of clean 6.6-rc7 version and didn't observe this
page fault.
During 20 reboots of  6.6-rc7 + your patches 1 and 2 -- this page
fault was observed one time only.
Couldn't say how reproducible is this. The log is attached.

Let me know if you want me to test/Ack patch 2 if you are going to send it.

Thanks,
Alexey

[-- Attachment #2: dmesg_permission_fault.txt --]
[-- Type: text/plain, Size: 158229 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000100000 [0x413fd0c1]
[    0.000000] Linux version 6.6.0-rc7+ (reimers@avadebian) (gcc (Debian 13.2.0-5) 13.2.0, GNU ld (GNU Binutils for Debian) 2.41) #26 SMP PREEMPT Thu Oct 26 22:05:11 BST 2023
[    0.000000] KASLR enabled
[    0.000000] printk: debug: ignoring loglevel setting.
[    0.000000] efi: EFI v2.7 by EDK II
[    0.000000] efi: TPMFinalLog=0xfffc0000 ACPI 2.0=0xfffe0018 SMBIOS 3.0=0xf9f30000 MEMATTR=0xf8ecd018 ESRT=0xf8459698 TPMEventLog=0xf8436018 RNG=0xfffef898 MEMRESERVE=0xf844ab98 
[    0.000000] random: crng init done
[    0.000000] esrt: Reserving ESRT space from 0x00000000f8459698 to 0x00000000f84596d0.
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x00000000FFFE0018 000024 (v02 Ampere)
[    0.000000] ACPI: XSDT 0x00000000FFFEFE98 0000AC (v01 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: FACP 0x00000000FFFEFB98 000114 (v06 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: DSDT 0x00000000F9EC0018 02C1BB (v02 Ampere Jade     00000001 INTL 20201217)
[    0.000000] ACPI: BERT 0x00000000FFFEFF98 000030 (v01 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: DBG2 0x00000000FFFEFA98 00005C (v00 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: GTDT 0x00000000FFFEE998 000110 (v03 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: SPCR 0x00000000FFFEFE18 000050 (v02 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: EINJ 0x00000000FFFEF598 000150 (v01 Ampere Altra    00000001 INTL 20201217)
[    0.000000] ACPI: HEST 0x00000000FFFEEB18 0001F4 (v01 Ampere Altra    00000001 INTL 20201217)
[    0.000000] ACPI: SSDT 0x00000000FFFEFA18 00002D (v02 Ampere Altra    00000001 INTL 20201217)
[    0.000000] ACPI: TPM2 0x00000000FFFEFD18 00004C (v04 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: APIC 0x00000000FFFE7518 000AF4 (v05 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: PPTT 0x00000000FFFEEF18 000554 (v02 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: SLIT 0x00000000FFFEFD98 00002D (v01 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: SRAT 0x00000000FFFECE18 000370 (v03 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: PCCT 0x00000000FFFEE318 000576 (v02 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: IORT 0x00000000FFFED218 000498 (v00 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: MCFG 0x00000000FFFEF718 00008C (v01 Ampere Altra    00000001 AMP. 01000013)
[    0.000000] ACPI: BDAT 0x00000000FFFEF818 000030 (v01 Ampere Altra    00000002 AMP. 01000013)
[    0.000000] ACPI: SPCR: console: pl011,mmio32,0x100002600000,115200
[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x88300000-0x883fffff]
[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x90000000-0xffffffff]
[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x80000000000-0x8007fffffff]
[    0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x80100000000-0x807ffffffff]
[    0.000000] NUMA: NODE_DATA [mem 0x807fc058880-0x807fc05afff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000088300000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000807ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000088300000-0x00000000883fffff]
[    0.000000]   node   0: [mem 0x0000000090000000-0x0000000091ffffff]
[    0.000000]   node   0: [mem 0x0000000092000000-0x00000000927bffff]
[    0.000000]   node   0: [mem 0x00000000927c0000-0x00000000f835ffff]
[    0.000000]   node   0: [mem 0x00000000f8360000-0x00000000f837ffff]
[    0.000000]   node   0: [mem 0x00000000f8380000-0x00000000f83affff]
[    0.000000]   node   0: [mem 0x00000000f83b0000-0x00000000f842ffff]
[    0.000000]   node   0: [mem 0x00000000f8430000-0x00000000f9151fff]
[    0.000000]   node   0: [mem 0x00000000f9152000-0x00000000f91a7fff]
[    0.000000]   node   0: [mem 0x00000000f91a8000-0x00000000f947ffff]
[    0.000000]   node   0: [mem 0x00000000f9480000-0x00000000f94bffff]
[    0.000000]   node   0: [mem 0x00000000f94c0000-0x00000000f986ffff]
[    0.000000]   node   0: [mem 0x00000000f9870000-0x00000000f98affff]
[    0.000000]   node   0: [mem 0x00000000f98b0000-0x00000000f994ffff]
[    0.000000]   node   0: [mem 0x00000000f9950000-0x00000000f998ffff]
[    0.000000]   node   0: [mem 0x00000000f9990000-0x00000000f9f1ffff]
[    0.000000]   node   0: [mem 0x00000000f9f20000-0x00000000f9f3ffff]
[    0.000000]   node   0: [mem 0x00000000f9f40000-0x00000000fa07ffff]
[    0.000000]   node   0: [mem 0x00000000fa080000-0x00000000fa0bffff]
[    0.000000]   node   0: [mem 0x00000000fa0c0000-0x00000000fa1affff]
[    0.000000]   node   0: [mem 0x00000000fa1b0000-0x00000000fa24ffff]
[    0.000000]   node   0: [mem 0x00000000fa250000-0x00000000fa31ffff]
[    0.000000]   node   0: [mem 0x00000000fa320000-0x00000000fa48ffff]
[    0.000000]   node   0: [mem 0x00000000fa490000-0x00000000fa49ffff]
[    0.000000]   node   0: [mem 0x00000000fa4a0000-0x00000000fa55ffff]
[    0.000000]   node   0: [mem 0x00000000fa560000-0x00000000fa70ffff]
[    0.000000]   node   0: [mem 0x00000000fa710000-0x00000000fa7affff]
[    0.000000]   node   0: [mem 0x00000000fa7b0000-0x00000000faa2ffff]
[    0.000000]   node   0: [mem 0x00000000faa30000-0x00000000faacffff]
[    0.000000]   node   0: [mem 0x00000000faad0000-0x00000000fab5ffff]
[    0.000000]   node   0: [mem 0x00000000fab60000-0x00000000fabfffff]
[    0.000000]   node   0: [mem 0x00000000fac00000-0x00000000fac8ffff]
[    0.000000]   node   0: [mem 0x00000000fac90000-0x00000000fad2ffff]
[    0.000000]   node   0: [mem 0x00000000fad30000-0x00000000fadfffff]
[    0.000000]   node   0: [mem 0x00000000fae00000-0x00000000fae9ffff]
[    0.000000]   node   0: [mem 0x00000000faea0000-0x00000000faf2ffff]
[    0.000000]   node   0: [mem 0x00000000faf30000-0x00000000fafcffff]
[    0.000000]   node   0: [mem 0x00000000fafd0000-0x00000000ffefffff]
[    0.000000]   node   0: [mem 0x00000000fff00000-0x00000000fff4ffff]
[    0.000000]   node   0: [mem 0x00000000fff50000-0x00000000fffaffff]
[    0.000000]   node   0: [mem 0x00000000fffb0000-0x00000000fffdffff]
[    0.000000]   node   0: [mem 0x00000000fffe0000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000080000000000-0x000008007fffffff]
[    0.000000]   node   0: [mem 0x0000080100000000-0x00000807ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000088300000-0x00000807ffffffff]
[    0.000000] On node 0, zone DMA: 768 pages in unavailable ranges
[    0.000000] On node 0, zone DMA: 31744 pages in unavailable ranges
[    0.000000] cma: Reserved 64 MiB at 0x00000000fbe00000 on node -1
[    0.000000] psci: probing for conduit method from ACPI.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: MIGRATE_INFO_TYPE not supported.
[    0.000000] psci: SMC Calling Convention v1.2
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x80000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x80100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x90000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x90100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe0000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xe0100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf0000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0xf0100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x100100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x110000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x110100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x160000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x160100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x170000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x170100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x180000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x180100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x190000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x190100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1e0000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1e0100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1f0000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1f0100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x200100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x210000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x210100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x260000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x260100 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x270000 -> Node 0
[    0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x270100 -> Node 0
[    0.000000] percpu: Embedded 31 pages/cpu s87720 r8192 d31064 u126976
[    0.000000] pcpu-alloc: s87720 r8192 d31064 u126976 alloc=31*4096
[    0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0] 06 [0] 07 
[    0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0] 14 [0] 15 
[    0.000000] pcpu-alloc: [0] 16 [0] 17 [0] 18 [0] 19 [0] 20 [0] 21 [0] 22 [0] 23 
[    0.000000] pcpu-alloc: [0] 24 [0] 25 [0] 26 [0] 27 [0] 28 [0] 29 [0] 30 [0] 31 
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: Virtualization Host Extensions
[    0.000000] CPU features: detected: Hardware dirty bit management
[    0.000000] CPU features: detected: Spectre-v4
[    0.000000] CPU features: detected: Spectre-BHB
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] CPU features: detected: ARM erratum 1418040
[    0.000000] alternatives: applying boot alternatives
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.6-rc7+ ignore_loglevel root=/dev/nvme1n1p2 ro nr_cpus=32
[    0.000000] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-6.6-rc7+", will be passed to user space.
[    0.000000] printk: log_buf_len individual max cpu contribution: 4096 bytes
[    0.000000] printk: log_buf_len total cpu_extra contributions: 126976 bytes
[    0.000000] printk: log_buf_len min size: 131072 bytes
[    0.000000] printk: log_buf_len: 262144 bytes
[    0.000000] printk: early log buf free: 120984(92%)
[    0.000000] Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes, linear)
[    0.000000] Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[    0.000000] Fallback order for Node 0: 0 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 8193276
[    0.000000] Policy zone: Normal
[    0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off
[    0.000000] software IO TLB: area num 32.
[    0.000000] software IO TLB: mapped [mem 0x00000000f4160000-0x00000000f8160000] (64MB)
[    0.000000] Memory: 1817360K/33293312K available (12672K kernel code, 2150K rwdata, 4156K rodata, 5568K init, 508K bss, 751976K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=32, Nodes=1
[    0.000000] ftrace: allocating 42112 entries in 165 pages
[    0.000000] ftrace: allocated 165 pages with 4 groups
[    0.000000] trace event string verifier disabled
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] 	Rude variant of Tasks RCU enabled.
[    0.000000] 	Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 672 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: GICv3 features: 16 PPIs
[    0.000000] GICv3: CPU0: found redistributor 100000 region 0:0x0000100100540000
[    0.000000] SRAT: PXM 0 -> ITS 0 -> Node 0
[    0.000000] SRAT: PXM 0 -> ITS 1 -> Node 0
[    0.000000] SRAT: PXM 0 -> ITS 2 -> Node 0
[    0.000000] SRAT: PXM 0 -> ITS 3 -> Node 0
[    0.000000] SRAT: PXM 0 -> ITS 4 -> Node 0
[    0.000000] SRAT: PXM 0 -> ITS 5 -> Node 0
[    0.000000] SRAT: PXM 0 -> ITS 6 -> Node 0
[    0.000000] SRAT: PXM 0 -> ITS 7 -> Node 0
[    0.000000] ITS [mem 0x100100040000-0x10010005ffff]
[    0.000000] ITS@0x0000100100040000: allocated 8192 Devices @80000250000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x0000100100040000: allocated 32768 Interrupt Collections @80000260000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] ITS [mem 0x100100060000-0x10010007ffff]
[    0.000000] ITS@0x0000100100060000: allocated 8192 Devices @80000280000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x0000100100060000: allocated 32768 Interrupt Collections @80000290000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] ITS [mem 0x100100080000-0x10010009ffff]
[    0.000000] ITS@0x0000100100080000: allocated 8192 Devices @800002b0000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x0000100100080000: allocated 32768 Interrupt Collections @800002c0000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] ITS [mem 0x1001000a0000-0x1001000bffff]
[    0.000000] ITS@0x00001001000a0000: allocated 8192 Devices @800002e0000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x00001001000a0000: allocated 32768 Interrupt Collections @800002f0000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] ITS [mem 0x1001000c0000-0x1001000dffff]
[    0.000000] ITS@0x00001001000c0000: allocated 8192 Devices @80000310000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x00001001000c0000: allocated 32768 Interrupt Collections @80000320000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] ITS [mem 0x1001000e0000-0x1001000fffff]
[    0.000000] ITS@0x00001001000e0000: allocated 8192 Devices @80000340000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x00001001000e0000: allocated 32768 Interrupt Collections @80000350000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] ITS [mem 0x100100100000-0x10010011ffff]
[    0.000000] ITS@0x0000100100100000: allocated 8192 Devices @80000370000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x0000100100100000: allocated 32768 Interrupt Collections @80000380000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] ITS [mem 0x100100120000-0x10010013ffff]
[    0.000000] ITS@0x0000100100120000: allocated 8192 Devices @800003a0000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x0000100100120000: allocated 32768 Interrupt Collections @800003b0000 (flat, esz 2, psz 64K, shr 1)
[    0.000000] GICv3: using LPI property table @0x00000800003c0000
[    0.000000] GICv3: CPU0: using allocated LPI pending table @0x00000800003d0000
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.000000] ACPI GTDT: found 1 memory-mapped timer block(s).
[    0.000000] arch_timer: cp15 and mmio timer(s) running at 25.00MHz (phys/phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x5c40939b5, max_idle_ns: 440795202646 ns
[    0.000000] sched_clock: 56 bits at 25MHz, resolution 40ns, wraps every 4398046511100ns
[    0.000086] Console: colour dummy device 80x25
[    0.000125] ACPI: Core revision 20230628
[    0.000393] Calibrating delay loop (skipped), value calculated using timer frequency.. 50.00 BogoMIPS (lpj=100000)
[    0.000397] pid_max: default: 32768 minimum: 301
[    0.000461] LSM: initializing lsm=capability,bpf
[    0.000480] LSM support for eBPF active
[    0.000530] Mount-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.000550] Mountpoint-cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.001120] RCU Tasks: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1.
[    0.001141] RCU Tasks Rude: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1.
[    0.001163] RCU Tasks Trace: Setting shift to 5 and lim to 1 rcu_task_cb_adjust=1.
[    0.001218] rcu: Hierarchical SRCU implementation.
[    0.001219] rcu: 	Max phase no-delay instances is 1000.
[    0.001488] Platform MSI: ITS@0x100100040000 domain created
[    0.001491] Platform MSI: ITS@0x100100060000 domain created
[    0.001493] Platform MSI: ITS@0x100100080000 domain created
[    0.001495] Platform MSI: ITS@0x1001000a0000 domain created
[    0.001497] Platform MSI: ITS@0x1001000c0000 domain created
[    0.001499] Platform MSI: ITS@0x1001000e0000 domain created
[    0.001501] Platform MSI: ITS@0x100100100000 domain created
[    0.001503] Platform MSI: ITS@0x100100120000 domain created
[    0.001509] PCI/MSI: ITS@0x100100040000 domain created
[    0.001511] PCI/MSI: ITS@0x100100060000 domain created
[    0.001513] PCI/MSI: ITS@0x100100080000 domain created
[    0.001515] PCI/MSI: ITS@0x1001000a0000 domain created
[    0.001517] PCI/MSI: ITS@0x1001000c0000 domain created
[    0.001518] PCI/MSI: ITS@0x1001000e0000 domain created
[    0.001520] PCI/MSI: ITS@0x100100100000 domain created
[    0.001522] PCI/MSI: ITS@0x100100120000 domain created
[    0.001526] fsl-mc MSI: ITS@0x100100040000 domain created
[    0.001528] fsl-mc MSI: ITS@0x100100060000 domain created
[    0.001530] fsl-mc MSI: ITS@0x100100080000 domain created
[    0.001532] fsl-mc MSI: ITS@0x1001000a0000 domain created
[    0.001534] fsl-mc MSI: ITS@0x1001000c0000 domain created
[    0.001536] fsl-mc MSI: ITS@0x1001000e0000 domain created
[    0.001538] fsl-mc MSI: ITS@0x100100100000 domain created
[    0.001540] fsl-mc MSI: ITS@0x100100120000 domain created
[    0.001550] Remapping and enabling EFI services.
[    0.002463] smp: Bringing up secondary CPUs ...
[    0.002723] Detected PIPT I-cache on CPU1
[    0.002757] GICv3: CPU1: found redistributor 100100 region 0:0x0000100100560000
[    0.002766] GICv3: CPU1: using allocated LPI pending table @0x00000800003e0000
[    0.002815] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.002828] CPU1: Booted secondary processor 0x0000100100 [0x413fd0c1]
[    0.003226] Detected PIPT I-cache on CPU2
[    0.003267] GICv3: CPU2: found redistributor 160000 region 0:0x00001001006c0000
[    0.003277] GICv3: CPU2: using allocated LPI pending table @0x00000800003f0000
[    0.003330] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.003344] CPU2: Booted secondary processor 0x0000160000 [0x413fd0c1]
[    0.003681] Detected PIPT I-cache on CPU3
[    0.003715] GICv3: CPU3: found redistributor 160100 region 0:0x00001001006e0000
[    0.003724] GICv3: CPU3: using allocated LPI pending table @0x0000080000800000
[    0.003775] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.003789] CPU3: Booted secondary processor 0x0000160100 [0x413fd0c1]
[    0.004129] Detected PIPT I-cache on CPU4
[    0.004165] GICv3: CPU4: found redistributor 180000 region 0:0x0000100100740000
[    0.004174] GICv3: CPU4: using allocated LPI pending table @0x0000080000810000
[    0.004226] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.004242] CPU4: Booted secondary processor 0x0000180000 [0x413fd0c1]
[    0.004622] Detected PIPT I-cache on CPU5
[    0.004657] GICv3: CPU5: found redistributor 180100 region 0:0x0000100100760000
[    0.004666] GICv3: CPU5: using allocated LPI pending table @0x0000080000820000
[    0.004716] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.004729] CPU5: Booted secondary processor 0x0000180100 [0x413fd0c1]
[    0.005034] Detected PIPT I-cache on CPU6
[    0.005074] GICv3: CPU6: found redistributor 1e0000 region 0:0x00001001008c0000
[    0.005083] GICv3: CPU6: using allocated LPI pending table @0x0000080000830000
[    0.005135] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.005146] CPU6: Booted secondary processor 0x00001e0000 [0x413fd0c1]
[    0.005465] Detected PIPT I-cache on CPU7
[    0.005503] GICv3: CPU7: found redistributor 1e0100 region 0:0x00001001008e0000
[    0.005512] GICv3: CPU7: using allocated LPI pending table @0x0000080000840000
[    0.005563] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.005576] CPU7: Booted secondary processor 0x00001e0100 [0x413fd0c1]
[    0.005902] Detected PIPT I-cache on CPU8
[    0.005934] GICv3: CPU8: found redistributor 80000 region 0:0x0000100100340000
[    0.005944] GICv3: CPU8: using allocated LPI pending table @0x0000080000850000
[    0.005997] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.006013] CPU8: Booted secondary processor 0x0000080000 [0x413fd0c1]
[    0.006476] Detected PIPT I-cache on CPU9
[    0.006506] GICv3: CPU9: found redistributor 80100 region 0:0x0000100100360000
[    0.006515] GICv3: CPU9: using allocated LPI pending table @0x0000080000860000
[    0.006566] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.006580] CPU9: Booted secondary processor 0x0000080100 [0x413fd0c1]
[    0.006906] Detected PIPT I-cache on CPU10
[    0.006942] GICv3: CPU10: found redistributor e0000 region 0:0x00001001004c0000
[    0.006952] GICv3: CPU10: using allocated LPI pending table @0x0000080000870000
[    0.007006] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.007022] CPU10: Booted secondary processor 0x00000e0000 [0x413fd0c1]
[    0.007411] Detected PIPT I-cache on CPU11
[    0.007444] GICv3: CPU11: found redistributor e0100 region 0:0x00001001004e0000
[    0.007454] GICv3: CPU11: using allocated LPI pending table @0x0000080000880000
[    0.007506] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.007518] CPU11: Booted secondary processor 0x00000e0100 [0x413fd0c1]
[    0.007898] Detected PIPT I-cache on CPU12
[    0.007942] GICv3: CPU12: found redistributor 200000 region 0:0x0000100100940000
[    0.007951] GICv3: CPU12: using allocated LPI pending table @0x0000080000890000
[    0.008003] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.008019] CPU12: Booted secondary processor 0x0000200000 [0x413fd0c1]
[    0.008431] Detected PIPT I-cache on CPU13
[    0.008472] GICv3: CPU13: found redistributor 200100 region 0:0x0000100100960000
[    0.008481] GICv3: CPU13: using allocated LPI pending table @0x00000800008a0000
[    0.008530] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.008543] CPU13: Booted secondary processor 0x0000200100 [0x413fd0c1]
[    0.008834] Detected PIPT I-cache on CPU14
[    0.008882] GICv3: CPU14: found redistributor 260000 region 0:0x0000100100ac0000
[    0.008891] GICv3: CPU14: using allocated LPI pending table @0x00000800008b0000
[    0.008944] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.008957] CPU14: Booted secondary processor 0x0000260000 [0x413fd0c1]
[    0.009303] Detected PIPT I-cache on CPU15
[    0.009346] GICv3: CPU15: found redistributor 260100 region 0:0x0000100100ae0000
[    0.009355] GICv3: CPU15: using allocated LPI pending table @0x00000800008c0000
[    0.009405] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.009419] CPU15: Booted secondary processor 0x0000260100 [0x413fd0c1]
[    0.009904] Detected PIPT I-cache on CPU16
[    0.009943] GICv3: CPU16: found redistributor 110000 region 0:0x0000100100580000
[    0.009952] GICv3: CPU16: using allocated LPI pending table @0x00000800008d0000
[    0.010004] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.010020] CPU16: Booted secondary processor 0x0000110000 [0x413fd0c1]
[    0.010485] Detected PIPT I-cache on CPU17
[    0.010521] GICv3: CPU17: found redistributor 110100 region 0:0x00001001005a0000
[    0.010531] GICv3: CPU17: using allocated LPI pending table @0x00000800008e0000
[    0.010582] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.010595] CPU17: Booted secondary processor 0x0000110100 [0x413fd0c1]
[    0.010918] Detected PIPT I-cache on CPU18
[    0.010960] GICv3: CPU18: found redistributor 170000 region 0:0x0000100100700000
[    0.010969] GICv3: CPU18: using allocated LPI pending table @0x00000800008f0000
[    0.011023] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.011039] CPU18: Booted secondary processor 0x0000170000 [0x413fd0c1]
[    0.011488] Detected PIPT I-cache on CPU19
[    0.011528] GICv3: CPU19: found redistributor 170100 region 0:0x0000100100720000
[    0.011538] GICv3: CPU19: using allocated LPI pending table @0x0000080000900000
[    0.011589] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.011603] CPU19: Booted secondary processor 0x0000170100 [0x413fd0c1]
[    0.011942] Detected PIPT I-cache on CPU20
[    0.011986] GICv3: CPU20: found redistributor 190000 region 0:0x0000100100780000
[    0.011995] GICv3: CPU20: using allocated LPI pending table @0x0000080000910000
[    0.012048] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.012063] CPU20: Booted secondary processor 0x0000190000 [0x413fd0c1]
[    0.012500] Detected PIPT I-cache on CPU21
[    0.012540] GICv3: CPU21: found redistributor 190100 region 0:0x00001001007a0000
[    0.012550] GICv3: CPU21: using allocated LPI pending table @0x0000080000920000
[    0.012600] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.012613] CPU21: Booted secondary processor 0x0000190100 [0x413fd0c1]
[    0.013086] Detected PIPT I-cache on CPU22
[    0.013133] GICv3: CPU22: found redistributor 1f0000 region 0:0x0000100100900000
[    0.013143] GICv3: CPU22: using allocated LPI pending table @0x0000080000930000
[    0.013196] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.013212] CPU22: Booted secondary processor 0x00001f0000 [0x413fd0c1]
[    0.013685] Detected PIPT I-cache on CPU23
[    0.013728] GICv3: CPU23: found redistributor 1f0100 region 0:0x0000100100920000
[    0.013738] GICv3: CPU23: using allocated LPI pending table @0x0000080000940000
[    0.013789] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.013802] CPU23: Booted secondary processor 0x00001f0100 [0x413fd0c1]
[    0.014302] Detected PIPT I-cache on CPU24
[    0.014341] GICv3: CPU24: found redistributor 90000 region 0:0x0000100100380000
[    0.014351] GICv3: CPU24: using allocated LPI pending table @0x0000080000950000
[    0.014404] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.014421] CPU24: Booted secondary processor 0x0000090000 [0x413fd0c1]
[    0.014885] Detected PIPT I-cache on CPU25
[    0.014920] GICv3: CPU25: found redistributor 90100 region 0:0x00001001003a0000
[    0.014930] GICv3: CPU25: using allocated LPI pending table @0x0000080000960000
[    0.014981] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.014995] CPU25: Booted secondary processor 0x0000090100 [0x413fd0c1]
[    0.015322] Detected PIPT I-cache on CPU26
[    0.015365] GICv3: CPU26: found redistributor f0000 region 0:0x0000100100500000
[    0.015375] GICv3: CPU26: using allocated LPI pending table @0x0000080000970000
[    0.015430] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.015446] CPU26: Booted secondary processor 0x00000f0000 [0x413fd0c1]
[    0.015893] Detected PIPT I-cache on CPU27
[    0.015931] GICv3: CPU27: found redistributor f0100 region 0:0x0000100100520000
[    0.015942] GICv3: CPU27: using allocated LPI pending table @0x0000080000980000
[    0.015994] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.016008] CPU27: Booted secondary processor 0x00000f0100 [0x413fd0c1]
[    0.016485] Detected PIPT I-cache on CPU28
[    0.016536] GICv3: CPU28: found redistributor 210000 region 0:0x0000100100980000
[    0.016545] GICv3: CPU28: using allocated LPI pending table @0x0000080000990000
[    0.016597] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.016613] CPU28: Booted secondary processor 0x0000210000 [0x413fd0c1]
[    0.017092] Detected PIPT I-cache on CPU29
[    0.017137] GICv3: CPU29: found redistributor 210100 region 0:0x00001001009a0000
[    0.017147] GICv3: CPU29: using allocated LPI pending table @0x00000800009a0000
[    0.017196] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.017210] CPU29: Booted secondary processor 0x0000210100 [0x413fd0c1]
[    0.017691] Detected PIPT I-cache on CPU30
[    0.017745] GICv3: CPU30: found redistributor 270000 region 0:0x0000100100b00000
[    0.017755] GICv3: CPU30: using allocated LPI pending table @0x00000800009b0000
[    0.017807] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.017822] CPU30: Booted secondary processor 0x0000270000 [0x413fd0c1]
[    0.018296] Detected PIPT I-cache on CPU31
[    0.018345] GICv3: CPU31: found redistributor 270100 region 0:0x0000100100b20000
[    0.018355] GICv3: CPU31: using allocated LPI pending table @0x00000800009c0000
[    0.018405] arch_timer: Enabling local workaround for ARM erratum 1418040
[    0.018418] CPU31: Booted secondary processor 0x0000270100 [0x413fd0c1]
[    0.018527] smp: Brought up 1 node, 32 CPUs
[    0.018531] SMP: Total of 32 processors activated.
[    0.018533] CPU features: detected: 32-bit EL0 Support
[    0.018535] CPU features: detected: Data cache clean to the PoU not required for I/D coherence
[    0.018537] CPU features: detected: Common not Private translations
[    0.018538] CPU features: detected: CRC32 instructions
[    0.018539] CPU features: detected: Data cache clean to Point of Persistence
[    0.018541] CPU features: detected: Enhanced Virtualization Traps
[    0.018543] CPU features: detected: RCpc load-acquire (LDAPR)
[    0.018544] CPU features: detected: LSE atomic instructions
[    0.018545] CPU features: detected: Privileged Access Never
[    0.018547] CPU features: detected: RAS Extension Support
[    0.018549] CPU features: detected: Speculative Store Bypassing Safe (SSBS)
[    0.018787] CPU: All CPU(s) started at EL2
[    0.018788] alternatives: applying system-wide alternatives
[    0.195225] node 0 deferred pages initialised in 172ms
[    0.195515] devtmpfs: initialized
[    0.200612] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.200639] futex hash table entries: 8192 (order: 7, 524288 bytes, linear)
[    0.201019] pinctrl core: initialized pinctrl subsystem
[    0.201163] SMBIOS 3.3.0 present.
[    0.201171] DMI: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[    0.201379] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.202235] DMA: preallocated 4096 KiB GFP_KERNEL pool for atomic allocations
[    0.202585] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.202929] DMA: preallocated 4096 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.202942] audit: initializing netlink subsys (disabled)
[    0.202996] audit: type=2000 audit(0.200:1): state=initialized audit_enabled=0 res=1
[    0.203142] thermal_sys: Registered thermal governor 'fair_share'
[    0.203144] thermal_sys: Registered thermal governor 'bang_bang'
[    0.203146] thermal_sys: Registered thermal governor 'step_wise'
[    0.203147] thermal_sys: Registered thermal governor 'user_space'
[    0.203148] thermal_sys: Registered thermal governor 'power_allocator'
[    0.203163] cpuidle: using governor ladder
[    0.203171] cpuidle: using governor menu
[    0.203373] Detected 15 PCC Subspaces
[    0.203374] Registering PCC driver as Mailbox controller
[    0.203431] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.203749] ASID allocator initialised with 32768 entries
[    0.203775] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[    0.203777] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.203838] Serial: AMBA PL011 UART driver
[    0.204382] Modules: 2G module region forced by RANDOMIZE_MODULE_REGION_FULL
[    0.204384] Modules: 0 pages in range for non-PLT usage
[    0.204385] Modules: 517984 pages in range for PLT usage
[    0.208292] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[    0.208295] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page
[    0.208297] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
[    0.208298] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page
[    0.208300] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[    0.208301] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page
[    0.208302] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
[    0.208303] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
[    0.208846] ACPI: Added _OSI(Module Device)
[    0.208848] ACPI: Added _OSI(Processor Device)
[    0.208849] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.208851] ACPI: Added _OSI(Processor Aggregator Device)
[    0.226435] ACPI: 2 ACPI AML tables successfully acquired and loaded
[    0.230050] ACPI: Interpreter enabled
[    0.230053] ACPI: Using GIC for interrupt routing
[    0.231021] ACPI: MCFG table detected, 6 entries
[    0.231170] HEST: Table parsing has been initialized.
[    0.231197] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 0
[    0.231199] {1}[Hardware Error]: event severity: recoverable
[    0.231201] {1}[Hardware Error]:  Error 0, type: recoverable
[    0.231203] {1}[Hardware Error]:   section type: unknown, e8ed898d-df16-43cc-8ecc-54f060ef157f
[    0.231205] {1}[Hardware Error]:   section length: 0x30
[    0.231209] {1}[Hardware Error]:   00000000: 00000005 ec30000e 00040110 80001001  ......0.........
[    0.231211] {1}[Hardware Error]:   00000010: 00000300 00000000 00000000 00000000  ................
[    0.231213] {1}[Hardware Error]:   00000020: 00000000 00000000 00000000 00000000  ................
[    0.231291] GHES: APEI firmware first mode is enabled by APEI bit.
[    0.231296] ACPI: IORT: SMMU-v3[33ffe0000000] Mapped to Proximity domain 0
[    0.231323] ACPI: IORT: SMMU-v3[37ffe0000000] Mapped to Proximity domain 0
[    0.231337] ACPI: IORT: SMMU-v3[3fffe0000000] Mapped to Proximity domain 0
[    0.231352] ACPI: IORT: SMMU-v3[23ffe0000000] Mapped to Proximity domain 0
[    0.231366] ACPI: IORT: SMMU-v3[2bffe0000000] Mapped to Proximity domain 0
[    0.231381] ACPI: IORT: SMMU-v3[2fffe0000000] Mapped to Proximity domain 0
[    0.258953] ARMH0011:00: ttyAMA0 at MMIO 0x100002600000 (irq = 55, base_baud = 0) is a SBSA
[    0.258962] printk: console [ttyAMA0] enabled
[    3.346845] ARMH0011:01: ttyAMA1 at MMIO 0x100002620000 (irq = 56, base_baud = 0) is a SBSA
[    3.356622] ACPI: PCI Root Bridge [PCI0] (domain 000c [bus 00-ff])
[    3.362805] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    3.371901] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME LTR]
[    3.380593] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability]
[    3.387196] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration
[    3.395366] acpi PNP0A08:00: MCFG quirk: ECAM at [mem 0x33fff0000000-0x33ffffffffff] for [bus 00-ff] with 0xffffb6de2d80ca60
[    3.407662] acpi PNP0A08:00: ECAM area [mem 0x33fff0000000-0x33ffffffffff] reserved by PNP0C02:00
[    3.416537] acpi PNP0A08:00: ECAM at [mem 0x33fff0000000-0x33ffffffffff] for [bus 00-ff]
[    3.424700] PCI host bridge to bus 000c:00
[    3.428787] pci_bus 000c:00: root bus resource [mem 0x40000000-0x4fffffff window]
[    3.436259] pci_bus 000c:00: root bus resource [mem 0x300000000000-0x33ffdfffffff window]
[    3.444425] pci_bus 000c:00: root bus resource [bus 00-ff]
[    3.449928] pci 000c:00:00.0: [1def:e100] type 00 class 0x060000
[    3.455998] pci 000c:00:01.0: [1def:e101] type 01 class 0x060400
[    3.462040] pci 000c:00:01.0: supports D1 D2
[    3.466303] pci 000c:00:01.0: PME# supported from D0 D1 D3hot
[    3.474226] pci_bus 000c:00: on NUMA node 0
[    3.478401] pci 000c:00:01.0: bridge window [io  0x1000-0x0fff] to [bus 01] add_size 1000
[    3.486569] pci 000c:00:01.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 01] add_size 200000 add_align 100000
[    3.498034] pci 000c:00:01.0: bridge window [mem 0x00100000-0x000fffff] to [bus 01] add_size 200000 add_align 100000
[    3.508547] pci 000c:00:01.0: BAR 14: assigned [mem 0x40000000-0x401fffff]
[    3.515412] pci 000c:00:01.0: BAR 15: assigned [mem 0x300000000000-0x3000001fffff 64bit pref]
[    3.523925] pci 000c:00:01.0: BAR 13: no space for [io  size 0x1000]
[    3.530267] pci 000c:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    3.536959] pci 000c:00:01.0: BAR 13: no space for [io  size 0x1000]
[    3.543301] pci 000c:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    3.549992] pci 000c:00:01.0: PCI bridge to [bus 01]
[    3.554952] pci 000c:00:01.0:   bridge window [mem 0x40000000-0x401fffff]
[    3.561729] pci 000c:00:01.0:   bridge window [mem 0x300000000000-0x3000001fffff 64bit pref]
[    3.570157] pci_bus 000c:00: resource 4 [mem 0x40000000-0x4fffffff window]
[    3.577022] pci_bus 000c:00: resource 5 [mem 0x300000000000-0x33ffdfffffff window]
[    3.584582] pci_bus 000c:01: resource 1 [mem 0x40000000-0x401fffff]
[    3.590837] pci_bus 000c:01: resource 2 [mem 0x300000000000-0x3000001fffff 64bit pref]
[    3.598770] ACPI: PCI Root Bridge [PCI1] (domain 000d [bus 00-ff])
[    3.604945] acpi PNP0A08:01: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    3.614035] acpi PNP0A08:01: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME LTR]
[    3.622724] acpi PNP0A08:01: _OSC: OS now controls [AER PCIeCapability]
[    3.629328] acpi PNP0A08:01: FADT indicates ASPM is unsupported, using BIOS configuration
[    3.637494] acpi PNP0A08:01: MCFG quirk: ECAM at [mem 0x37fff0000000-0x37ffffffffff] for [bus 00-ff] with 0xffffb6de2d80ca60
[    3.649788] acpi PNP0A08:01: ECAM area [mem 0x37fff0000000-0x37ffffffffff] reserved by PNP0C02:00
[    3.658659] acpi PNP0A08:01: ECAM at [mem 0x37fff0000000-0x37ffffffffff] for [bus 00-ff]
[    3.666812] PCI host bridge to bus 000d:00
[    3.670899] pci_bus 000d:00: root bus resource [mem 0x50000000-0x5fffffff window]
[    3.678371] pci_bus 000d:00: root bus resource [mem 0x340000000000-0x37ffdfffffff window]
[    3.686536] pci_bus 000d:00: root bus resource [bus 00-ff]
[    3.692035] pci 000d:00:00.0: [1def:e100] type 00 class 0x060000
[    3.698093] pci 000d:00:01.0: [1def:e101] type 01 class 0x060400
[    3.704120] pci 000d:00:01.0: supports D1 D2
[    3.708379] pci 000d:00:01.0: PME# supported from D0 D1 D3hot
[    3.714227] pci 000d:01:00.0: [1002:1478] type 01 class 0x060400
[    3.720240] pci 000d:01:00.0: reg 0x10: [mem 0x50200000-0x50203fff]
[    3.726601] pci 000d:01:00.0: PME# supported from D0 D3hot D3cold
[    3.732770] pci 000d:01:00.0: 63.008 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x8 link at 000d:00:01.0 (capable of 126.024 Gb/s with 16.0 GT/s PCIe x8 link)
[    3.748116] pci 000d:02:00.0: [1002:1479] type 01 class 0x060400
[    3.754234] pci 000d:02:00.0: PME# supported from D0 D3hot D3cold
[    3.760543] pci 000d:03:00.0: [1002:7480] type 00 class 0x030000
[    3.766557] pci 000d:03:00.0: reg 0x10: [mem 0x340000000000-0x34000fffffff 64bit pref]
[    3.774475] pci 000d:03:00.0: reg 0x18: [mem 0x340010000000-0x3400101fffff 64bit pref]
[    3.782397] pci 000d:03:00.0: reg 0x20: [io  0x5fffe000-0x5fffe0ff]
[    3.788661] pci 000d:03:00.0: reg 0x24: [mem 0x50000000-0x500fffff]
[    3.794925] pci 000d:03:00.0: reg 0x30: [mem 0xfffe0000-0xffffffff pref]
[    3.801711] pci 000d:03:00.0: PME# supported from D1 D2 D3hot D3cold
[    3.808154] pci 000d:03:00.0: 63.008 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x8 link at 000d:00:01.0 (capable of 126.024 Gb/s with 16.0 GT/s PCIe x8 link)
[    3.823445] pci 000d:03:00.1: [1002:ab30] type 00 class 0x040300
[    3.829455] pci 000d:03:00.1: reg 0x10: [mem 0x50100000-0x50103fff]
[    3.835804] pci 000d:03:00.1: PME# supported from D1 D2 D3hot D3cold
[    3.842296] pci_bus 000d:00: on NUMA node 0
[    3.846475] pci 000d:00:01.0: BAR 15: assigned [mem 0x340000000000-0x340017ffffff 64bit pref]
[    3.854989] pci 000d:00:01.0: BAR 14: assigned [mem 0x50000000-0x502fffff]
[    3.861852] pci 000d:00:01.0: BAR 13: no space for [io  size 0x1000]
[    3.868195] pci 000d:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    3.874886] pci 000d:01:00.0: BAR 15: assigned [mem 0x340000000000-0x340017ffffff 64bit pref]
[    3.883399] pci 000d:01:00.0: BAR 14: assigned [mem 0x50000000-0x501fffff]
[    3.890262] pci 000d:01:00.0: BAR 0: assigned [mem 0x50200000-0x50203fff]
[    3.897045] pci 000d:01:00.0: BAR 13: no space for [io  size 0x1000]
[    3.903387] pci 000d:01:00.0: BAR 13: failed to assign [io  size 0x1000]
[    3.910078] pci 000d:02:00.0: BAR 15: assigned [mem 0x340000000000-0x340017ffffff 64bit pref]
[    3.918591] pci 000d:02:00.0: BAR 14: assigned [mem 0x50000000-0x501fffff]
[    3.925454] pci 000d:02:00.0: BAR 13: no space for [io  size 0x1000]
[    3.931797] pci 000d:02:00.0: BAR 13: failed to assign [io  size 0x1000]
[    3.938488] pci 000d:03:00.0: BAR 0: assigned [mem 0x340000000000-0x34000fffffff 64bit pref]
[    3.946926] pci 000d:03:00.0: BAR 2: assigned [mem 0x340010000000-0x3400101fffff 64bit pref]
[    3.955363] pci 000d:03:00.0: BAR 5: assigned [mem 0x50000000-0x500fffff]
[    3.962146] pci 000d:03:00.0: BAR 6: assigned [mem 0x50100000-0x5011ffff pref]
[    3.969356] pci 000d:03:00.1: BAR 0: assigned [mem 0x50120000-0x50123fff]
[    3.976139] pci 000d:03:00.0: BAR 4: no space for [io  size 0x0100]
[    3.982394] pci 000d:03:00.0: BAR 4: failed to assign [io  size 0x0100]
[    3.988997] pci 000d:02:00.0: PCI bridge to [bus 03]
[    3.993957] pci 000d:02:00.0:   bridge window [mem 0x50000000-0x501fffff]
[    4.000739] pci 000d:02:00.0:   bridge window [mem 0x340000000000-0x340017ffffff 64bit pref]
[    4.009183] pci 000d:01:00.0: PCI bridge to [bus 02-03]
[    4.014404] pci 000d:01:00.0:   bridge window [mem 0x50000000-0x501fffff]
[    4.021185] pci 000d:01:00.0:   bridge window [mem 0x340000000000-0x340017ffffff 64bit pref]
[    4.029618] pci 000d:00:01.0: PCI bridge to [bus 01-03]
[    4.034833] pci 000d:00:01.0:   bridge window [mem 0x50000000-0x502fffff]
[    4.041610] pci 000d:00:01.0:   bridge window [mem 0x340000000000-0x340017ffffff 64bit pref]
[    4.050036] pci_bus 000d:00: Some PCI device resources are unassigned, try booting with pci=realloc
[    4.059070] pci_bus 000d:00: resource 4 [mem 0x50000000-0x5fffffff window]
[    4.065934] pci_bus 000d:00: resource 5 [mem 0x340000000000-0x37ffdfffffff window]
[    4.073492] pci_bus 000d:01: resource 1 [mem 0x50000000-0x502fffff]
[    4.079747] pci_bus 000d:01: resource 2 [mem 0x340000000000-0x340017ffffff 64bit pref]
[    4.087653] pci_bus 000d:02: resource 1 [mem 0x50000000-0x501fffff]
[    4.093908] pci_bus 000d:02: resource 2 [mem 0x340000000000-0x340017ffffff 64bit pref]
[    4.101813] pci_bus 000d:03: resource 1 [mem 0x50000000-0x501fffff]
[    4.108069] pci_bus 000d:03: resource 2 [mem 0x340000000000-0x340017ffffff 64bit pref]
[    4.116009] ACPI: PCI Root Bridge [PCI3] (domain 0000 [bus 00-ff])
[    4.122186] acpi PNP0A08:03: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    4.131275] acpi PNP0A08:03: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME LTR]
[    4.139965] acpi PNP0A08:03: _OSC: OS now controls [AER PCIeCapability]
[    4.146568] acpi PNP0A08:03: FADT indicates ASPM is unsupported, using BIOS configuration
[    4.154734] acpi PNP0A08:03: MCFG quirk: ECAM at [mem 0x3ffff0000000-0x3fffffffffff] for [bus 00-ff] with 0xffffb6de2d80ca60
[    4.167032] acpi PNP0A08:03: ECAM area [mem 0x3ffff0000000-0x3fffffffffff] reserved by PNP0C02:00
[    4.175906] acpi PNP0A08:03: ECAM at [mem 0x3ffff0000000-0x3fffffffffff] for [bus 00-ff]
[    4.184062] PCI host bridge to bus 0000:00
[    4.188149] pci_bus 0000:00: root bus resource [mem 0x70000000-0x7fffffff window]
[    4.195620] pci_bus 0000:00: root bus resource [mem 0x3c0000000000-0x3fffdfffffff window]
[    4.203786] pci_bus 0000:00: root bus resource [bus 00-ff]
[    4.209285] pci 0000:00:00.0: [1def:e100] type 00 class 0x060000
[    4.215341] pci 0000:00:01.0: [1def:e101] type 01 class 0x060400
[    4.221383] pci 0000:00:01.0: supports D1 D2
[    4.225642] pci 0000:00:01.0: PME# supported from D0 D1 D3hot
[    4.233567] pci_bus 0000:00: on NUMA node 0
[    4.237740] pci 0000:00:01.0: bridge window [io  0x1000-0x0fff] to [bus 01] add_size 1000
[    4.245907] pci 0000:00:01.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 01] add_size 200000 add_align 100000
[    4.257372] pci 0000:00:01.0: bridge window [mem 0x00100000-0x000fffff] to [bus 01] add_size 200000 add_align 100000
[    4.267884] pci 0000:00:01.0: BAR 14: assigned [mem 0x70000000-0x701fffff]
[    4.274748] pci 0000:00:01.0: BAR 15: assigned [mem 0x3c0000000000-0x3c00001fffff 64bit pref]
[    4.283261] pci 0000:00:01.0: BAR 13: no space for [io  size 0x1000]
[    4.289603] pci 0000:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    4.296293] pci 0000:00:01.0: BAR 13: no space for [io  size 0x1000]
[    4.302636] pci 0000:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    4.309325] pci 0000:00:01.0: PCI bridge to [bus 01]
[    4.314280] pci 0000:00:01.0:   bridge window [mem 0x70000000-0x701fffff]
[    4.321057] pci 0000:00:01.0:   bridge window [mem 0x3c0000000000-0x3c00001fffff 64bit pref]
[    4.329484] pci_bus 0000:00: resource 4 [mem 0x70000000-0x7fffffff window]
[    4.336347] pci_bus 0000:00: resource 5 [mem 0x3c0000000000-0x3fffdfffffff window]
[    4.343905] pci_bus 0000:01: resource 1 [mem 0x70000000-0x701fffff]
[    4.350160] pci_bus 0000:01: resource 2 [mem 0x3c0000000000-0x3c00001fffff 64bit pref]
[    4.358090] ACPI: PCI Root Bridge [PCI4] (domain 0002 [bus 00-ff])
[    4.364266] acpi PNP0A08:04: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    4.373353] acpi PNP0A08:04: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME LTR]
[    4.382042] acpi PNP0A08:04: _OSC: OS now controls [AER PCIeCapability]
[    4.388646] acpi PNP0A08:04: FADT indicates ASPM is unsupported, using BIOS configuration
[    4.396812] acpi PNP0A08:04: MCFG quirk: ECAM at [mem 0x23fff0000000-0x23ffffffffff] for [bus 00-ff] with 0xffffb6de2d80ca60
[    4.409106] acpi PNP0A08:04: ECAM area [mem 0x23fff0000000-0x23ffffffffff] reserved by PNP0C02:00
[    4.417976] acpi PNP0A08:04: ECAM at [mem 0x23fff0000000-0x23ffffffffff] for [bus 00-ff]
[    4.426155] PCI host bridge to bus 0002:00
[    4.430241] pci_bus 0002:00: root bus resource [mem 0x08000000-0x0fffffff window]
[    4.437713] pci_bus 0002:00: root bus resource [mem 0x200000000000-0x23ffdfffffff window]
[    4.445878] pci_bus 0002:00: root bus resource [bus 00-ff]
[    4.451378] pci 0002:00:00.0: [1def:e110] type 00 class 0x060000
[    4.457435] pci 0002:00:01.0: [1def:e111] type 01 class 0x060400
[    4.463472] pci 0002:00:01.0: supports D1 D2
[    4.467730] pci 0002:00:01.0: PME# supported from D0 D1 D3hot
[    4.473560] pci 0002:00:05.0: [1def:e115] type 01 class 0x060400
[    4.479595] pci 0002:00:05.0: supports D1 D2
[    4.483854] pci 0002:00:05.0: PME# supported from D0 D1 D3hot
[    4.489719] acpiphp: Slot [1] registered
[    4.497799] acpiphp: Slot [3] registered
[    4.505844] pci_bus 0002:00: on NUMA node 0
[    4.510017] pci 0002:00:01.0: bridge window [io  0x1000-0x0fff] to [bus 01] add_size 1000
[    4.518184] pci 0002:00:01.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 01] add_size 200000 add_align 100000
[    4.529649] pci 0002:00:01.0: bridge window [mem 0x00100000-0x000fffff] to [bus 01] add_size 200000 add_align 100000
[    4.540159] pci 0002:00:05.0: bridge window [io  0x1000-0x0fff] to [bus 02] add_size 1000
[    4.548325] pci 0002:00:05.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 02] add_size 200000 add_align 100000
[    4.559789] pci 0002:00:05.0: bridge window [mem 0x00100000-0x000fffff] to [bus 02] add_size 200000 add_align 100000
[    4.570301] pci 0002:00:01.0: BAR 14: assigned [mem 0x08000000-0x081fffff]
[    4.577166] pci 0002:00:01.0: BAR 15: assigned [mem 0x200000000000-0x2000001fffff 64bit pref]
[    4.585679] pci 0002:00:05.0: BAR 14: assigned [mem 0x08200000-0x083fffff]
[    4.592542] pci 0002:00:05.0: BAR 15: assigned [mem 0x200000200000-0x2000003fffff 64bit pref]
[    4.601055] pci 0002:00:01.0: BAR 13: no space for [io  size 0x1000]
[    4.607397] pci 0002:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    4.614087] pci 0002:00:05.0: BAR 13: no space for [io  size 0x1000]
[    4.620429] pci 0002:00:05.0: BAR 13: failed to assign [io  size 0x1000]
[    4.627119] pci 0002:00:05.0: BAR 13: no space for [io  size 0x1000]
[    4.633462] pci 0002:00:05.0: BAR 13: failed to assign [io  size 0x1000]
[    4.640152] pci 0002:00:01.0: BAR 13: no space for [io  size 0x1000]
[    4.646494] pci 0002:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    4.653184] pci 0002:00:01.0: PCI bridge to [bus 01]
[    4.658138] pci 0002:00:01.0:   bridge window [mem 0x08000000-0x081fffff]
[    4.664914] pci 0002:00:01.0:   bridge window [mem 0x200000000000-0x2000001fffff 64bit pref]
[    4.673341] pci 0002:00:05.0: PCI bridge to [bus 02]
[    4.678295] pci 0002:00:05.0:   bridge window [mem 0x08200000-0x083fffff]
[    4.685071] pci 0002:00:05.0:   bridge window [mem 0x200000200000-0x2000003fffff 64bit pref]
[    4.693498] pci_bus 0002:00: resource 4 [mem 0x08000000-0x0fffffff window]
[    4.700362] pci_bus 0002:00: resource 5 [mem 0x200000000000-0x23ffdfffffff window]
[    4.707920] pci_bus 0002:01: resource 1 [mem 0x08000000-0x081fffff]
[    4.714176] pci_bus 0002:01: resource 2 [mem 0x200000000000-0x2000001fffff 64bit pref]
[    4.722081] pci_bus 0002:02: resource 1 [mem 0x08200000-0x083fffff]
[    4.728337] pci_bus 0002:02: resource 2 [mem 0x200000200000-0x2000003fffff 64bit pref]
[    4.736350] ACPI: PCI Root Bridge [PCI6] (domain 0004 [bus 00-ff])
[    4.742525] acpi PNP0A08:06: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    4.751612] acpi PNP0A08:06: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME LTR]
[    4.760301] acpi PNP0A08:06: _OSC: OS now controls [AER PCIeCapability]
[    4.766904] acpi PNP0A08:06: FADT indicates ASPM is unsupported, using BIOS configuration
[    4.775083] acpi PNP0A08:06: MCFG quirk: ECAM at [mem 0x2bfff0000000-0x2bffffffffff] for [bus 00-ff] with 0xffffb6de2d80ca60
[    4.787375] acpi PNP0A08:06: ECAM area [mem 0x2bfff0000000-0x2bffffffffff] reserved by PNP0C02:00
[    4.796248] acpi PNP0A08:06: ECAM at [mem 0x2bfff0000000-0x2bffffffffff] for [bus 00-ff]
[    4.804414] PCI host bridge to bus 0004:00
[    4.808501] pci_bus 0004:00: root bus resource [mem 0x20000000-0x2fffffff window]
[    4.815972] pci_bus 0004:00: root bus resource [mem 0x280000000000-0x2bffdfffffff window]
[    4.824138] pci_bus 0004:00: root bus resource [bus 00-ff]
[    4.829637] pci 0004:00:00.0: [1def:e110] type 00 class 0x060000
[    4.835692] pci 0004:00:01.0: [1def:e111] type 01 class 0x060400
[    4.841721] pci 0004:00:01.0: supports D1 D2
[    4.845980] pci 0004:00:01.0: PME# supported from D0 D1 D3hot
[    4.851788] pci 0004:00:03.0: [1def:e113] type 01 class 0x060400
[    4.857816] pci 0004:00:03.0: supports D1 D2
[    4.862075] pci 0004:00:03.0: PME# supported from D0 D1 D3hot
[    4.867880] pci 0004:00:05.0: [1def:e115] type 01 class 0x060400
[    4.873917] pci 0004:00:05.0: supports D1 D2
[    4.878176] pci 0004:00:05.0: PME# supported from D0 D1 D3hot
[    4.884027] pci 0004:01:00.0: [1a03:1150] type 01 class 0x060400
[    4.890071] pci 0004:01:00.0: enabling Extended Tags
[    4.895081] pci 0004:01:00.0: supports D1 D2
[    4.899340] pci 0004:01:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    4.906077] pci_bus 0004:02: extended config space not accessible
[    4.912182] pci 0004:02:00.0: [1a03:2000] type 00 class 0x030000
[    4.918194] pci 0004:02:00.0: reg 0x10: [mem 0x20000000-0x20ffffff]
[    4.924458] pci 0004:02:00.0: reg 0x14: [mem 0x21000000-0x2101ffff]
[    4.930722] pci 0004:02:00.0: reg 0x18: [io  0x2ffff000-0x2ffff07f]
[    4.937023] pci 0004:02:00.0: BAR 0: assigned to efifb
[    4.942185] pci 0004:02:00.0: supports D1 D2
[    4.946444] pci 0004:02:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    4.953183] pci 0004:03:00.0: [1912:0014] type 00 class 0x0c0330
[    4.959197] pci 0004:03:00.0: reg 0x10: [mem 0x21200000-0x21201fff 64bit]
[    4.966070] pci 0004:03:00.0: PME# supported from D0 D3hot D3cold
[    4.972327] pci 0004:04:00.0: working around ROM BAR overlap defect
[    4.978583] pci 0004:04:00.0: [8086:1533] type 00 class 0x020000
[    4.984605] pci 0004:04:00.0: reg 0x10: [mem 0x21100000-0x2117ffff]
[    4.990890] pci 0004:04:00.0: reg 0x18: [io  0x2fffe000-0x2fffe01f]
[    4.997160] pci 0004:04:00.0: reg 0x1c: [mem 0x21180000-0x21183fff]
[    5.003587] pci 0004:04:00.0: PME# supported from D0 D3hot D3cold
[    5.009884] pci_bus 0004:00: on NUMA node 0
[    5.014059] pci 0004:00:01.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 01-02] add_size 200000 add_align 100000
[    5.025785] pci 0004:00:03.0: bridge window [io  0x1000-0x0fff] to [bus 03] add_size 1000
[    5.033951] pci 0004:00:03.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 03] add_size 200000 add_align 100000
[    5.045417] pci 0004:00:03.0: bridge window [mem 0x00100000-0x001fffff] to [bus 03] add_size 100000 add_align 100000
[    5.055927] pci 0004:00:05.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 04] add_size 200000 add_align 100000
[    5.067392] pci 0004:00:05.0: bridge window [mem 0x00100000-0x001fffff] to [bus 04] add_size 100000 add_align 100000
[    5.077906] pci 0004:00:01.0: BAR 14: assigned [mem 0x20000000-0x217fffff]
[    5.084770] pci 0004:00:01.0: BAR 15: assigned [mem 0x280000000000-0x2800001fffff 64bit pref]
[    5.093283] pci 0004:00:03.0: BAR 14: assigned [mem 0x21800000-0x219fffff]
[    5.100147] pci 0004:00:03.0: BAR 15: assigned [mem 0x280000200000-0x2800003fffff 64bit pref]
[    5.108659] pci 0004:00:05.0: BAR 14: assigned [mem 0x21a00000-0x21bfffff]
[    5.115523] pci 0004:00:05.0: BAR 15: assigned [mem 0x280000400000-0x2800005fffff 64bit pref]
[    5.124035] pci 0004:00:01.0: BAR 13: no space for [io  size 0x1000]
[    5.130377] pci 0004:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    5.137067] pci 0004:00:03.0: BAR 13: no space for [io  size 0x1000]
[    5.143410] pci 0004:00:03.0: BAR 13: failed to assign [io  size 0x1000]
[    5.150100] pci 0004:00:05.0: BAR 13: no space for [io  size 0x1000]
[    5.156442] pci 0004:00:05.0: BAR 13: failed to assign [io  size 0x1000]
[    5.163134] pci 0004:00:01.0: BAR 13: no space for [io  size 0x1000]
[    5.169476] pci 0004:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    5.176165] pci 0004:00:05.0: BAR 13: no space for [io  size 0x1000]
[    5.182507] pci 0004:00:05.0: BAR 13: failed to assign [io  size 0x1000]
[    5.189197] pci 0004:00:03.0: BAR 13: no space for [io  size 0x1000]
[    5.195539] pci 0004:00:03.0: BAR 13: failed to assign [io  size 0x1000]
[    5.202229] pci 0004:01:00.0: BAR 14: assigned [mem 0x20000000-0x217fffff]
[    5.209093] pci 0004:01:00.0: BAR 13: no space for [io  size 0x1000]
[    5.215436] pci 0004:01:00.0: BAR 13: failed to assign [io  size 0x1000]
[    5.222126] pci 0004:02:00.0: BAR 0: assigned [mem 0x20000000-0x20ffffff]
[    5.228905] pci 0004:02:00.0: BAR 1: assigned [mem 0x21000000-0x2101ffff]
[    5.235684] pci 0004:02:00.0: BAR 2: no space for [io  size 0x0080]
[    5.241940] pci 0004:02:00.0: BAR 2: failed to assign [io  size 0x0080]
[    5.248543] pci 0004:01:00.0: PCI bridge to [bus 02]
[    5.253498] pci 0004:01:00.0:   bridge window [mem 0x20000000-0x217fffff]
[    5.260280] pci 0004:00:01.0: PCI bridge to [bus 01-02]
[    5.265494] pci 0004:00:01.0:   bridge window [mem 0x20000000-0x217fffff]
[    5.272271] pci 0004:00:01.0:   bridge window [mem 0x280000000000-0x2800001fffff 64bit pref]
[    5.280698] pci 0004:03:00.0: BAR 0: assigned [mem 0x21800000-0x21801fff 64bit]
[    5.288003] pci 0004:00:03.0: PCI bridge to [bus 03]
[    5.292958] pci 0004:00:03.0:   bridge window [mem 0x21800000-0x219fffff]
[    5.299734] pci 0004:00:03.0:   bridge window [mem 0x280000200000-0x2800003fffff 64bit pref]
[    5.308161] pci 0004:04:00.0: BAR 0: assigned [mem 0x21a00000-0x21a7ffff]
[    5.314942] pci 0004:04:00.0: BAR 3: assigned [mem 0x21a80000-0x21a83fff]
[    5.321722] pci 0004:04:00.0: BAR 2: no space for [io  size 0x0020]
[    5.327978] pci 0004:04:00.0: BAR 2: failed to assign [io  size 0x0020]
[    5.334581] pci 0004:00:05.0: PCI bridge to [bus 04]
[    5.339535] pci 0004:00:05.0:   bridge window [mem 0x21a00000-0x21bfffff]
[    5.346313] pci 0004:00:05.0:   bridge window [mem 0x280000400000-0x2800005fffff 64bit pref]
[    5.354740] pci_bus 0004:00: Some PCI device resources are unassigned, try booting with pci=realloc
[    5.363774] pci_bus 0004:00: resource 4 [mem 0x20000000-0x2fffffff window]
[    5.370637] pci_bus 0004:00: resource 5 [mem 0x280000000000-0x2bffdfffffff window]
[    5.378194] pci_bus 0004:01: resource 1 [mem 0x20000000-0x217fffff]
[    5.384450] pci_bus 0004:01: resource 2 [mem 0x280000000000-0x2800001fffff 64bit pref]
[    5.392354] pci_bus 0004:02: resource 1 [mem 0x20000000-0x217fffff]
[    5.398610] pci_bus 0004:03: resource 1 [mem 0x21800000-0x219fffff]
[    5.404866] pci_bus 0004:03: resource 2 [mem 0x280000200000-0x2800003fffff 64bit pref]
[    5.412771] pci_bus 0004:04: resource 1 [mem 0x21a00000-0x21bfffff]
[    5.419027] pci_bus 0004:04: resource 2 [mem 0x280000400000-0x2800005fffff 64bit pref]
[    5.426964] ACPI: PCI Root Bridge [PCI7] (domain 0005 [bus 00-ff])
[    5.433139] acpi PNP0A08:07: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[    5.442230] acpi PNP0A08:07: _OSC: platform does not support [PCIeHotplug SHPCHotplug PME LTR]
[    5.450920] acpi PNP0A08:07: _OSC: OS now controls [AER PCIeCapability]
[    5.457523] acpi PNP0A08:07: FADT indicates ASPM is unsupported, using BIOS configuration
[    5.465689] acpi PNP0A08:07: MCFG quirk: ECAM at [mem 0x2ffff0000000-0x2fffffffffff] for [bus 00-ff] with 0xffffb6de2d80ca60
[    5.477987] acpi PNP0A08:07: ECAM area [mem 0x2ffff0000000-0x2fffffffffff] reserved by PNP0C02:00
[    5.486858] acpi PNP0A08:07: ECAM at [mem 0x2ffff0000000-0x2fffffffffff] for [bus 00-ff]
[    5.495011] PCI host bridge to bus 0005:00
[    5.499097] pci_bus 0005:00: root bus resource [mem 0x30000000-0x3fffffff window]
[    5.506568] pci_bus 0005:00: root bus resource [mem 0x2c0000000000-0x2fffdfffffff window]
[    5.514733] pci_bus 0005:00: root bus resource [bus 00-ff]
[    5.520232] pci 0005:00:00.0: [1def:e110] type 00 class 0x060000
[    5.526286] pci 0005:00:01.0: [1def:e111] type 01 class 0x060400
[    5.532324] pci 0005:00:01.0: supports D1 D2
[    5.536583] pci 0005:00:01.0: PME# supported from D0 D1 D3hot
[    5.542403] pci 0005:00:03.0: [1def:e113] type 01 class 0x060400
[    5.548440] pci 0005:00:03.0: supports D1 D2
[    5.552699] pci 0005:00:03.0: PME# supported from D0 D1 D3hot
[    5.558516] pci 0005:00:05.0: [1def:e115] type 01 class 0x060400
[    5.564539] pci 0005:00:05.0: supports D1 D2
[    5.568798] pci 0005:00:05.0: PME# supported from D0 D1 D3hot
[    5.574597] pci 0005:00:07.0: [1def:e117] type 01 class 0x060400
[    5.580620] pci 0005:00:07.0: supports D1 D2
[    5.584879] pci 0005:00:07.0: PME# supported from D0 D1 D3hot
[    5.594908] pci 0005:03:00.0: [c0a9:540a] type 00 class 0x010802
[    5.600954] pci 0005:03:00.0: reg 0x10: [mem 0x30100000-0x30103fff 64bit]
[    5.608532] pci 0005:04:00.0: [126f:2263] type 00 class 0x010802
[    5.614544] pci 0005:04:00.0: reg 0x10: [mem 0x30000000-0x30003fff 64bit]
[    5.621545] pci_bus 0005:00: on NUMA node 0
[    5.625719] pci 0005:00:01.0: bridge window [io  0x1000-0x0fff] to [bus 01] add_size 1000
[    5.633885] pci 0005:00:01.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 01] add_size 200000 add_align 100000
[    5.645350] pci 0005:00:01.0: bridge window [mem 0x00100000-0x000fffff] to [bus 01] add_size 200000 add_align 100000
[    5.655860] pci 0005:00:03.0: bridge window [io  0x1000-0x0fff] to [bus 02] add_size 1000
[    5.664025] pci 0005:00:03.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 02] add_size 200000 add_align 100000
[    5.675490] pci 0005:00:03.0: bridge window [mem 0x00100000-0x000fffff] to [bus 02] add_size 200000 add_align 100000
[    5.686000] pci 0005:00:05.0: bridge window [io  0x1000-0x0fff] to [bus 03] add_size 1000
[    5.694166] pci 0005:00:05.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 03] add_size 200000 add_align 100000
[    5.705631] pci 0005:00:05.0: bridge window [mem 0x00100000-0x001fffff] to [bus 03] add_size 100000 add_align 100000
[    5.716140] pci 0005:00:07.0: bridge window [io  0x1000-0x0fff] to [bus 04] add_size 1000
[    5.724307] pci 0005:00:07.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 04] add_size 200000 add_align 100000
[    5.735772] pci 0005:00:07.0: bridge window [mem 0x00100000-0x001fffff] to [bus 04] add_size 100000 add_align 100000
[    5.746287] pci 0005:00:01.0: BAR 14: assigned [mem 0x30000000-0x301fffff]
[    5.753151] pci 0005:00:01.0: BAR 15: assigned [mem 0x2c0000000000-0x2c00001fffff 64bit pref]
[    5.761664] pci 0005:00:03.0: BAR 14: assigned [mem 0x30200000-0x303fffff]
[    5.768540] pci 0005:00:03.0: BAR 15: assigned [mem 0x2c0000200000-0x2c00003fffff 64bit pref]
[    5.777053] pci 0005:00:05.0: BAR 14: assigned [mem 0x30400000-0x305fffff]
[    5.783917] pci 0005:00:05.0: BAR 15: assigned [mem 0x2c0000400000-0x2c00005fffff 64bit pref]
[    5.792430] pci 0005:00:07.0: BAR 14: assigned [mem 0x30600000-0x307fffff]
[    5.799293] pci 0005:00:07.0: BAR 15: assigned [mem 0x2c0000600000-0x2c00007fffff 64bit pref]
[    5.807807] pci 0005:00:01.0: BAR 13: no space for [io  size 0x1000]
[    5.814149] pci 0005:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    5.820839] pci 0005:00:03.0: BAR 13: no space for [io  size 0x1000]
[    5.827189] pci 0005:00:03.0: BAR 13: failed to assign [io  size 0x1000]
[    5.833878] pci 0005:00:05.0: BAR 13: no space for [io  size 0x1000]
[    5.840220] pci 0005:00:05.0: BAR 13: failed to assign [io  size 0x1000]
[    5.846910] pci 0005:00:07.0: BAR 13: no space for [io  size 0x1000]
[    5.853252] pci 0005:00:07.0: BAR 13: failed to assign [io  size 0x1000]
[    5.859946] pci 0005:00:07.0: BAR 13: no space for [io  size 0x1000]
[    5.866289] pci 0005:00:07.0: BAR 13: failed to assign [io  size 0x1000]
[    5.872978] pci 0005:00:05.0: BAR 13: no space for [io  size 0x1000]
[    5.879320] pci 0005:00:05.0: BAR 13: failed to assign [io  size 0x1000]
[    5.886010] pci 0005:00:03.0: BAR 13: no space for [io  size 0x1000]
[    5.892352] pci 0005:00:03.0: BAR 13: failed to assign [io  size 0x1000]
[    5.899042] pci 0005:00:01.0: BAR 13: no space for [io  size 0x1000]
[    5.905384] pci 0005:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[    5.912074] pci 0005:00:01.0: PCI bridge to [bus 01]
[    5.917029] pci 0005:00:01.0:   bridge window [mem 0x30000000-0x301fffff]
[    5.923806] pci 0005:00:01.0:   bridge window [mem 0x2c0000000000-0x2c00001fffff 64bit pref]
[    5.932234] pci 0005:00:03.0: PCI bridge to [bus 02]
[    5.937187] pci 0005:00:03.0:   bridge window [mem 0x30200000-0x303fffff]
[    5.943964] pci 0005:00:03.0:   bridge window [mem 0x2c0000200000-0x2c00003fffff 64bit pref]
[    5.952392] pci 0005:03:00.0: BAR 0: assigned [mem 0x30400000-0x30403fff 64bit]
[    5.959708] pci 0005:00:05.0: PCI bridge to [bus 03]
[    5.964663] pci 0005:00:05.0:   bridge window [mem 0x30400000-0x305fffff]
[    5.971440] pci 0005:00:05.0:   bridge window [mem 0x2c0000400000-0x2c00005fffff 64bit pref]
[    5.979867] pci 0005:04:00.0: BAR 0: assigned [mem 0x30600000-0x30603fff 64bit]
[    5.987171] pci 0005:00:07.0: PCI bridge to [bus 04]
[    5.992125] pci 0005:00:07.0:   bridge window [mem 0x30600000-0x307fffff]
[    5.998902] pci 0005:00:07.0:   bridge window [mem 0x2c0000600000-0x2c00007fffff 64bit pref]
[    6.007328] pci_bus 0005:00: resource 4 [mem 0x30000000-0x3fffffff window]
[    6.014192] pci_bus 0005:00: resource 5 [mem 0x2c0000000000-0x2fffdfffffff window]
[    6.021749] pci_bus 0005:01: resource 1 [mem 0x30000000-0x301fffff]
[    6.028004] pci_bus 0005:01: resource 2 [mem 0x2c0000000000-0x2c00001fffff 64bit pref]
[    6.035909] pci_bus 0005:02: resource 1 [mem 0x30200000-0x303fffff]
[    6.042165] pci_bus 0005:02: resource 2 [mem 0x2c0000200000-0x2c00003fffff 64bit pref]
[    6.050070] pci_bus 0005:03: resource 1 [mem 0x30400000-0x305fffff]
[    6.056326] pci_bus 0005:03: resource 2 [mem 0x2c0000400000-0x2c00005fffff 64bit pref]
[    6.064231] pci_bus 0005:04: resource 1 [mem 0x30600000-0x307fffff]
[    6.070486] pci_bus 0005:04: resource 2 [mem 0x2c0000600000-0x2c00007fffff 64bit pref]
[    6.078800] iommu: Default domain type: Translated
[    6.083583] iommu: DMA domain TLB invalidation policy: strict mode
[    6.089857] pps_core: LinuxPPS API ver. 1 registered
[    6.094812] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    6.103934] PTP clock support registered
[    6.107850] EDAC MC: Ver: 3.0.0
[    6.111172] scmi_core: SCMI protocol bus registered
[    6.116119] efivars: Registered efivars operations
[    6.121386] NetLabel: Initializing
[    6.124777] NetLabel:  domain hash size = 128
[    6.129127] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
[    6.134794] NetLabel:  unlabeled traffic allowed by default
[    6.140403] pci 000d:03:00.0: vgaarb: setting as boot VGA device
[    6.146397] pci 000d:03:00.0: vgaarb: bridge control possible
[    6.152131] pci 000d:03:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[    6.160485] pci 0004:02:00.0: vgaarb: setting as boot VGA device (overriding previous)
[    6.168389] pci 0004:02:00.0: vgaarb: bridge control possible
[    6.174122] pci 0004:02:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[    6.182464] vgaarb: loaded
[    6.185316] clocksource: Switched to clocksource arch_sys_counter
[    6.191564] VFS: Disk quotas dquot_6.6.0
[    6.195506] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    6.205357] pnp: PnP ACPI init
[    6.209440] system 00:00: [mem 0x3bfff0000000-0x3bffffffffff window] has been reserved
[    6.217352] system 00:00: [mem 0x3ffff0000000-0x3fffffffffff window] could not be reserved
[    6.225606] system 00:00: [mem 0x23fff0000000-0x23ffffffffff window] could not be reserved
[    6.233860] system 00:00: [mem 0x27fff0000000-0x27ffffffffff window] has been reserved
[    6.241766] system 00:00: [mem 0x2bfff0000000-0x2bffffffffff window] could not be reserved
[    6.250019] system 00:00: [mem 0x2ffff0000000-0x2fffffffffff window] could not be reserved
[    6.258273] system 00:00: [mem 0x7bfff0000000-0x7bffffffffff window] has been reserved
[    6.266182] system 00:00: [mem 0x7ffff0000000-0x7fffffffffff window] has been reserved
[    6.274088] system 00:00: [mem 0x63fff0000000-0x63ffffffffff window] has been reserved
[    6.281994] system 00:00: [mem 0x67fff0000000-0x67ffffffffff window] has been reserved
[    6.289900] system 00:00: [mem 0x6bfff0000000-0x6bffffffffff window] has been reserved
[    6.297807] system 00:00: [mem 0x6ffff0000000-0x6fffffffffff window] has been reserved
[    6.305714] system 00:00: [mem 0x33fff0000000-0x33ffffffffff window] could not be reserved
[    6.313967] system 00:00: [mem 0x37fff0000000-0x37ffffffffff window] could not be reserved
[    6.322227] pnp: PnP ACPI: found 1 devices
[    6.328225] NET: Registered PF_INET protocol family
[    6.333211] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    6.344905] tcp_listen_portaddr_hash hash table entries: 16384 (order: 6, 262144 bytes, linear)
[    6.353721] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    6.361534] TCP established hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    6.370621] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)
[    6.378899] TCP: Hash tables configured (established 262144 bind 65536)
[    6.385686] MPTCP token hash table entries: 32768 (order: 7, 786432 bytes, linear)
[    6.393372] UDP hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    6.400594] UDP-Lite hash table entries: 16384 (order: 7, 524288 bytes, linear)
[    6.408292] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    6.413951] NET: Registered PF_XDP protocol family
[    6.418779] pci 000d:03:00.1: D0 power state depends on 000d:03:00.0
[    6.425204] pci 0004:03:00.0: enabling device (0000 -> 0002)
[    6.430896] PCI: CLS 128 bytes, default 64
[    6.436167] kvm [1]: IPA Size Limit: 48 bits
[    6.440449] kvm [1]: GICv3: no GICV resource entry
[    6.445231] kvm [1]: disabling GICv2 emulation
[    6.449681] kvm [1]: GIC system register CPU interface enabled
[    6.455509] kvm [1]: vgic interrupt IRQ9
[    6.459447] kvm [1]: VHE mode initialized successfully
[    6.465288] Initialise system trusted keyrings
[    6.469738] Key type blacklist registered
[    6.473809] workingset: timestamp_bits=44 max_order=23 bucket_order=0
[    6.480253] zbud: loaded
[    6.483024] Key type asymmetric registered
[    6.487116] Asymmetric key parser 'x509' registered
[    6.493528] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 246)
[    6.500976] io scheduler mq-deadline registered
[    6.509169] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    6.516246] efifb: probing for efifb
[    6.519831] efifb: framebuffer at 0x20000000, using 3072k, total 3072k
[    6.526351] efifb: mode is 1024x768x32, linelength=4096, pages=1
[    6.532346] efifb: scrolling: redraw
[    6.535912] efifb: Truecolor: size=8:8:8:8, shift=24:16:8:0
[    6.551092] Console: switching to colour frame buffer device 128x48
[    6.566861] fb0: EFI VGA frame buffer device
[    6.571668] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0
[    6.580016] ACPI: button: Power Button [PWRB]
[    6.594167] ACPI GTDT: found 1 SBSA generic Watchdog(s).
[    6.600850] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    6.607746] Serial: AMBA driver
[    6.610889] SuperH (H)SCI(F) driver initialized
[    6.708340] arm-smmu-v3 arm-smmu-v3.0.auto: option mask 0x0
[    6.713920] arm-smmu-v3 arm-smmu-v3.0.auto: ias 48-bit, oas 48-bit (features 0x000c1fff)
[    6.726270] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 65536 entries for cmdq
[    6.744073] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 32768 entries for evtq
[    6.751299] arm-smmu-v3 arm-smmu-v3.0.auto: allocated 65536 entries for priq
[    6.759034] arm-smmu-v3 arm-smmu-v3.1.auto: option mask 0x0
[    6.764616] arm-smmu-v3 arm-smmu-v3.1.auto: ias 48-bit, oas 48-bit (features 0x000c1fff)
[    6.772883] arm-smmu-v3 arm-smmu-v3.1.auto: allocated 65536 entries for cmdq
[    6.780021] arm-smmu-v3 arm-smmu-v3.1.auto: allocated 32768 entries for evtq
[    6.787217] arm-smmu-v3 arm-smmu-v3.1.auto: allocated 65536 entries for priq
[    6.794995] arm-smmu-v3 arm-smmu-v3.2.auto: option mask 0x0
[    6.800576] arm-smmu-v3 arm-smmu-v3.2.auto: ias 48-bit, oas 48-bit (features 0x000c1fff)
[    6.808750] arm-smmu-v3 arm-smmu-v3.2.auto: allocated 65536 entries for cmdq
[    6.815961] arm-smmu-v3 arm-smmu-v3.2.auto: allocated 32768 entries for evtq
[    6.823096] arm-smmu-v3 arm-smmu-v3.2.auto: allocated 65536 entries for priq
[    6.830840] arm-smmu-v3 arm-smmu-v3.3.auto: option mask 0x0
[    6.836420] arm-smmu-v3 arm-smmu-v3.3.auto: ias 48-bit, oas 48-bit (features 0x000c1fff)
[    6.844592] arm-smmu-v3 arm-smmu-v3.3.auto: allocated 65536 entries for cmdq
[    6.851797] arm-smmu-v3 arm-smmu-v3.3.auto: allocated 32768 entries for evtq
[    6.858944] arm-smmu-v3 arm-smmu-v3.3.auto: allocated 65536 entries for priq
[    6.866614] arm-smmu-v3 arm-smmu-v3.4.auto: option mask 0x0
[    6.872193] arm-smmu-v3 arm-smmu-v3.4.auto: ias 48-bit, oas 48-bit (features 0x000c1fff)
[    6.880439] arm-smmu-v3 arm-smmu-v3.4.auto: allocated 65536 entries for cmdq
[    6.887566] arm-smmu-v3 arm-smmu-v3.4.auto: allocated 32768 entries for evtq
[    6.894770] arm-smmu-v3 arm-smmu-v3.4.auto: allocated 65536 entries for priq
[    6.902586] arm-smmu-v3 arm-smmu-v3.5.auto: option mask 0x0
[    6.908165] arm-smmu-v3 arm-smmu-v3.5.auto: ias 48-bit, oas 48-bit (features 0x000c1fff)
[    6.916411] arm-smmu-v3 arm-smmu-v3.5.auto: allocated 65536 entries for cmdq
[    6.923551] arm-smmu-v3 arm-smmu-v3.5.auto: allocated 32768 entries for evtq
[    6.930755] arm-smmu-v3 arm-smmu-v3.5.auto: allocated 65536 entries for priq
[    6.942032] nvme 0005:03:00.0: Adding to iommu group 0
[    6.945409] mousedev: PS/2 mouse device common for all mice
[    6.952808] nvme 0005:04:00.0: Adding to iommu group 1
[    6.953268] nvme nvme0: pci function 0005:03:00.0
[    6.963075] rtc-efi rtc-efi.0: registered as rtc0
[    6.967860] nvme nvme1: pci function 0005:04:00.0
[    6.972636] rtc-efi rtc-efi.0: setting system clock to 2023-10-26T22:20:37 UTC (1698358837)
[    6.981131] nvme nvme0: missing or invalid SUBNQN field.
[    6.988737] ledtrig-cpu: registered to indicate activity on CPUs
[    6.988800] nvme nvme1: missing or invalid SUBNQN field.
[    6.995073] SMCCC: SOC_ID: ID = jep106:0a16:0001 Revision = 0x000000a1
[    7.007511] hw perfevents: enabled with armv8_pmuv3_0 PMU driver, 7 counters available
[    7.016022] NET: Registered PF_INET6 protocol family
[    7.022985] Segment Routing with IPv6
[    7.026655] In-situ OAM (IOAM) with IPv6
[    7.028074] nvme nvme1: 15/0/0 default/read/poll queues
[    7.030593] mip6: Mobile IPv6
[    7.038745] NET: Registered PF_PACKET protocol family
[    7.039258] nvme nvme1: Ignoring bogus Namespace Identifiers
[    7.049490] mpls_gso: MPLS GSO support
[    7.051220]  nvme1n1: p1 p2 p3
[    7.054787] registered taskstats version 1
[    7.060569] Loading compiled-in X.509 certificates
[    7.068968] Key type .fscrypt registered
[    7.072885] Key type fscrypt-provisioning registered
[    7.078833] Key type encrypted registered
[    7.111406] nvme nvme0: allocated 32 MiB host memory buffer.
[    7.150480] nvme nvme0: 8/0/0 default/read/poll queues
[    7.160945] nvme nvme0: Ignoring bogus Namespace Identifiers
[    7.176903]  nvme0n1: p1 p2 p3
[    7.259235] pcieport 000c:00:01.0: Adding to iommu group 2
[    7.265082] pcieport 000c:00:01.0: AER: enabled with IRQ 106
[    7.270915] pcieport 000d:00:01.0: Adding to iommu group 3
[    7.276678] pcieport 000d:00:01.0: AER: enabled with IRQ 107
[    7.282463] pcieport 000d:01:00.0: Adding to iommu group 4
[    7.288233] pcieport 000d:02:00.0: Adding to iommu group 5
[    7.294073] pcieport 0000:00:01.0: Adding to iommu group 6
[    7.299868] pcieport 0000:00:01.0: AER: enabled with IRQ 109
[    7.305688] pcieport 0002:00:01.0: Adding to iommu group 7
[    7.311479] pcieport 0002:00:01.0: AER: enabled with IRQ 110
[    7.317249] pcieport 0002:00:05.0: Adding to iommu group 8
[    7.322998] pcieport 0002:00:05.0: AER: enabled with IRQ 110
[    7.328847] pcieport 0004:00:01.0: Adding to iommu group 9
[    7.334649] pcieport 0004:00:01.0: AER: enabled with IRQ 111
[    7.340416] pcieport 0004:00:03.0: Adding to iommu group 10
[    7.346197] pcieport 0004:00:03.0: AER: enabled with IRQ 111
[    7.351948] pcieport 0004:00:05.0: Adding to iommu group 11
[    7.357740] pcieport 0004:00:05.0: AER: enabled with IRQ 111
[    7.363572] pcieport 0004:01:00.0: Adding to iommu group 12
[    7.369424] pcieport 0005:00:01.0: Adding to iommu group 13
[    7.375288] pcieport 0005:00:01.0: AER: enabled with IRQ 79
[    7.380968] pcieport 0005:00:03.0: Adding to iommu group 14
[    7.386806] pcieport 0005:00:03.0: AER: enabled with IRQ 79
[    7.392484] pcieport 0005:00:05.0: Adding to iommu group 15
[    7.398293] pcieport 0005:00:05.0: AER: enabled with IRQ 79
[    7.403967] pcieport 0005:00:07.0: Adding to iommu group 16
[    7.409881] pcieport 0005:00:07.0: AER: enabled with IRQ 79
[    7.415662] clk: Disabling unused clocks
[    7.428776] EXT4-fs (nvme1n1p2): mounted filesystem aa19fa06-115b-412d-8bc2-f0715936b9a5 ro with ordered data mode. Quota mode: none.
[    7.440795] VFS: Mounted root (ext4 filesystem) readonly on device 259:2.
[    7.449391] Freeing unused kernel memory: 5568K
[    7.560459] Checked W+X mappings: passed, no W+X pages found
[    7.566120] Run /sbin/init as init process
[    7.570208]   with arguments:
[    7.573163]     /sbin/init
[    7.575860]   with environment:
[    7.578990]     HOME=/
[    7.581339]     TERM=linux
[    7.584034]     BOOT_IMAGE=/boot/vmlinuz-6.6-rc7+
[    7.661527] autofs4: module verification failed: signature and/or required key missing - tainting kernel
[    7.672167] systemd[1]: Inserted module 'autofs4'
[    7.700810] systemd[1]: systemd 254.5-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[    7.732699] systemd[1]: Detected architecture arm64.
[    7.754009] systemd[1]: Hostname set to <avadebian>.
[    7.940613] systemd[1]: Queued start job for default target graphical.target.
[    7.971067] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[    7.994161] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[    8.018177] systemd[1]: Created slice system-serial\x2dgetty.slice - Slice /system/serial-getty.
[    8.046156] systemd[1]: Created slice user.slice - User and Session Slice.
[    8.069659] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[    8.097851] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[    8.129610] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[    8.157562] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[    8.181453] systemd[1]: Reached target slices.target - Slice Units.
[    8.205539] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[    8.233694] systemd[1]: Listening on dm-event.socket - Device-mapper event daemon FIFOs.
[    8.257741] systemd[1]: Listening on lvm2-lvmpolld.socket - LVM2 poll daemon socket.
[    8.281713] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[    8.309618] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[    8.337784] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[    8.365797] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[    8.390215] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[    8.417763] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[    8.465611] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[    8.490925] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[    8.514962] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[    8.542931] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[    8.569779] systemd[1]: Finished blk-availability.service - Availability of block devices.
[    8.621750] systemd[1]: Starting keyboard-setup.service - Set the console keyboard layout...
[    8.646857] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[    8.674665] systemd[1]: Starting lvm2-monitor.service - Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
[    8.702603] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[    8.726599] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[    8.736301] device-mapper: uevent: version 1.0.3
[    8.743782] device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: dm-devel@redhat.com
[    8.766996] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[    8.790802] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[    8.805193] pstore: Using crash dump compression: deflate
[    8.805410] ACPI: bus type drm_connector registered
[    8.815002] pstore: Registered efi_pstore as persistent store backend
[    8.826707] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[    8.844294] fuse: init (API version 7.39)
[    8.850717] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[    8.868979] loop: module loaded
[    8.876018] systemd[1]: Starting systemd-fsck-root.service - File System Check on Root Device...
[    8.904003] systemd[1]: Starting systemd-journald.service - Journal Service...
[    8.913041] systemd[1]: Starting systemd-modules-load.service - Load Kernel Modules...
[    8.940827] systemd-journald[433]: Collecting audit messages is disabled.
[    8.949396] systemd[1]: systemd-pcrmachine.service - TPM2 PCR Machine ID Measurement was skipped because of an unmet condition check (ConditionPathExists=/sys/firmware/efi/efivars/StubPcrKernelImage-4a67b082-0a4c-41cf-b6c7-440b29bb8c4f).
[    8.971824] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[    8.999415] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[    9.021484] systemd[1]: Started systemd-journald.service - Journal Service.
[    9.508468] EXT4-fs (nvme1n1p2): re-mounted aa19fa06-115b-412d-8bc2-f0715936b9a5 r/w. Quota mode: none.
[    9.669361] systemd-journald[433]: Received client request to flush runtime journal.
[    9.694723] systemd-journald[433]: /var/log/journal/698d8624b408416f8010372e8e433376/system.journal: Journal file uses a different sequence number ID, rotating.
[    9.710745] systemd-journald[433]: Rotating system journal.
[   10.407002] CPPC Cpufreq:FIE not enabled on systems with registers in PCC
[   10.491815] xgene-slimpro-hwmon APMC0D29:00: APM X-Gene SoC HW monitor driver registered
[   10.547231] ACPI: bus type USB registered
[   10.548957] igb: Intel(R) Gigabit Ethernet Network Driver
[   10.556628] igb: Copyright (c) 2007-2014 Intel Corporation.
[   10.562362] usbcore: registered new interface driver usbfs
[   10.570603] usbcore: registered new interface driver hub
[   10.570603] igb 0004:04:00.0: Adding to iommu group 17
[   10.581967] usbcore: registered new device driver usb
[   10.589722] IPMI message handler: version 39.2
[   10.590953] arm_spe_pmu arm,spe-v1: profiling buffer inaccessible. Try passing "kpti=off" on the kernel command line
[   10.597579] sbsa-gwdt sbsa-gwdt.0: Initialized with 10s timeout @ 25000000 Hz, action=0.
[   10.604692] arm_spe_pmu: probe of arm,spe-v1 failed with error -1
[   10.625711] ipmi device interface
[   10.629040] pps pps0: new PPS source ptp0
[   10.633323] igb 0004:04:00.0: added PHC on eth0
[   10.633814] ipmi_ssif: IPMI SSIF Interface driver
[   10.640567] igb 0004:04:00.0: Intel(R) Gigabit Ethernet Network Connection
[   10.657718] igb 0004:04:00.0: eth0: (PCIe:2.5Gb/s:Width x1) 00:30:64:73:32:54
[   10.665670] igb 0004:04:00.0: eth0: PBA No: 000300-000
[   10.665675] igb 0004:04:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[   10.665843] xhci_hcd 0004:03:00.0: Adding to iommu group 18
[   10.669684] xhci_hcd 0004:03:00.0: xHCI Host Controller
[   10.669694] xhci_hcd 0004:03:00.0: new USB bus registered, assigned bus number 1
[   10.669756] xhci_hcd 0004:03:00.0: Zeroing 64bit base registers, expecting fault
[   10.682977] igb 0004:04:00.0 enP4p4s0: renamed from eth0
[   10.760895] snd_hda_intel 000d:03:00.1: Adding to iommu group 19
[   10.814659] Adding 1000444k swap on /dev/nvme1n1p3.  Priority:-2 extents:1 across:1000444k SS
[   10.871350] xhci_hcd 0004:03:00.0: hcc params 0x014051cf hci version 0x100 quirks 0x0000001100000410
[   10.880997] xhci_hcd 0004:03:00.0: xHCI Host Controller
[   10.886223] xhci_hcd 0004:03:00.0: new USB bus registered, assigned bus number 2
[   10.886228] xhci_hcd 0004:03:00.0: Host supports USB 3.0 SuperSpeed
[   10.899937] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.06
[   10.908198] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   10.915412] usb usb1: Product: xHCI Host Controller
[   10.920282] usb usb1: Manufacturer: Linux 6.6.0-rc7+ xhci-hcd
[   10.920284] usb usb1: SerialNumber: 0004:03:00.0
[   10.920420] snd_hda_intel 000d:03:00.1: enabling device (0000 -> 0002)
[   10.944267] hub 1-0:1.0: USB hub found
[   10.950795] hub 1-0:1.0: 4 ports detected
[   10.950984] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[   10.968783] snd_hda_intel 000d:03:00.1: Force to snoop mode by module option
[   10.978432] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.06
[   10.989442] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   10.989444] usb usb2: Product: xHCI Host Controller
[   10.989446] usb usb2: Manufacturer: Linux 6.6.0-rc7+ xhci-hcd
[   10.989447] usb usb2: SerialNumber: 0004:03:00.0
[   10.989757] hub 2-0:1.0: USB hub found
[   11.029768] hub 2-0:1.0: 4 ports detected
[   11.047714] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci000d:00/000d:00:01.0/000d:01:00.0/000d:02:00.0/000d:03:00.1/sound/card0/input1
[   11.062383] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci000d:00/000d:00:01.0/000d:01:00.0/000d:02:00.0/000d:03:00.1/sound/card0/input2
[   11.077786] input: HDA ATI HDMI HDMI/DP,pcm=8 as /devices/pci000d:00/000d:00:01.0/000d:01:00.0/000d:02:00.0/000d:03:00.1/sound/card0/input3
[   11.093238] input: HDA ATI HDMI HDMI/DP,pcm=9 as /devices/pci000d:00/000d:00:01.0/000d:01:00.0/000d:02:00.0/000d:03:00.1/sound/card0/input4
[   11.214181] alsactl[747]: memfd_create() called without MFD_EXEC or MFD_NOEXEC_SEAL set
[   11.237326] usb 1-3: new full-speed USB device number 2 using xhci_hcd
[   11.315454] [drm] amdgpu kernel modesetting enabled.
[   11.322523] amdgpu 000d:03:00.0: Adding to iommu group 20
[   11.330995] amdgpu 000d:03:00.0: enabling device (0000 -> 0002)
[   11.339091] [drm] initializing kernel modesetting (IP DISCOVERY 0x1002:0x7480 0x1DA2:0xE448 0xCF).
[   11.350830] [drm] register mmio base: 0x50000000
[   11.350832] [drm] register mmio size: 1048576
[   11.368995] [drm] add ip block number 0 <soc21_common>
[   11.376420] [drm] add ip block number 1 <gmc_v11_0>
[   11.384066] [drm] add ip block number 2 <ih_v6_0>
[   11.391545] [drm] add ip block number 3 <psp>
[   11.398670] [drm] add ip block number 4 <smu>
[   11.405791] [drm] add ip block number 5 <dm>
[   11.412818] [drm] add ip block number 6 <gfx_v11_0>
[   11.420461] [drm] add ip block number 7 <sdma_v6_0>
[   11.428119] [drm] add ip block number 8 <vcn_v4_0>
[   11.435672] [drm] add ip block number 9 <jpeg_v4_0>
[   11.443328] [drm] add ip block number 10 <mes_v11_0>
[   11.455074] [drm] BIOS signature incorrect ee a3
[   11.470500] amdgpu 000d:03:00.0: amdgpu: Fetched VBIOS from ROM BAR
[   11.478575] amdgpu: ATOM BIOS: 113-4481LHS-UC2
[   11.485781] usb 1-3: New USB device found, idVendor=062a, idProduct=4101, bcdDevice= 1.03
[   11.496794] usb 1-3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   11.506671] usb 1-3: Product: 2.4G Keyboard Mouse
[   11.514130] usb 1-3: Manufacturer: MOSART Semi.
[   11.578863] amdgpu 000d:03:00.0: amdgpu: CP RS64 enable
[   11.587185] usb 2-4: new SuperSpeed USB device number 2 using xhci_hcd
[   11.617860] [drm] VCN(0) encode/decode are enabled in VM mode
[   11.629662] usb 2-4: New USB device found, idVendor=0424, idProduct=7206, bcdDevice= 6.25
[   11.640138] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   11.650426] amdgpu 000d:03:00.0: [drm:jpeg_v4_0_early_init [amdgpu]] JPEG decode is enabled in VM mode
[   11.662104] usb 2-4: Product: USB7206 Smart Hub
[   11.669456] usb 2-4: Manufacturer: Microchip
[   11.676446] amdgpu 000d:03:00.0: Direct firmware load for amdgpu/gc_11_0_2_mes_2.bin failed with error -2
[   11.688854] [drm] try to fall back to amdgpu/gc_11_0_2_mes.bin
[   11.700698] hub 2-4:1.0: USB hub found
[   11.707041] hub 2-4:1.0: 3 ports detected
[   11.755104] amdgpu 000d:03:00.0: amdgpu: Trusted Memory Zone (TMZ) feature not supported
[   11.763232] usb 1-4: new high-speed USB device number 3 using xhci_hcd
[   11.769765] amdgpu 000d:03:00.0: amdgpu: PCIE atomic ops is not supported
[   11.776567] [drm] GPU posting now...
[   11.780204] [drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
[   11.788764] amdgpu 000d:03:00.0: BAR 2: releasing [mem 0x340010000000-0x3400101fffff 64bit pref]
[   11.789392] NET: Registered PF_QIPCRTR protocol family
[   11.797545] amdgpu 000d:03:00.0: BAR 0: releasing [mem 0x340000000000-0x34000fffffff 64bit pref]
[   11.811476] pcieport 000d:02:00.0: BAR 15: releasing [mem 0x340000000000-0x340017ffffff 64bit pref]
[   11.820515] pcieport 000d:01:00.0: BAR 15: releasing [mem 0x340000000000-0x340017ffffff 64bit pref]
[   11.829553] pcieport 000d:00:01.0: BAR 15: releasing [mem 0x340000000000-0x340017ffffff 64bit pref]
[   11.838593] pcieport 000d:00:01.0: bridge window [io  0x1000-0x0fff] to [bus 01-03] add_size 1000
[   11.847464] pcieport 000d:00:01.0: BAR 15: assigned [mem 0x340000000000-0x3402ffffffff 64bit pref]
[   11.856433] pcieport 000d:00:01.0: BAR 13: no space for [io  size 0x1000]
[   11.863215] pcieport 000d:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[   11.870345] pcieport 000d:00:01.0: BAR 13: no space for [io  size 0x1000]
[   11.877124] pcieport 000d:00:01.0: BAR 13: failed to assign [io  size 0x1000]
[   11.884252] pcieport 000d:01:00.0: BAR 15: assigned [mem 0x340000000000-0x3402ffffffff 64bit pref]
[   11.893203] pcieport 000d:02:00.0: BAR 15: assigned [mem 0x340000000000-0x3402ffffffff 64bit pref]
[   11.902153] amdgpu 000d:03:00.0: BAR 0: assigned [mem 0x340000000000-0x3401ffffffff 64bit pref]
[   11.910854] amdgpu 000d:03:00.0: BAR 2: assigned [mem 0x340200000000-0x3402001fffff 64bit pref]
[   11.919561] pcieport 000d:00:01.0: PCI bridge to [bus 01-03]
[   11.925221] pcieport 000d:00:01.0:   bridge window [mem 0x50000000-0x502fffff]
[   11.932445] pcieport 000d:00:01.0:   bridge window [mem 0x340000000000-0x3402ffffffff 64bit pref]
[   11.941310] pcieport 000d:01:00.0: PCI bridge to [bus 02-03]
[   11.946968] pcieport 000d:01:00.0:   bridge window [mem 0x50000000-0x501fffff]
[   11.954194] pcieport 000d:01:00.0:   bridge window [mem 0x340000000000-0x3402ffffffff 64bit pref]
[   11.963078] pcieport 000d:02:00.0: PCI bridge to [bus 03]
[   11.968485] pcieport 000d:02:00.0:   bridge window [mem 0x50000000-0x501fffff]
[   11.975717] pcieport 000d:02:00.0:   bridge window [mem 0x340000000000-0x3402ffffffff 64bit pref]
[   11.984601] amdgpu 000d:03:00.0: amdgpu: VRAM: 8176M 0x0000008000000000 - 0x00000081FEFFFFFF (8176M used)
[   11.994173] amdgpu 000d:03:00.0: amdgpu: GART: 512M 0x0000000000000000 - 0x000000001FFFFFFF
[   12.002535] amdgpu 000d:03:00.0: amdgpu: AGP: 267894784M 0x0000008400000000 - 0x0000FFFFFFFFFFFF
[   12.011338] [drm] Detected VRAM RAM=8176M, BAR=8192M
[   12.016299] usb 1-4: New USB device found, idVendor=0424, idProduct=4206, bcdDevice= 6.25
[   12.024469] [drm] RAM width 128bits GDDR6
[   12.028526] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   12.035707] [drm] amdgpu: 8176M of VRAM memory ready
[   12.040664] usb 1-4: Product: USB4206 Smart Hub
[   12.045188] [drm] amdgpu: 15892M of GTT memory ready.
[   12.050233] usb 1-4: Manufacturer: Microchip
[   12.054576] [drm] GART: num cpu pages 131072, num gpu pages 131072
[   12.060913] [drm] PCIE GART of 512M enabled (table at 0x00000081FEB00000).
[   12.068591] [drm] Loading DMUB firmware via PSP: version=0x07000C00
[   12.075096] hub 1-4:1.0: USB hub found
[   12.079162] hub 1-4:1.0: 5 ports detected
[   12.084597] [drm] Found VCN firmware Version ENC: 1.9 DEC: 5 VEP: 0 Revision: 1
[   12.091962] amdgpu 000d:03:00.0: amdgpu: Will use PSP to load VCN firmware
[   12.119435] hid: raw HID events driver (C) Jiri Kosina
[   12.158773] [drm] reserve 0x1300000 from 0x81fc000000 for PSP TMR
[   12.166163] usbcore: registered new interface driver usbhid
[   12.171735] usbhid: USB HID core driver
[   12.183354] input: MOSART Semi. 2.4G Keyboard Mouse as /devices/pci0004:00/0004:00:03.0/0004:03:00.0/usb1/1-3/1-3:1.0/0003:062A:4101.0001/input/input5
[   12.253607] hid-generic 0003:062A:4101.0001: input,hidraw0: USB HID v1.10 Keyboard [MOSART Semi. 2.4G Keyboard Mouse] on usb-0004:03:00.0-3/input0
[   12.255296] amdgpu 000d:03:00.0: amdgpu: RAS: optional ras ta ucode is not available
[   12.266941] input: MOSART Semi. 2.4G Keyboard Mouse as /devices/pci0004:00/0004:00:03.0/0004:03:00.0/usb1/1-3/1-3:1.1/0003:062A:4101.0002/input/input6
[   12.281940] amdgpu 000d:03:00.0: amdgpu: RAP: optional rap ta ucode is not available
[   12.288070] input: MOSART Semi. 2.4G Keyboard Mouse Consumer Control as /devices/pci0004:00/0004:00:03.0/0004:03:00.0/usb1/1-3/1-3:1.1/0003:062A:4101.0002/input/input7
[   12.295661] amdgpu 000d:03:00.0: amdgpu: SECUREDISPLAY: securedisplay ta ucode is not available
[   12.338670] amdgpu 000d:03:00.0: amdgpu: SMU is initialized successfully!
[   12.345784] [drm] Display Core v3.2.247 initialized on DCN 3.2.1
[   12.351787] [drm] DP-HDMI FRL PCON supported
[   12.358074] [drm] DMUB hardware initialized: version=0x07000C00
[   12.367976] snd_hda_intel 000d:03:00.1: bound 000d:03:00.0 (ops amdgpu_dm_audio_component_bind_ops [amdgpu])
[   12.377461] input: MOSART Semi. 2.4G Keyboard Mouse System Control as /devices/pci0004:00/0004:00:03.0/0004:03:00.0/usb1/1-3/1-3:1.1/0003:062A:4101.0002/input/input8
[   12.393206] input: MOSART Semi. 2.4G Keyboard Mouse as /devices/pci0004:00/0004:00:03.0/0004:03:00.0/usb1/1-3/1-3:1.1/0003:062A:4101.0002/input/input9
[   12.406399] [drm] kiq ring mec 3 pipe 1 q 0
[   12.406799] hid-generic 0003:062A:4101.0002: input,hiddev0,hidraw1: USB HID v1.10 Mouse [MOSART Semi. 2.4G Keyboard Mouse] on usb-0004:03:00.0-3/input1
[   12.415493] [drm] VCN decode and encode initialized successfully(under DPG Mode).
[   12.432567] amdgpu 000d:03:00.0: [drm:jpeg_v4_0_hw_init [amdgpu]] JPEG decode initialized successfully.
[   12.442516] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:72 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.454080] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b0000 from client 10
[   12.457317] usb 1-4.4: new high-speed USB device number 4 using xhci_hcd
[   12.463548] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000890
[   12.463551] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CPF (0x4)
[   12.484914] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   12.490474] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   12.496121] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x9
[   12.502202] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   12.507934] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   12.512716] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:221 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.524355] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b1000 from client 10
[   12.533821] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x000009BB
[   12.541464] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CPF (0x4)
[   12.548499] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x1
[   12.554059] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x5
[   12.558700] [drm:mes_v11_0_submit_pkt_and_poll_completion.constprop.0 [amdgpu]] *ERROR* MES failed to response msg=0
[   12.559705] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0xb
[   12.570224] [drm:amdgpu_device_init [amdgpu]] *ERROR* hw_init of IP block <mes_v11_0> failed -110
[   12.576292] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x1
[   12.576294] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   12.576299] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:192 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.585170] amdgpu 000d:03:00.0: amdgpu: amdgpu_device_ip_init failed
[   12.590890] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b2000 from client 10
[   12.590892] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   12.590894] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   12.595675] amdgpu 000d:03:00.0: amdgpu: Fatal error during GPU init
[   12.607307] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   12.607309] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   12.607311] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   12.607312] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   12.613754] amdgpu 000d:03:00.0: amdgpu: amdgpu: finishing device.
[   12.623204] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   12.623210] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:221 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.631102] [drm] DSC precompute is not needed.
[   12.638059] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b3000 from client 10
[   12.638062] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   12.638063] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   12.638065] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   12.638066] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   12.644449] ------------[ cut here ]------------
[   12.649965] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   12.655611] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   12.661690] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   12.667423] Modules linked in:
[   12.673588] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   12.678366]  hid_generic(E)
[   12.690005] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:88 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.694518]  usbhid(E)
[   12.703982] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b4000 from client 10
[   12.711624]  hid(E)
[   12.718832] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   12.724391]  qrtr(E)
[   12.730036] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   12.734640]  iptable_nat(E)
[   12.740719] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   12.751226]  nf_nat(E)
[   12.756958] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   12.759999]  nf_conntrack(E)
[   12.764777] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   12.767558]  nf_defrag_ipv6(E)
[   12.779106] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   12.781453]  nf_defrag_ipv4(E)
[   12.790918] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   12.793004]  libcrc32c(E)
[   12.800650] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:221 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.802819]  iptable_mangle(E)
[   12.810027] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b5000 from client 10
[   12.812807]  amdgpu(E+)
[   12.818366] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   12.820713]  iptable_filter(E)
[   12.826358] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   12.829226]  snd_hda_codec_hdmi(E)
[   12.835305] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   12.838346]  snd_hda_intel(E)
[   12.844079] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   12.847120]  snd_intel_dspcfg(E)
[   12.851897] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   12.854505]  snd_hda_codec(E)
[   12.866139] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   12.869181]  aes_ce_blk(E)
[   12.878646] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   12.881080]  snd_hda_core(E)
[   12.888725] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:208 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.891763]  amdxcp(E)
[   12.898970] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b6000 from client 10
[   12.902359]  aes_ce_cipher(E)
[   12.907917] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   12.910872]  drm_exec(E)
[   12.916518] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   12.919733]  crct10dif_ce(E)
[   12.925812] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   12.928767]  gpu_sched(E)
[   12.934499] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   12.937193]  polyval_ce(E)
[   12.941970] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   12.944838]  snd_hwdep(E)
[   12.956473] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   12.958820]  drm_buddy(E)
[   12.968285] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   12.971240]  snd_pcm(E)
[   12.978885] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:221 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   12.981402]  polyval_generic(E)
[   12.988610] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b7000 from client 10
[   12.991478]  video(E)
[   12.997036] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   12.999643]  ghash_ce(E)
[   13.005288] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   13.007983]  drm_suballoc_helper(E)
[   13.014062] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   13.016669]  snd_timer(E)
[   13.022401] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   13.025008]  drm_ttm_helper(E)
[   13.029786] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   13.032220]  gf128mul(E)
[   13.043854] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   13.046982]  sha2_ce(E)
[   13.056447] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   13.058706]  ttm(E)
[   13.066351] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:72 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   13.068868]  snd(E)
[   13.076076] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b8000 from client 10
[   13.079552]  ipmi_ssif(E)
[   13.085110] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   13.087717]  xhci_pci(E)
[   13.093362] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   13.096404]  sha256_arm64(E)
[   13.102483] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   13.105003]  drm_display_helper(E)
[   13.110735] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   13.113168]  ipmi_devintf(E)
[   13.117945] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   13.120032]  soundcore(E)
[   13.131580] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   13.133666]  xhci_hcd(E)
[   13.143130] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   13.145737]  sha1_ce(E)
[   13.153382] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault (src_id:0 ring:221 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
[   13.155899]  drm_shmem_helper(E)
[   13.163107] amdgpu 000d:03:00.0: amdgpu:   in page starting at address 0x00000000044b9000 from client 10
[   13.165974]  sbsa_gwdt(E)
[   13.171533] amdgpu 000d:03:00.0: amdgpu: GCVM_L2_PROTECTION_FAULT_STATUS:0x00000000
[   13.174922]  ipmi_msghandler(E)
[   13.180567] amdgpu 000d:03:00.0: amdgpu: 	 Faulty UTCL2 client ID: CB/DB (0x0)
[   13.183435]  arm_spe_pmu(E)
[   13.189514] amdgpu 000d:03:00.0: amdgpu: 	 MORE_FAULTS: 0x0
[   13.192121]  drm_kms_helper(E)
[   13.197853] amdgpu 000d:03:00.0: amdgpu: 	 WALKER_ERROR: 0x0
[   13.200373]  usbcore(E)
[   13.205150] amdgpu 000d:03:00.0: amdgpu: 	 PERMISSION_FAULTS: 0x0
[   13.207584]  igb(E)
[   13.219218] amdgpu 000d:03:00.0: amdgpu: 	 MAPPING_ERROR: 0x0
[   13.222433]  usb_common(E)
[   13.231898] amdgpu 000d:03:00.0: amdgpu: 	 RW: 0x0
[   13.234505]  i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   13.317462] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G            E      6.6.0-rc7+ #26
[   13.325541] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   13.339348] Workqueue: events work_for_cpu_fn
[   13.343698] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   13.350648] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.355901] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   13.362453] sp : ffff80008023bc10
[   13.365755] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   13.372880] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   13.380005] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   13.387129] x20: ffff07ff9e4100e8 x19: ffff07ff9e418580 x18: 0000000000000006
[   13.394254] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   13.401378] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   13.408502] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   13.415626] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   13.422750] x5 : 0000000000000000 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   13.429874] x2 : 0000000000000000 x1 : ffff07ff9e4278f0 x0 : ffff07ff96258140
[   13.436999] Call trace:
[   13.439434]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.444339]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   13.449935]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   13.455706]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   13.461474]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   13.466722]  local_pci_probe+0x48/0xb8
[   13.470463]  work_for_cpu_fn+0x24/0x40
[   13.474201]  process_one_work+0x170/0x3d0
[   13.478201]  worker_thread+0x2bc/0x3e0
[   13.481939]  kthread+0x118/0x128
[   13.485158]  ret_from_fork+0x10/0x20
[   13.488723] ---[ end trace 0000000000000000 ]---
[   13.493915] igb 0004:04:00.0 enP4p4s0: igb: enP4p4s0 NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX/TX
[   13.505441] usb 1-4.4: New USB device found, idVendor=046b, idProduct=ff01, bcdDevice= 1.00
[   13.513802] usb 1-4.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[   13.521120] ------------[ cut here ]------------
[   13.525725] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.536714] Modules linked in: hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E) efivarfs(E) ip_tables(E)
[   13.536780]  x_tables(E) autofs4(E)
[   13.630366] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   13.638444] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   13.652251] Workqueue: events work_for_cpu_fn
[   13.656599] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   13.663549] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.668800] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   13.675351] sp : ffff80008023bc10
[   13.678653] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   13.685778] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   13.692902] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   13.700026] x20: ffff07ff9e4100f0 x19: ffff07ff9e418ce8 x18: 0000000000000006
[   13.707150] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   13.714274] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   13.721399] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   13.728523] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   13.735647] x5 : 0000000000000002 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   13.742771] x2 : 0000000000000002 x1 : ffff07ff9e4278f0 x0 : ffff07ff96258148
[   13.749895] Call trace:
[   13.752329]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.757233]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   13.762828]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   13.768596]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   13.774365]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   13.779612]  local_pci_probe+0x48/0xb8
[   13.783352]  work_for_cpu_fn+0x24/0x40
[   13.787090]  process_one_work+0x170/0x3d0
[   13.791089]  worker_thread+0x2bc/0x3e0
[   13.794828]  kthread+0x118/0x128
[   13.798045]  ret_from_fork+0x10/0x20
[   13.801609] ---[ end trace 0000000000000000 ]---
[   13.806224] usb 1-4.4: Product: Virtual Hub
[   13.810401] usb 1-4.4: Manufacturer: American Megatrends Inc.
[   13.816141] usb 1-4.4: SerialNumber: serial
[   13.820837] ------------[ cut here ]------------
[   13.825444] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.836431] Modules linked in: llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E) efivarfs(E)
[   13.836496]  ip_tables(E) x_tables(E) autofs4(E)
[   13.930690] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   13.938768] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   13.952575] Workqueue: events work_for_cpu_fn
[   13.956922] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   13.963872] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   13.969122] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   13.975672] sp : ffff80008023bc10
[   13.978974] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   13.986098] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   13.993222] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   14.000346] x20: ffff07ff9e4100f8 x19: ffff07ff9e419098 x18: 0000000000000006
[   14.007470] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   14.014594] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   14.021718] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   14.028843] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   14.035967] x5 : 0000000000000003 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   14.043091] x2 : 0000000000000003 x1 : ffff07ff9e4278f0 x0 : ffff07ff9625814c
[   14.050215] Call trace:
[   14.052650]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.057554]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   14.063149]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   14.068918]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   14.074686]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   14.079933]  local_pci_probe+0x48/0xb8
[   14.083672]  work_for_cpu_fn+0x24/0x40
[   14.087410]  process_one_work+0x170/0x3d0
[   14.091409]  worker_thread+0x2bc/0x3e0
[   14.095148]  kthread+0x118/0x128
[   14.098365]  ret_from_fork+0x10/0x20
[   14.101930] ---[ end trace 0000000000000000 ]---
[   14.106648] ------------[ cut here ]------------
[   14.111255] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.122240] Modules linked in: llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E) efivarfs(E)
[   14.122303]  ip_tables(E) x_tables(E) autofs4(E)
[   14.216496] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   14.224574] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   14.238380] Workqueue: events work_for_cpu_fn
[   14.242727] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   14.249677] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.254927] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   14.261478] sp : ffff80008023bc10
[   14.264779] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   14.271904] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   14.279029] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   14.286153] x20: ffff07ff9e410100 x19: ffff07ff9e419448 x18: 0000000000000006
[   14.293277] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   14.300401] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   14.307525] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   14.314649] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   14.321773] x5 : 0000000000000004 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   14.328897] x2 : 0000000000000004 x1 : ffff07ff9e4278f0 x0 : ffff07ff96258150
[   14.336022] Call trace:
[   14.338456]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.343360]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   14.348955]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   14.354723]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   14.360492]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   14.365739]  local_pci_probe+0x48/0xb8
[   14.369477]  work_for_cpu_fn+0x24/0x40
[   14.373215]  process_one_work+0x170/0x3d0
[   14.377214]  worker_thread+0x2bc/0x3e0
[   14.380953]  kthread+0x118/0x128
[   14.384170]  ret_from_fork+0x10/0x20
[   14.387735] ---[ end trace 0000000000000000 ]---
[   14.392372] ------------[ cut here ]------------
[   14.396978] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.407960] Modules linked in: llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E) efivarfs(E)
[   14.408021]  ip_tables(E) x_tables(E) autofs4(E)
[   14.502214] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   14.510292] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   14.524098] Workqueue: events work_for_cpu_fn
[   14.528445] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   14.535394] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.540643] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   14.547192] sp : ffff80008023bc10
[   14.550494] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   14.557619] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   14.564744] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   14.571868] x20: ffff07ff9e410108 x19: ffff07ff9e4197f8 x18: 0000000000000006
[   14.578992] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   14.586116] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   14.593240] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   14.600364] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   14.607488] x5 : 0000000000000005 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   14.614612] x2 : 0000000000000005 x1 : ffff07ff9e4278f0 x0 : ffff07ff96258154
[   14.621736] Call trace:
[   14.624170]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.629072]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   14.634667]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   14.640436]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   14.646203]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   14.651450]  local_pci_probe+0x48/0xb8
[   14.655189]  work_for_cpu_fn+0x24/0x40
[   14.658927]  process_one_work+0x170/0x3d0
[   14.662926]  worker_thread+0x2bc/0x3e0
[   14.666664]  kthread+0x118/0x128
[   14.669882]  ret_from_fork+0x10/0x20
[   14.673446] ---[ end trace 0000000000000000 ]---
[   14.678138] ------------[ cut here ]------------
[   14.682745] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.693729] Modules linked in: stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E)
[   14.693795]  efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   14.788597] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   14.796675] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   14.810481] Workqueue: events work_for_cpu_fn
[   14.814828] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   14.821777] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.827027] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   14.833577] sp : ffff80008023bc10
[   14.836878] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   14.844003] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   14.851127] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   14.858251] x20: ffff07ff9e410110 x19: ffff07ff9e419ba8 x18: 0000000000000006
[   14.865375] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   14.872499] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   14.879623] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   14.886747] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   14.893872] x5 : 0000000000000002 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   14.900996] x2 : 0000000000000002 x1 : ffff07ff9e4278f0 x0 : ffff07ff96258148
[   14.908120] Call trace:
[   14.910554]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.915457]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   14.921051]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   14.926820]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   14.932588]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   14.937835]  local_pci_probe+0x48/0xb8
[   14.941574]  work_for_cpu_fn+0x24/0x40
[   14.945312]  process_one_work+0x170/0x3d0
[   14.949311]  worker_thread+0x2bc/0x3e0
[   14.953050]  kthread+0x118/0x128
[   14.956267]  ret_from_fork+0x10/0x20
[   14.959831] ---[ end trace 0000000000000000 ]---
[   14.964491] ------------[ cut here ]------------
[   14.969097] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   14.980079] Modules linked in: stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E)
[   14.980140]  efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   15.074942] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   15.083019] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   15.096826] Workqueue: events work_for_cpu_fn
[   15.101172] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   15.108122] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.113371] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   15.119921] sp : ffff80008023bc10
[   15.123223] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   15.130347] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   15.137471] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   15.144595] x20: ffff07ff9e410118 x19: ffff07ff9e419f58 x18: 0000000000000006
[   15.151719] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   15.158844] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   15.165968] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   15.173092] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   15.180216] x5 : 0000000000000003 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   15.187340] x2 : 0000000000000003 x1 : ffff07ff9e4278f0 x0 : ffff07ff9625814c
[   15.194464] Call trace:
[   15.196898]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.201801]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   15.207396]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   15.213163]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   15.218931]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   15.224179]  local_pci_probe+0x48/0xb8
[   15.227917]  work_for_cpu_fn+0x24/0x40
[   15.231655]  process_one_work+0x170/0x3d0
[   15.235654]  worker_thread+0x2bc/0x3e0
[   15.239393]  kthread+0x118/0x128
[   15.242610]  ret_from_fork+0x10/0x20
[   15.246175] ---[ end trace 0000000000000000 ]---
[   15.250864] ------------[ cut here ]------------
[   15.255471] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.266453] Modules linked in: stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E)
[   15.266515]  efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   15.361316] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   15.369394] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   15.383201] Workqueue: events work_for_cpu_fn
[   15.387547] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   15.394497] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.399746] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   15.406295] sp : ffff80008023bc10
[   15.409596] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   15.416721] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   15.423846] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   15.430970] x20: ffff07ff9e410120 x19: ffff07ff9e41a308 x18: 0000000000000006
[   15.438094] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   15.445218] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   15.452342] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   15.459466] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   15.466590] x5 : 0000000000000004 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   15.473714] x2 : 0000000000000004 x1 : ffff07ff9e4278f0 x0 : ffff07ff96258150
[   15.480838] Call trace:
[   15.483272]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.488173]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   15.493767]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   15.499535]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   15.505303]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   15.510549]  local_pci_probe+0x48/0xb8
[   15.514288]  work_for_cpu_fn+0x24/0x40
[   15.518026]  process_one_work+0x170/0x3d0
[   15.522025]  worker_thread+0x2bc/0x3e0
[   15.525764]  kthread+0x118/0x128
[   15.528981]  ret_from_fork+0x10/0x20
[   15.532545] ---[ end trace 0000000000000000 ]---
[   15.537202] hub 1-4.4:1.0: USB hub found
[   15.541151] ------------[ cut here ]------------
[   15.545756] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.556739] Modules linked in: mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E) configfs(E)
[   15.556805]  efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   15.652215] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   15.660293] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   15.674099] Workqueue: events work_for_cpu_fn
[   15.678446] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   15.685396] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.690645] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   15.697194] sp : ffff80008023bc10
[   15.700496] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   15.707621] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   15.714745] x23: ffff07ff9e4278f0 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   15.721869] x20: ffff07ff9e410128 x19: ffff07ff9e41a6b8 x18: 0000000000000006
[   15.728993] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   15.736118] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   15.743242] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   15.750366] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   15.757490] x5 : 0000000000000005 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   15.764614] x2 : 0000000000000005 x1 : ffff07ff9e4278f0 x0 : ffff07ff96258154
[   15.771738] Call trace:
[   15.774172]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.779074]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   15.784669]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   15.790438]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   15.796205]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   15.801452]  local_pci_probe+0x48/0xb8
[   15.805191]  work_for_cpu_fn+0x24/0x40
[   15.808929]  process_one_work+0x170/0x3d0
[   15.812929]  worker_thread+0x2bc/0x3e0
[   15.816667]  kthread+0x118/0x128
[   15.819885]  ret_from_fork+0x10/0x20
[   15.823449] ---[ end trace 0000000000000000 ]---
[   15.828152] hub 1-4.4:1.0: 5 ports detected
[   15.832345] ------------[ cut here ]------------
[   15.836950] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.847934] Modules linked in: garp(E) mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E) dax(E)
[   15.847999]  configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   15.944104] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   15.952182] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   15.965988] Workqueue: events work_for_cpu_fn
[   15.970335] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   15.977285] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   15.982535] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   15.989085] sp : ffff80008023bc10
[   15.992387] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   15.999512] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   16.006637] x23: ffff07ff9e42fc70 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   16.013761] x20: ffff07ff9e410130 x19: ffff07ff9e428500 x18: 0000000000000006
[   16.020885] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   16.028009] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   16.035133] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   16.042258] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   16.049382] x5 : 0000000000000000 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   16.056506] x2 : 0000000000000000 x1 : ffff07ff9e42fc70 x0 : ffff07ffaf205cd0
[   16.063630] Call trace:
[   16.066064]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.070967]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   16.076563]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   16.082332]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   16.088101]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   16.093348]  local_pci_probe+0x48/0xb8
[   16.097087]  work_for_cpu_fn+0x24/0x40
[   16.100825]  process_one_work+0x170/0x3d0
[   16.104825]  worker_thread+0x2bc/0x3e0
[   16.108563]  kthread+0x118/0x128
[   16.111780]  ret_from_fork+0x10/0x20
[   16.115345] ---[ end trace 0000000000000000 ]---
[   16.120568] ------------[ cut here ]------------
[   16.121033] 8021q: 802.1Q VLAN Support v1.8
[   16.125174] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.129399] 8021q: adding VLAN 0 to HW filter on device enP4p4s0
[   16.139863] Modules linked in: 8021q(E+) garp(E) mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E)
[   16.145921]  dax(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   16.242897] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   16.250975] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   16.264781] Workqueue: events work_for_cpu_fn
[   16.269127] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   16.276077] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.281328] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   16.287877] sp : ffff80008023bc10
[   16.291180] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   16.298304] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   16.305428] x23: ffff07ff9e42fc70 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   16.312552] x20: ffff07ff9e410138 x19: ffff07ff9e428c78 x18: 0000000000000006
[   16.319676] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   16.326800] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   16.333924] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   16.341048] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   16.348172] x5 : 0000000000000001 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   16.355296] x2 : 0000000000000001 x1 : ffff07ff9e42fc70 x0 : ffff07ffaf205cd4
[   16.362421] Call trace:
[   16.364855]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.369757]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   16.375353]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   16.381121]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   16.386889]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   16.392137]  local_pci_probe+0x48/0xb8
[   16.395876]  work_for_cpu_fn+0x24/0x40
[   16.399614]  process_one_work+0x170/0x3d0
[   16.403613]  worker_thread+0x2bc/0x3e0
[   16.407352]  kthread+0x118/0x128
[   16.410569]  ret_from_fork+0x10/0x20
[   16.414134] ---[ end trace 0000000000000000 ]---
[   16.418776] ------------[ cut here ]------------
[   16.423381] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.434364] Modules linked in: 8021q(E) garp(E) mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E)
[   16.434426]  dax(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   16.531313] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   16.539390] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   16.553196] Workqueue: events work_for_cpu_fn
[   16.557543] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   16.564493] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.569742] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   16.576292] sp : ffff80008023bc10
[   16.579594] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   16.586718] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   16.593843] x23: ffff07ff9e433440 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   16.600967] x20: ffff07ff9e410140 x19: ffff07ff9e432928 x18: 0000000000000006
[   16.608091] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   16.615215] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   16.622339] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   16.629463] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   16.636587] x5 : 0000000000000000 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   16.643711] x2 : 0000000000000000 x1 : ffff07ff9e433440 x0 : ffff07ffaf2059a8
[   16.650835] Call trace:
[   16.653269]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.658172]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   16.663767]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   16.669535]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   16.675304]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   16.680551]  local_pci_probe+0x48/0xb8
[   16.684290]  work_for_cpu_fn+0x24/0x40
[   16.688028]  process_one_work+0x170/0x3d0
[   16.692027]  worker_thread+0x2bc/0x3e0
[   16.695765]  kthread+0x118/0x128
[   16.698983]  ret_from_fork+0x10/0x20
[   16.702547] ---[ end trace 0000000000000000 ]---
[   16.707308] ------------[ cut here ]------------
[   16.711916] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.722899] Modules linked in: 8021q(E) garp(E) mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E)
[   16.722962]  dax(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   16.819850] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   16.827928] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   16.841736] Workqueue: events work_for_cpu_fn
[   16.846083] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   16.853032] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.858282] lr : amdgpu_fence_driver_hw_fini+0x118/0x160 [amdgpu]
[   16.864832] sp : ffff80008023bc10
[   16.868134] x29: ffff80008023bc20 x28: 0000000000000000 x27: 0000000000000000
[   16.875258] x26: ffff08075be33aa8 x25: ffff07ff9e400010 x24: ffff07ff9e400000
[   16.882382] x23: ffff07ff9e4381e8 x22: ffff07ff9e400010 x21: ffff07ff9e4104c8
[   16.889506] x20: ffff07ff9e410148 x19: ffff07ff9e436468 x18: 0000000000000006
[   16.896630] x17: 0000000000000020 x16: ffffb6de2cbe7140 x15: fffffffffffffefb
[   16.903754] x14: 0000000000000000 x13: ffff07ff84071d40 x12: ffff07ff922f94e0
[   16.910879] x11: 00000000000003e8 x10: ffff07ff922f94e0 x9 : ffffb6dde3e695e0
[   16.918003] x8 : ffffb6dde3a5e000 x7 : 0000000000000000 x6 : 0000000075c03baa
[   16.925127] x5 : 0000000000000000 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   16.932251] x2 : 0000000000000000 x1 : ffff07ff9e4381e8 x0 : ffff07ffaf205880
[   16.939375] Call trace:
[   16.941809]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   16.946712]  amdgpu_device_fini_hw+0xb8/0x380 [amdgpu]
[   16.952307]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   16.958074]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   16.963843]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   16.969091]  local_pci_probe+0x48/0xb8
[   16.972829]  work_for_cpu_fn+0x24/0x40
[   16.976567]  process_one_work+0x170/0x3d0
[   16.980566]  worker_thread+0x2bc/0x3e0
[   16.984305]  kthread+0x118/0x128
[   16.987522]  ret_from_fork+0x10/0x20
[   16.991087] ---[ end trace 0000000000000000 ]---
[   16.995787] ------------[ cut here ]------------
[   17.000394] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.011378] Modules linked in: 8021q(E) garp(E) mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E)
[   17.011441]  dax(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   17.108327] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   17.116405] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   17.130211] Workqueue: events work_for_cpu_fn
[   17.134558] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   17.141507] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.146757] lr : gfx_v11_0_hw_fini+0x2c/0x130 [amdgpu]
[   17.152355] sp : ffff80008023bc40
[   17.155657] x29: ffff80008023bc40 x28: 0000000000000000 x27: 0000000000000000
[   17.162782] x26: ffffb6dde457b160 x25: ffffb6dde457ae08 x24: ffff07ff9e44144a
[   17.169906] x23: ffff07ff9e400010 x22: ffff07ff9e444000 x21: ffff07ff9e40f000
[   17.177030] x20: ffff07ff9e400000 x19: ffff07ff9e400000 x18: ffff80008023ba18
[   17.184154] x17: 0000000000000020 x16: ffffb6de2d6fe6c0 x15: ffffffffffffffff
[   17.191278] x14: 00000000009f000f x13: 1fffe0fff0046821 x12: ffff80008023ba18
[   17.198402] x11: 0000000000000040 x10: ffffb6de2e0dbab0 x9 : ffffb6dde3fa1664
[   17.205526] x8 : ffff80008023bb28 x7 : 0000000000000000 x6 : 0000000000000000
[   17.212650] x5 : 0000000000000000 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   17.219774] x2 : 0000000000000000 x1 : ffff07ff9e427908 x0 : ffff07ffaf2054b8
[   17.226898] Call trace:
[   17.229333]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.234236]  amdgpu_device_fini_hw+0x20c/0x380 [amdgpu]
[   17.239919]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   17.245688]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   17.251457]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   17.256704]  local_pci_probe+0x48/0xb8
[   17.260442]  work_for_cpu_fn+0x24/0x40
[   17.264181]  process_one_work+0x170/0x3d0
[   17.268180]  worker_thread+0x2bc/0x3e0
[   17.271919]  kthread+0x118/0x128
[   17.275136]  ret_from_fork+0x10/0x20
[   17.278701] ---[ end trace 0000000000000000 ]---
[   17.283338] ------------[ cut here ]------------
[   17.287945] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.298928] Modules linked in: 8021q(E) garp(E) mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E)
[   17.298993]  dax(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   17.395880] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   17.403957] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   17.417764] Workqueue: events work_for_cpu_fn
[   17.422110] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   17.429059] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.434308] lr : gfx_v11_0_hw_fini+0x40/0x130 [amdgpu]
[   17.439906] sp : ffff80008023bc40
[   17.443208] x29: ffff80008023bc40 x28: 0000000000000000 x27: 0000000000000000
[   17.450333] x26: ffffb6dde457b160 x25: ffffb6dde457ae08 x24: ffff07ff9e44144a
[   17.457457] x23: ffff07ff9e400010 x22: ffff07ff9e444000 x21: ffff07ff9e40f000
[   17.464581] x20: ffff07ff9e400000 x19: ffff07ff9e400000 x18: ffff80008023ba18
[   17.471705] x17: 0000000000000020 x16: ffffb6de2d6fe6c0 x15: ffffffffffffffff
[   17.478829] x14: 00000000009f000f x13: 1fffe0fff0046821 x12: ffff80008023ba18
[   17.485953] x11: 0000000000000040 x10: ffffb6de2e0dbab0 x9 : ffffb6dde3fa1678
[   17.493077] x8 : ffff80008023bb28 x7 : 0000000000000000 x6 : 0000000000000000
[   17.500201] x5 : 0000000000000000 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   17.507325] x2 : 0000000000000000 x1 : ffff07ff9e427920 x0 : ffff07ffaf2052b0
[   17.514449] Call trace:
[   17.516883]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.521785]  amdgpu_device_fini_hw+0x20c/0x380 [amdgpu]
[   17.527466]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   17.533234]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   17.539002]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   17.544249]  local_pci_probe+0x48/0xb8
[   17.547988]  work_for_cpu_fn+0x24/0x40
[   17.551726]  process_one_work+0x170/0x3d0
[   17.555725]  worker_thread+0x2bc/0x3e0
[   17.559463]  kthread+0x118/0x128
[   17.562680]  ret_from_fork+0x10/0x20
[   17.566244] ---[ end trace 0000000000000000 ]---
[   17.620923] ------------[ cut here ]------------
[   17.625532] WARNING: CPU: 0 PID: 10 at drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:615 amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.636522] Modules linked in: 8021q(E) garp(E) mrp(E) stp(E) llc(E) hid_generic(E) usbhid(E) hid(E) qrtr(E) iptable_nat(E) nf_nat(E) nf_conntrack(E) nf_defrag_ipv6(E) nf_defrag_ipv4(E) libcrc32c(E) iptable_mangle(E) amdgpu(E+) iptable_filter(E) snd_hda_codec_hdmi(E) snd_hda_intel(E) snd_intel_dspcfg(E) snd_hda_codec(E) aes_ce_blk(E) snd_hda_core(E) amdxcp(E) aes_ce_cipher(E) drm_exec(E) crct10dif_ce(E) gpu_sched(E) polyval_ce(E) snd_hwdep(E) drm_buddy(E) snd_pcm(E) polyval_generic(E) video(E) ghash_ce(E) drm_suballoc_helper(E) snd_timer(E) drm_ttm_helper(E) gf128mul(E) sha2_ce(E) ttm(E) snd(E) ipmi_ssif(E) xhci_pci(E) sha256_arm64(E) drm_display_helper(E) ipmi_devintf(E) soundcore(E) xhci_hcd(E) sha1_ce(E) drm_shmem_helper(E) sbsa_gwdt(E) ipmi_msghandler(E) arm_spe_pmu(E) drm_kms_helper(E) usbcore(E) igb(E) usb_common(E) i2c_algo_bit(E) i2c_designware_platform(E) arm_cmn(E) xgene_hwmon(E) arm_dsu_pmu(E) i2c_designware_core(E) cppc_cpufreq(E) evdev(E) binfmt_misc(E) loop(E) fuse(E) efi_pstore(E) drm(E) dm_mod(E)
[   17.636590]  dax(E) configfs(E) efivarfs(E) ip_tables(E) x_tables(E) autofs4(E)
[   17.733478] CPU: 0 PID: 10 Comm: kworker/0:1 Tainted: G        W   E      6.6.0-rc7+ #26
[   17.741556] Hardware name: ADLINK Ampere Altra Developer Platform/Ampere Altra Developer Platform, BIOS TianoCore 2.04.100.10 (SYS: 2.06.20220308) 04/18/2
[   17.755362] Workqueue: events work_for_cpu_fn
[   17.759710] pstate: 20400009 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   17.766659] pc : amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.771910] lr : gmc_v11_0_hw_fini+0x34/0x88 [amdgpu]
[   17.777420] sp : ffff80008023bc40
[   17.780722] x29: ffff80008023bc40 x28: 0000000000000000 x27: 0000000000000000
[   17.787846] x26: ffffb6dde457b160 x25: ffffb6dde457ae08 x24: ffff07ff9e44144a
[   17.794970] x23: ffff07ff9e400010 x22: ffff07ff9e444000 x21: ffff07ff9e40f000
[   17.802094] x20: ffff07ff9e400000 x19: ffff07ff9e400000 x18: 0000000000000000
[   17.809218] x17: 0000000000000000 x16: ffffb6de2d6c7d48 x15: 0000000000000000
[   17.816343] x14: 0000000000000000 x13: 0000000000000000 x12: ffffb6de2e0bdb18
[   17.823467] x11: 071c71c71c71c71c x10: 0000000000000be0 x9 : ffffb6dde3f1bc34
[   17.830592] x8 : ffff80008023ba50 x7 : 0000000000000000 x6 : fffffffffffff000
[   17.837716] x5 : 0000000000000000 x4 : ffff07ff9e400000 x3 : 0000000000000000
[   17.844840] x2 : 0000000000000000 x1 : ffff07ff9e400c38 x0 : ffff07ffaf2057f0
[   17.851964] Call trace:
[   17.854399]  amdgpu_irq_put+0xa8/0xc8 [amdgpu]
[   17.859301]  amdgpu_device_fini_hw+0x20c/0x380 [amdgpu]
[   17.864982]  amdgpu_driver_unload_kms+0x54/0x80 [amdgpu]
[   17.870750]  amdgpu_driver_load_kms+0x100/0x1c0 [amdgpu]
[   17.876517]  amdgpu_pci_probe+0x134/0x428 [amdgpu]
[   17.881764]  local_pci_probe+0x48/0xb8
[   17.885503]  work_for_cpu_fn+0x24/0x40
[   17.889241]  process_one_work+0x170/0x3d0
[   17.893241]  worker_thread+0x2bc/0x3e0
[   17.896980]  kthread+0x118/0x128
[   17.900197]  ret_from_fork+0x10/0x20
[   17.903762] ---[ end trace 0000000000000000 ]---
[   17.908712] usb 1-4.5: new high-speed USB device number 5 using xhci_hcd
[   17.915662] amdgpu: probe of 000d:03:00.0 failed with error -110
[   18.016683] usb 1-4.5: New USB device found, idVendor=0424, idProduct=724a, bcdDevice= 6.25
[   18.025031] usb 1-4.5: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   18.032332] usb 1-4.5: Product: USB2 Controller Hub
[   18.037201] usb 1-4.5: Manufacturer: Microchip Tech
[   18.111383] mc: Linux media interface: v0.10
[   18.133337] usb 1-4.4.3: new low-speed USB device number 6 using xhci_hcd
[   18.268315] usb 1-4.4.3: New USB device found, idVendor=046b, idProduct=ff10, bcdDevice= 1.00
[   18.276836] usb 1-4.4.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[   18.284309] usb 1-4.4.3: Product: Virtual Keyboard and Mouse
[   18.289960] usb 1-4.4.3: Manufacturer: American Megatrends Inc.
[   18.315919] input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0004:00/0004:00:03.0/0004:03:00.0/usb1/1-4/1-4.4/1-4.4.3/1-4.4.3:1.0/0003:046B:FF10.0003/input/input10
[   18.332875] hid-generic 0003:046B:FF10.0003: input,hidraw2: USB HID v1.10 Keyboard [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0004:03:00.0-4.4.3/input0
[   18.356775] input: American Megatrends Inc. Virtual Keyboard and Mouse as /devices/pci0004:00/0004:00:03.0/0004:03:00.0/usb1/1-4/1-4.4/1-4.4.3/1-4.4.3:1.1/0003:046B:FF10.0004/input/input11
[   18.373636] hid-generic 0003:046B:FF10.0004: input,hidraw3: USB HID v1.10 Mouse [American Megatrends Inc. Virtual Keyboard and Mouse] on usb-0004:03:00.0-4.4.3/input1
[   18.379828] [drm] amdgpu: ttm finalized
[   18.404554] usbcore: registered new interface driver snd-usb-audio
[   89.402355] systemd-journald[433]: /var/log/journal/698d8624b408416f8010372e8e433376/user-1000.journal: Journal file uses a different sequence number ID, rotating.

Re: [REGRESSION] rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"

[Christian König]

Hi Alexey,

trying to answer some of the questions since Alex is currently on vacation.

Am 30.10.23 um 17:01 schrieb Alexey Klimov:
> Hi Alex,
>
> On Thu, 26 Oct 2023 at 19:53, Alex Deucher <alexdeucher@gmail.com> wrote:
>> On Thu, Oct 26, 2023 at 1:33 PM Alexey Klimov <alexey.klimov@linaro.org> wrote:
>>> #regzbot introduced: 1cfb4d612127
>>> #regzbot title: rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"
>>>
>>> Hi all,
>>>
>>> I've been playing with RX7600 and it was observed that amdgpu stopped working between kernel 6.2 and 6.5.
>>> Then I narrowed it down to 6.4 <-> 6.5-rc1 and finally bisect pointed at 1cfb4d6121276a829aa94d0e32a7f5e1830ebc21
>>> And I manually checked if it boots/works on the previous commit and the mentioned one.
>>>
>>> I guess the log also reveals warning in error path. Please see below.
>>>
>>> I didn't check any further. This is simple debian testing system with the following cmdline options:
>>> root@avadebian:~# cat /proc/cmdline
>>> BOOT_IMAGE=/boot/vmlinuz-6.6-rc7+ ignore_loglevel root=/dev/nvme1n1p2 ro nr_cpus=32
>>>
>>> So far simple revert (patch is below) returns things back to normal-ish: there are huge graphics artifacts on Xorg/X11 under 6.1 to upstream kernel. Wayland-based sway works great without issues. Not sure where should I report this.
>>>
>>> Please let me know if I can help debugging, testing or provide some other logs regarding 1cfb4d612127? Any cmdline options to collect more info?
>> Please make sure you have this patch as well:
>> e602157ec089240861cd641ee2c7c64eeaec09bf ("drm/amdgpu: fix S3 issue if
>> MQD in VRAM")
>> Please open a ticket here so we can track this:
>> https://gitlab.freedesktop.org/drm/amd/-/issues/
> The patch was there during testing and I will open a ticket there.
>
>> I think I see the problem.  Please see if attached patch 1 fixes the
>> issue.  If this fixes it, that would also explain the issues you are
>> seeing with Xorg.  It would appear there are limitations around MMIO
>> access on your platform and unfortunately most graphics APIs require
>> unaligned access to MMIO space with the CPU.  We can fix the kernel
>> side pretty easily, but userspace will be a problem.
> Does it mean that we don't have unaligned access to PCIe MMIO space on
> this Adlink Ampere AVA arm64 platform?

Yes, that is perfectly possible and makes that platform unusable for 
most gfx applications.

We had tons of reports for different ARM boards and HW generations and 
even looped in some ARM engineers.

Essentially if you want to run high level GFX stacks like Vulkan and 
OpenGL on a platform with AMD or NVIDIA hardware your platform needs to 
fulfill certain requirements:

1. Correctly implement the PCIe spec!

     We actually have tons of boards where people attach an PCIe root 
complex to the ARM CPU and expect that to work. The problem is that this 
isn't PCIe compliant!
     You actually need the ARM IP for PCIe for this to work correctly, 
without that the root complex can't do system memory coherent 
transactions for example.

2. Be able to run all types of memory accesses on PCIe BARs. For example 
some platforms can't do large reads and writes (vector operations) to 
PCIe BARs, but can do them to system memory.

     This is actually not a hardware requirement, but one of the Vulkan 
and OpenGL stack and applications based on them.
     You can work around this by disallowing CPU access to PCIe BARs, 
but that either cripples performance or even results in applications not 
working at all.

> Do you know if it is related to the thing that PCIe BARs are mapped as
> a device memory and not a normal memory? (and they should be mapped as
> normal memory)

Depends on what you mean with this. When changing the mapping type 
results in allowing unaligned and bigger accesses then yes that would help.

We will upstream the patches to make at least the kernel side work as 
expected, but that's fixing only halve of the problem.

Regards,
Christian.

>
> [..]
>
>> Just removing the addition of the AMDGPU_GEM_DOMAIN_VRAM domain here
>> will revert the behavior.  Since this is an important optimization and
>> we aren't seeing any issues on x86, I'd prefer to just limit your arch
>> to GTT if we can't resolve it some other way.
>>
>> Try patch 1 and if that doesn't work we can fall back to some variant
>> of patch 2.
> The patch 1 alone doesn't fix the issue. Both patches 1 & 2 do work
> and amdgpu initializes. Still issues with Xorg and wayland works okay.
>
> Apart from that I observed "amdgpu: [gfxhub] page fault" one time:
>
> [   12.432567] amdgpu 000d:03:00.0: [drm:jpeg_v4_0_hw_init [amdgpu]]
> JPEG decode initialized successfully.
> [   12.442516] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault
> (src_id:0 ring:72 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
> [   12.454080] amdgpu 000d:03:00.0: amdgpu:   in page starting at
> address 0x00000000044b0000 from client 10
> [   12.457317] usb 1-4.4: new high-speed USB device number 4 using xhci_hcd
> [   12.463548] amdgpu 000d:03:00.0: amdgpu:
> GCVM_L2_PROTECTION_FAULT_STATUS:0x00000890
> [   12.463551] amdgpu 000d:03:00.0: amdgpu: Faulty UTCL2 client ID: CPF (0x4)
> [   12.484914] amdgpu 000d:03:00.0: amdgpu: MORE_FAULTS: 0x0
> [   12.490474] amdgpu 000d:03:00.0: amdgpu: WALKER_ERROR: 0x0
> [   12.496121] amdgpu 000d:03:00.0: amdgpu: PERMISSION_FAULTS: 0x9
> [   12.502202] amdgpu 000d:03:00.0: amdgpu: MAPPING_ERROR: 0x0
> [   12.507934] amdgpu 000d:03:00.0: amdgpu: RW: 0x0
> [   12.512716] amdgpu 000d:03:00.0: amdgpu: [gfxhub] page fault
> (src_id:0 ring:221 vmid:0 pasid:0, for process  pid 0 thread  pid 0)
> [   12.524355] amdgpu 000d:03:00.0: amdgpu:   in page starting at
> address 0x00000000044b1000 from client 10
> [   12.533821] amdgpu 000d:03:00.0: amdgpu:
> GCVM_L2_PROTECTION_FAULT_STATUS:0x000009BB
> [   12.541464] amdgpu 000d:03:00.0: amdgpu: Faulty UTCL2 client ID: CPF (0x4)
> [   12.548499] amdgpu 000d:03:00.0: amdgpu: MORE_FAULTS: 0x1
> [   12.554059] amdgpu 000d:03:00.0: amdgpu: WALKER_ERROR: 0x5
> [   12.558700] [drm:mes_v11_0_submit_pkt_and_poll_completion.constprop.0
> [amdgpu]] *ERROR* MES failed to response msg=0
>
> I am not sure if it related to clean 6.6 kernel or to additional patches 1 & 2.
> I did around 20 boots of clean 6.6-rc7 version and didn't observe this
> page fault.
> During 20 reboots of  6.6-rc7 + your patches 1 and 2 -- this page
> fault was observed one time only.
> Couldn't say how reproducible is this. The log is attached.
>
> Let me know if you want me to test/Ack patch 2 if you are going to send it.
>
> Thanks,
> Alexey

Re: [REGRESSION] rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"

[Linux regression tracking #update (Thorsten Leemhuis)]

[TLDR: This mail in primarily relevant for Linux kernel regression
tracking. See link in footer if these mails annoy you.]

On 26.10.23 19:33, Alexey Klimov wrote:
> #regzbot introduced: 1cfb4d612127
> #regzbot title: rx7600 stopped working after "1cfb4d612127 drm/amdgpu: put MQDs in VRAM"
> 
> Hi all,
> 
> I've been playing with RX7600 and it was observed that amdgpu stopped working between kernel 6.2 and 6.5.
> Then I narrowed it down to 6.4 <-> 6.5-rc1 and finally bisect pointed at 1cfb4d6121276a829aa94d0e32a7f5e1830ebc21
> And I manually checked if it boots/works on the previous commit and the mentioned one.

#regzbot fix: ba0fb4b48c19a
#regzbot ignore-activity

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)
--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
That page also explains what to do if mails like this annoy you.