[PATCH 00/14] Introduce xe_devcoredump.

[PATCH 00/14] Introduce xe_devcoredump.

[Rodrigo Vivi]

Xe needs to align with other drivers on the way that the error states are
dumped, avoiding a Xe only error_state solution. The goal is to use devcoredump
infrastructure to report error states, since it produces a standardized way
by exposing a virtual and temporary /sys/class/devcoredump device.

The initial goal is to have the simple_error_state in the devcoredump
so we start using the infrastructure.

But this is just a start point to start building a useful and
organized crash dump, using standard infrastructure. Later this
will be changed to have output that can be parsed by tools and
used for error replay.

Later, when we are in-tree, the goal is to collaborate with devcoredump
infrastructure with overall possible improvements, like multiple file support
for better organization of the dumps, snapshot support, dmesg extra print,
and whatever may make sense and help the overall infrastructure.

Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Rodrigo Vivi (14):
  drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI
  drm/xe: Introduce the dev_coredump infrastructure.
  drm/xe: Do not take any action if our device was removed.
  drm/xe: Extract non mapped regions out of GuC CTB into its own struct.
  drm/xe: Convert GuC CT print to snapshot capture and print.
  drm/xe: Add GuC CT snapshot to xe_devcoredump.
  drm/xe: Introduce guc_submit_types.h with relevant structs.
  drm/xe: Convert GuC Engine print to snapshot capture and print.
  drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.
  drm/xe: Convert Xe HW Engine print to snapshot capture and print.
  drm/xe: Add HW Engine snapshot to xe_devcoredump.
  drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.
  drm/xe: Convert VM print to snapshot capture and print.
  drm/xe: Add VM snapshot to xe_devcoredump.

 drivers/gpu/drm/xe/Kconfig                |   1 +
 drivers/gpu/drm/xe/Makefile               |   1 +
 drivers/gpu/drm/xe/regs/xe_engine_regs.h  |   3 +-
 drivers/gpu/drm/xe/xe_devcoredump.c       | 227 ++++++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  60 +++++
 drivers/gpu/drm/xe/xe_device_types.h      |   4 +
 drivers/gpu/drm/xe/xe_execlist.c          |   4 +-
 drivers/gpu/drm/xe/xe_gt_debugfs.c        |   2 +-
 drivers/gpu/drm/xe/xe_guc_ct.c            | 275 +++++++++++++++-------
 drivers/gpu/drm/xe/xe_guc_ct.h            |   7 +-
 drivers/gpu/drm/xe/xe_guc_ct_types.h      |  46 +++-
 drivers/gpu/drm/xe/xe_guc_fwif.h          |  29 ---
 drivers/gpu/drm/xe/xe_guc_submit.c        | 258 ++++++++++++++------
 drivers/gpu/drm/xe/xe_guc_submit.h        |  10 +-
 drivers/gpu/drm/xe/xe_guc_submit_types.h  | 155 ++++++++++++
 drivers/gpu/drm/xe/xe_hw_engine.c         | 210 ++++++++++++-----
 drivers/gpu/drm/xe/xe_hw_engine.h         |   8 +-
 drivers/gpu/drm/xe/xe_hw_engine_types.h   |  78 ++++++
 drivers/gpu/drm/xe/xe_pci.c               |   2 +
 drivers/gpu/drm/xe/xe_vm.c                | 140 +++++++++--
 drivers/gpu/drm/xe/xe_vm.h                |   6 +-
 drivers/gpu/drm/xe/xe_vm_types.h          |  18 ++
 23 files changed, 1288 insertions(+), 278 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
 create mode 100644 drivers/gpu/drm/xe/xe_guc_submit_types.h

--
2.39.2

[Intel-xe] [PATCH 00/14] Introduce xe_devcoredump.

[Rodrigo Vivi]

Xe needs to align with other drivers on the way that the error states are
dumped, avoiding a Xe only error_state solution. The goal is to use devcoredump
infrastructure to report error states, since it produces a standardized way
by exposing a virtual and temporary /sys/class/devcoredump device.

The initial goal is to have the simple_error_state in the devcoredump
so we start using the infrastructure.

But this is just a start point to start building a useful and
organized crash dump, using standard infrastructure. Later this
will be changed to have output that can be parsed by tools and
used for error replay.

Later, when we are in-tree, the goal is to collaborate with devcoredump
infrastructure with overall possible improvements, like multiple file support
for better organization of the dumps, snapshot support, dmesg extra print,
and whatever may make sense and help the overall infrastructure.

Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Rodrigo Vivi (14):
  drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI
  drm/xe: Introduce the dev_coredump infrastructure.
  drm/xe: Do not take any action if our device was removed.
  drm/xe: Extract non mapped regions out of GuC CTB into its own struct.
  drm/xe: Convert GuC CT print to snapshot capture and print.
  drm/xe: Add GuC CT snapshot to xe_devcoredump.
  drm/xe: Introduce guc_submit_types.h with relevant structs.
  drm/xe: Convert GuC Engine print to snapshot capture and print.
  drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.
  drm/xe: Convert Xe HW Engine print to snapshot capture and print.
  drm/xe: Add HW Engine snapshot to xe_devcoredump.
  drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.
  drm/xe: Convert VM print to snapshot capture and print.
  drm/xe: Add VM snapshot to xe_devcoredump.

 drivers/gpu/drm/xe/Kconfig                |   1 +
 drivers/gpu/drm/xe/Makefile               |   1 +
 drivers/gpu/drm/xe/regs/xe_engine_regs.h  |   3 +-
 drivers/gpu/drm/xe/xe_devcoredump.c       | 227 ++++++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  60 +++++
 drivers/gpu/drm/xe/xe_device_types.h      |   4 +
 drivers/gpu/drm/xe/xe_execlist.c          |   4 +-
 drivers/gpu/drm/xe/xe_gt_debugfs.c        |   2 +-
 drivers/gpu/drm/xe/xe_guc_ct.c            | 275 +++++++++++++++-------
 drivers/gpu/drm/xe/xe_guc_ct.h            |   7 +-
 drivers/gpu/drm/xe/xe_guc_ct_types.h      |  46 +++-
 drivers/gpu/drm/xe/xe_guc_fwif.h          |  29 ---
 drivers/gpu/drm/xe/xe_guc_submit.c        | 258 ++++++++++++++------
 drivers/gpu/drm/xe/xe_guc_submit.h        |  10 +-
 drivers/gpu/drm/xe/xe_guc_submit_types.h  | 155 ++++++++++++
 drivers/gpu/drm/xe/xe_hw_engine.c         | 210 ++++++++++++-----
 drivers/gpu/drm/xe/xe_hw_engine.h         |   8 +-
 drivers/gpu/drm/xe/xe_hw_engine_types.h   |  78 ++++++
 drivers/gpu/drm/xe/xe_pci.c               |   2 +
 drivers/gpu/drm/xe/xe_vm.c                | 140 +++++++++--
 drivers/gpu/drm/xe/xe_vm.h                |   6 +-
 drivers/gpu/drm/xe/xe_vm_types.h          |  18 ++
 23 files changed, 1288 insertions(+), 278 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
 create mode 100644 drivers/gpu/drm/xe/xe_guc_submit_types.h

--
2.39.2

[PATCH 01/14] drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI

[Rodrigo Vivi]

On xe_hw_engine_print_state we were printing:
value_of(0x510) + 4 instead of
value_of(0x514) as desired.

So, let's properly define a RING_EXECLIST_SQ_CONTENTS_HI
register to fix the issue and also to avoid other issues
like that.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 ++-
 drivers/gpu/drm/xe/xe_execlist.c         | 4 ++--
 drivers/gpu/drm/xe/xe_hw_engine.c        | 4 ++--
 3 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/xe/regs/xe_engine_regs.h b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
index 2aa67d001c34..a1e1d1c206fa 100644
--- a/drivers/gpu/drm/xe/regs/xe_engine_regs.h
+++ b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
@@ -84,7 +84,8 @@
 						 RING_FORCE_TO_NONPRIV_DENY)
 #define   RING_MAX_NONPRIV_SLOTS  12
 
-#define RING_EXECLIST_SQ_CONTENTS(base)		_MMIO((base) + 0x510)
+#define RING_EXECLIST_SQ_CONTENTS_LO(base)	_MMIO((base) + 0x510)
+#define RING_EXECLIST_SQ_CONTENTS_HI(base)	_MMIO((base) + 0x510 + 4)
 
 #define RING_EXECLIST_CONTROL(base)		_MMIO((base) + 0x550)
 #define	  EL_CTRL_LOAD				REG_BIT(0)
diff --git a/drivers/gpu/drm/xe/xe_execlist.c b/drivers/gpu/drm/xe/xe_execlist.c
index 02021457b1f0..37ac6473195e 100644
--- a/drivers/gpu/drm/xe/xe_execlist.c
+++ b/drivers/gpu/drm/xe/xe_execlist.c
@@ -84,9 +84,9 @@ static void __start_lrc(struct xe_hw_engine *hwe, struct xe_lrc *lrc,
 	xe_mmio_write32(gt, RING_MODE_GEN7(hwe->mmio_base).reg,
 			_MASKED_BIT_ENABLE(GEN11_GFX_DISABLE_LEGACY_MODE));
 
-	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 0,
+	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_LO(hwe->mmio_base).reg,
 			lower_32_bits(lrc_desc));
-	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 4,
+	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_HI(hwe->mmio_base).reg,
 			upper_32_bits(lrc_desc));
 	xe_mmio_write32(gt, RING_EXECLIST_CONTROL(hwe->mmio_base).reg,
 			EL_CTRL_LOAD);
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
index 4b56c35b988d..23b9f120c258 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.c
+++ b/drivers/gpu/drm/xe/xe_hw_engine.c
@@ -528,10 +528,10 @@ void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p)
 		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg));
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_LO: 0x%08x\n",
 		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS(0).reg));
+					 RING_EXECLIST_SQ_CONTENTS_LO(0).reg));
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_HI: 0x%08x\n",
 		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS(0).reg) + 4);
+					 RING_EXECLIST_SQ_CONTENTS_HI(0).reg));
 	drm_printf(p, "\tRING_EXECLIST_CONTROL: 0x%08x\n",
 		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg));
 
-- 
2.39.2

[Intel-xe] [PATCH 01/14] drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI

[Rodrigo Vivi]

On xe_hw_engine_print_state we were printing:
value_of(0x510) + 4 instead of
value_of(0x514) as desired.

So, let's properly define a RING_EXECLIST_SQ_CONTENTS_HI
register to fix the issue and also to avoid other issues
like that.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 ++-
 drivers/gpu/drm/xe/xe_execlist.c         | 4 ++--
 drivers/gpu/drm/xe/xe_hw_engine.c        | 4 ++--
 3 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/xe/regs/xe_engine_regs.h b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
index 2aa67d001c34..a1e1d1c206fa 100644
--- a/drivers/gpu/drm/xe/regs/xe_engine_regs.h
+++ b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
@@ -84,7 +84,8 @@
 						 RING_FORCE_TO_NONPRIV_DENY)
 #define   RING_MAX_NONPRIV_SLOTS  12
 
-#define RING_EXECLIST_SQ_CONTENTS(base)		_MMIO((base) + 0x510)
+#define RING_EXECLIST_SQ_CONTENTS_LO(base)	_MMIO((base) + 0x510)
+#define RING_EXECLIST_SQ_CONTENTS_HI(base)	_MMIO((base) + 0x510 + 4)
 
 #define RING_EXECLIST_CONTROL(base)		_MMIO((base) + 0x550)
 #define	  EL_CTRL_LOAD				REG_BIT(0)
diff --git a/drivers/gpu/drm/xe/xe_execlist.c b/drivers/gpu/drm/xe/xe_execlist.c
index 02021457b1f0..37ac6473195e 100644
--- a/drivers/gpu/drm/xe/xe_execlist.c
+++ b/drivers/gpu/drm/xe/xe_execlist.c
@@ -84,9 +84,9 @@ static void __start_lrc(struct xe_hw_engine *hwe, struct xe_lrc *lrc,
 	xe_mmio_write32(gt, RING_MODE_GEN7(hwe->mmio_base).reg,
 			_MASKED_BIT_ENABLE(GEN11_GFX_DISABLE_LEGACY_MODE));
 
-	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 0,
+	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_LO(hwe->mmio_base).reg,
 			lower_32_bits(lrc_desc));
-	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 4,
+	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_HI(hwe->mmio_base).reg,
 			upper_32_bits(lrc_desc));
 	xe_mmio_write32(gt, RING_EXECLIST_CONTROL(hwe->mmio_base).reg,
 			EL_CTRL_LOAD);
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
index 4b56c35b988d..23b9f120c258 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.c
+++ b/drivers/gpu/drm/xe/xe_hw_engine.c
@@ -528,10 +528,10 @@ void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p)
 		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg));
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_LO: 0x%08x\n",
 		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS(0).reg));
+					 RING_EXECLIST_SQ_CONTENTS_LO(0).reg));
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_HI: 0x%08x\n",
 		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS(0).reg) + 4);
+					 RING_EXECLIST_SQ_CONTENTS_HI(0).reg));
 	drm_printf(p, "\tRING_EXECLIST_CONTROL: 0x%08x\n",
 		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg));
 
-- 
2.39.2

Re: [Intel-xe] [PATCH 01/14] drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI

[Lucas De Marchi]

On Wed, Apr 26, 2023 at 04:57:00PM -0400, Rodrigo Vivi wrote:
>On xe_hw_engine_print_state we were printing:
>value_of(0x510) + 4 instead of
>value_of(0x514) as desired.
>
>So, let's properly define a RING_EXECLIST_SQ_CONTENTS_HI
>register to fix the issue and also to avoid other issues
>like that.
>
>Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
>---
> drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 ++-
> drivers/gpu/drm/xe/xe_execlist.c         | 4 ++--
> drivers/gpu/drm/xe/xe_hw_engine.c        | 4 ++--
> 3 files changed, 6 insertions(+), 5 deletions(-)
>
>diff --git a/drivers/gpu/drm/xe/regs/xe_engine_regs.h b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
>index 2aa67d001c34..a1e1d1c206fa 100644
>--- a/drivers/gpu/drm/xe/regs/xe_engine_regs.h
>+++ b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
>@@ -84,7 +84,8 @@
> 						 RING_FORCE_TO_NONPRIV_DENY)
> #define   RING_MAX_NONPRIV_SLOTS  12
>
>-#define RING_EXECLIST_SQ_CONTENTS(base)		_MMIO((base) + 0x510)
>+#define RING_EXECLIST_SQ_CONTENTS_LO(base)	_MMIO((base) + 0x510)
>+#define RING_EXECLIST_SQ_CONTENTS_HI(base)	_MMIO((base) + 0x510 + 4)
>
> #define RING_EXECLIST_CONTROL(base)		_MMIO((base) + 0x550)
> #define	  EL_CTRL_LOAD				REG_BIT(0)
>diff --git a/drivers/gpu/drm/xe/xe_execlist.c b/drivers/gpu/drm/xe/xe_execlist.c
>index 02021457b1f0..37ac6473195e 100644
>--- a/drivers/gpu/drm/xe/xe_execlist.c
>+++ b/drivers/gpu/drm/xe/xe_execlist.c
>@@ -84,9 +84,9 @@ static void __start_lrc(struct xe_hw_engine *hwe, struct xe_lrc *lrc,
> 	xe_mmio_write32(gt, RING_MODE_GEN7(hwe->mmio_base).reg,
> 			_MASKED_BIT_ENABLE(GEN11_GFX_DISABLE_LEGACY_MODE));
>
>-	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 0,
>+	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_LO(hwe->mmio_base).reg,
> 			lower_32_bits(lrc_desc));
>-	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 4,
>+	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_HI(hwe->mmio_base).reg,
> 			upper_32_bits(lrc_desc));
> 	xe_mmio_write32(gt, RING_EXECLIST_CONTROL(hwe->mmio_base).reg,
> 			EL_CTRL_LOAD);
>diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
>index 4b56c35b988d..23b9f120c258 100644
>--- a/drivers/gpu/drm/xe/xe_hw_engine.c
>+++ b/drivers/gpu/drm/xe/xe_hw_engine.c
>@@ -528,10 +528,10 @@ void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p)
> 		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg));
> 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_LO: 0x%08x\n",
> 		hw_engine_mmio_read32(hwe,

after this series and mine about register refactor we should probably go
and fix the coding style in this function.

... thankfully it was just this print that was wrong and the rest was
correct.

Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

thanks
Lucas De Marchi

>-					 RING_EXECLIST_SQ_CONTENTS(0).reg));
>+					 RING_EXECLIST_SQ_CONTENTS_LO(0).reg));
> 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_HI: 0x%08x\n",
> 		hw_engine_mmio_read32(hwe,
>-					 RING_EXECLIST_SQ_CONTENTS(0).reg) + 4);
>+					 RING_EXECLIST_SQ_CONTENTS_HI(0).reg));
> 	drm_printf(p, "\tRING_EXECLIST_CONTROL: 0x%08x\n",
> 		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg));
>
>-- 
>2.39.2
>

Re: [Intel-xe] [PATCH 01/14] drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI

[Rodrigo Vivi]

On Wed, Apr 26, 2023 at 02:40:18PM -0700, Lucas De Marchi wrote:
> On Wed, Apr 26, 2023 at 04:57:00PM -0400, Rodrigo Vivi wrote:
> > On xe_hw_engine_print_state we were printing:
> > value_of(0x510) + 4 instead of
> > value_of(0x514) as desired.
> > 
> > So, let's properly define a RING_EXECLIST_SQ_CONTENTS_HI
> > register to fix the issue and also to avoid other issues
> > like that.
> > 
> > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > ---
> > drivers/gpu/drm/xe/regs/xe_engine_regs.h | 3 ++-
> > drivers/gpu/drm/xe/xe_execlist.c         | 4 ++--
> > drivers/gpu/drm/xe/xe_hw_engine.c        | 4 ++--
> > 3 files changed, 6 insertions(+), 5 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/regs/xe_engine_regs.h b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
> > index 2aa67d001c34..a1e1d1c206fa 100644
> > --- a/drivers/gpu/drm/xe/regs/xe_engine_regs.h
> > +++ b/drivers/gpu/drm/xe/regs/xe_engine_regs.h
> > @@ -84,7 +84,8 @@
> > 						 RING_FORCE_TO_NONPRIV_DENY)
> > #define   RING_MAX_NONPRIV_SLOTS  12
> > 
> > -#define RING_EXECLIST_SQ_CONTENTS(base)		_MMIO((base) + 0x510)
> > +#define RING_EXECLIST_SQ_CONTENTS_LO(base)	_MMIO((base) + 0x510)
> > +#define RING_EXECLIST_SQ_CONTENTS_HI(base)	_MMIO((base) + 0x510 + 4)
> > 
> > #define RING_EXECLIST_CONTROL(base)		_MMIO((base) + 0x550)
> > #define	  EL_CTRL_LOAD				REG_BIT(0)
> > diff --git a/drivers/gpu/drm/xe/xe_execlist.c b/drivers/gpu/drm/xe/xe_execlist.c
> > index 02021457b1f0..37ac6473195e 100644
> > --- a/drivers/gpu/drm/xe/xe_execlist.c
> > +++ b/drivers/gpu/drm/xe/xe_execlist.c
> > @@ -84,9 +84,9 @@ static void __start_lrc(struct xe_hw_engine *hwe, struct xe_lrc *lrc,
> > 	xe_mmio_write32(gt, RING_MODE_GEN7(hwe->mmio_base).reg,
> > 			_MASKED_BIT_ENABLE(GEN11_GFX_DISABLE_LEGACY_MODE));
> > 
> > -	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 0,
> > +	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_LO(hwe->mmio_base).reg,
> > 			lower_32_bits(lrc_desc));
> > -	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS(hwe->mmio_base).reg + 4,
> > +	xe_mmio_write32(gt, RING_EXECLIST_SQ_CONTENTS_HI(hwe->mmio_base).reg,
> > 			upper_32_bits(lrc_desc));
> > 	xe_mmio_write32(gt, RING_EXECLIST_CONTROL(hwe->mmio_base).reg,
> > 			EL_CTRL_LOAD);
> > diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
> > index 4b56c35b988d..23b9f120c258 100644
> > --- a/drivers/gpu/drm/xe/xe_hw_engine.c
> > +++ b/drivers/gpu/drm/xe/xe_hw_engine.c
> > @@ -528,10 +528,10 @@ void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p)
> > 		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg));
> > 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_LO: 0x%08x\n",
> > 		hw_engine_mmio_read32(hwe,
> 
> after this series and mine about register refactor we should probably go
> and fix the coding style in this function.

indeed

> 
> ... thankfully it was just this print that was wrong and the rest was
> correct.
> 
> Reviewed-by: Lucas De Marchi <lucas.demarchi@intel.com>

Thank you!
Since this was a small but needed fix I went already and merged this
patch individually already.

> 
> thanks
> Lucas De Marchi
> 
> > -					 RING_EXECLIST_SQ_CONTENTS(0).reg));
> > +					 RING_EXECLIST_SQ_CONTENTS_LO(0).reg));
> > 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_HI: 0x%08x\n",
> > 		hw_engine_mmio_read32(hwe,
> > -					 RING_EXECLIST_SQ_CONTENTS(0).reg) + 4);
> > +					 RING_EXECLIST_SQ_CONTENTS_HI(0).reg));
> > 	drm_printf(p, "\tRING_EXECLIST_CONTROL: 0x%08x\n",
> > 		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg));
> > 
> > -- 
> > 2.39.2
> > 

[PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Rodrigo Vivi]

The goal is to use devcoredump infrastructure to report error states
captured at the crash time.

The error state will contain useful information for GPU hang debug, such
as INSTDONE registers and the current buffers getting executed, as well
as any other information that helps user space and allow later replays of
the error.

The proposal here is to avoid a Xe only error_state like i915 and use
a standard dev_coredump infrastructure to expose the error state.

For our own case, the data is only useful if it is a snapshot of the
time when the GPU crash has happened, since we reset the GPU immediately
after and the registers might have changed. So the proposal here is to
have an internal snapshot to be printed out later.

Also, usually a subsequent GPU hang can be only a cause of the initial
one. So we only save the 'first' hang. The dev_coredump has a delayed
work queue where it remove the coredump and free all the data withing a
few moments of the error. When that happens we also reset our capture
state and allow further snapshots.

Right now this infra only print out the time of the hang. More information
will be migrated here on subsequent work. Also, in order to organize the
dump better, the goal is to propose dev_coredump changes itself to allow
multiple files and different controls. But for now we start Xe usage of
it without any dependency on dev_coredump core changes.

Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/Kconfig                |   1 +
 drivers/gpu/drm/xe/Makefile               |   1 +
 drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
 drivers/gpu/drm/xe/xe_device_types.h      |   4 +
 drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
 drivers/gpu/drm/xe/xe_pci.c               |   2 +
 8 files changed, 223 insertions(+)
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h

diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
index f6f3b491d162..d44794f99338 100644
--- a/drivers/gpu/drm/xe/Kconfig
+++ b/drivers/gpu/drm/xe/Kconfig
@@ -35,6 +35,7 @@ config DRM_XE
 	select DRM_TTM_HELPER
 	select DRM_SCHED
 	select MMU_NOTIFIER
+	select WANT_DEV_COREDUMP
 	help
 	  Experimental driver for Intel Xe series GPUs
 
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index ee4a95beec20..9d675f7c77aa 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -34,6 +34,7 @@ xe-y += xe_bb.o \
 	xe_bo.o \
 	xe_bo_evict.o \
 	xe_debugfs.o \
+	xe_devcoredump.o \
 	xe_device.o \
 	xe_dma_buf.o \
 	xe_engine.o \
diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
new file mode 100644
index 000000000000..d9531183f03a
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include "xe_devcoredump.h"
+#include "xe_devcoredump_types.h"
+
+#include <linux/devcoredump.h>
+#include <generated/utsrelease.h>
+
+#include "xe_engine.h"
+#include "xe_gt.h"
+
+/**
+ * DOC: Xe device coredump
+ *
+ * Devices overview:
+ * Xe uses dev_coredump infrastructure for exposing the crash errors in a
+ * standardized way.
+ * devcoredump exposes a temporary device under /sys/class/devcoredump/
+ * which is linked with our card device directly.
+ * The core dump can be accessed either from
+ * /sys/class/drm/card<n>/device/devcoredump/ or from
+ * /sys/class/devcoredump/devcd<m> where
+ * /sys/class/devcoredump/devcd<m>/failing_device is a link to
+ * /sys/class/drm/card<n>/device/.
+ *
+ * Snapshot at hang:
+ * The 'data' file is printed with a drm_printer pointer at devcoredump read
+ * time. For this reason, we need to take snapshots from when the hang has
+ * happened, and not only when the user is reading the file. Otherwise the
+ * information is outdated since the resets might have happened in between.
+ *
+ * 'First' failure snapshot:
+ * In general, the first hang is the most critical one since the following hangs
+ * can be a consequence of the initial hang. For this reason we only take the
+ * snapshot of the 'first' failure and ignore subsequent calls of this function,
+ * at least while the coredump device is alive. Dev_coredump has a delayed work
+ * queue that will eventually delete the device and free all the dump
+ * information. At this time we also clear the faulty_engine and allow the next
+ * hang capture.
+ */
+
+static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
+				   size_t count, void *data, size_t datalen)
+{
+	struct xe_devcoredump *coredump = data;
+	struct xe_devcoredump_snapshot *ss;
+	struct drm_printer p;
+	struct drm_print_iterator iter;
+	struct timespec64 ts;
+
+	iter.data = buffer;
+	iter.offset = 0;
+	iter.start = offset;
+	iter.remain = count;
+
+	mutex_lock(&coredump->lock);
+
+	ss = &coredump->snapshot;
+	p = drm_coredump_printer(&iter);
+
+	drm_printf(&p, "**** Xe Device Coredump ****\n");
+	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
+	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
+
+	ts = ktime_to_timespec64(ss->snapshot_time);
+	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
+	ts = ktime_to_timespec64(ss->boot_time);
+	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
+	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
+	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
+
+	mutex_unlock(&coredump->lock);
+
+	return count - iter.remain;
+}
+
+static void xe_devcoredump_free(void *data)
+{
+	struct xe_devcoredump *coredump = data;
+	struct xe_device *xe = container_of(coredump, struct xe_device,
+					    devcoredump);
+	mutex_lock(&coredump->lock);
+
+	coredump->faulty_engine = NULL;
+	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
+
+	mutex_unlock(&coredump->lock);
+}
+
+static void devcoredump_snapshot(struct xe_devcoredump *coredump)
+{
+	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
+
+	lockdep_assert_held(&coredump->lock);
+	ss->snapshot_time = ktime_get_real();
+	ss->boot_time = ktime_get_boottime();
+}
+
+/**
+ * xe_devcoredump - Take the required snapshots and initialize coredump device.
+ * @e: The faulty xe_engine, where the issue was detected.
+ *
+ * This function should be called at the crash time. It is skipped if we still
+ * have the core dump device available with the information of the 'first'
+ * snapshot.
+ */
+void xe_devcoredump(struct xe_engine *e)
+{
+	struct xe_device *xe = gt_to_xe(e->gt);
+	struct xe_devcoredump *coredump = &xe->devcoredump;
+
+	mutex_lock(&coredump->lock);
+	if (coredump->faulty_engine) {
+		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
+		mutex_unlock(&coredump->lock);
+		return;
+	}
+	coredump->faulty_engine = e;
+	devcoredump_snapshot(coredump);
+	mutex_unlock(&coredump->lock);
+
+	drm_info(&xe->drm, "Xe device coredump has been created\n");
+	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
+
+	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
+		      xe_devcoredump_read, xe_devcoredump_free);
+}
+
+/**
+ * xe_devcoredump_init - Initialize xe_devcoredump.
+ * @xe: Xe device.
+ *
+ * This function should be called at the probe so the mutex lock can be
+ * initialized.
+ */
+void xe_devcoredump_init(struct xe_device *xe)
+{
+	struct xe_devcoredump *coredump = &xe->devcoredump;
+
+	mutex_init(&coredump->lock);
+}
diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
new file mode 100644
index 000000000000..30941d2e554b
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_devcoredump.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_DEVCOREDUMP_H_
+#define _XE_DEVCOREDUMP_H_
+
+struct xe_device;
+struct xe_engine;
+
+void xe_devcoredump_init(struct xe_device *xe);
+
+#ifdef CONFIG_DEV_COREDUMP
+void xe_devcoredump(struct xe_engine *e);
+#else
+static inline void xe_devcoredump(struct xe_engine *e)
+{
+}
+#endif
+
+#endif
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
new file mode 100644
index 000000000000..3f395fa9104e
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_DEVCOREDUMP_TYPES_H_
+#define _XE_DEVCOREDUMP_TYPES_H_
+
+#include <linux/ktime.h>
+#include <linux/mutex.h>
+
+struct xe_device;
+
+/**
+ * struct xe_devcoredump_snapshot - Crash snapshot
+ *
+ * This struct contains all the useful information quickly captured at the time
+ * of the crash. So, any subsequent reads of the coredump points to a data that
+ * shows the state of the GPU of when the issue has happened.
+ */
+struct xe_devcoredump_snapshot {
+	/** @snapshot_time:  Time of this capture. */
+	ktime_t snapshot_time;
+	/** @boot_time:  Relative boot time so the uptime can be calculated. */
+	ktime_t boot_time;
+};
+
+/**
+ * struct xe_devcoredump - Xe devcoredump main structure
+ *
+ * This struct represents the live and active dev_coredump node.
+ * It is created/populated at the time of a crash/error. Then it
+ * is read later when user access the device coredump data file
+ * for reading the information.
+ */
+struct xe_devcoredump {
+	/** @xe: Xe device. */
+	struct xe_device *xe;
+	/** @falty_engine: Engine where the crash/error happened. */
+	struct xe_engine *faulty_engine;
+	/** @lock: Protects data from races between capture and read out. */
+	struct mutex lock;
+	/** @snapshot: Snapshot is captured at time of the first crash */
+	struct xe_devcoredump_snapshot snapshot;
+};
+
+#endif
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 1cb404e48aaa..2a0995824692 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -12,6 +12,7 @@
 #include <drm/drm_file.h>
 #include <drm/ttm/ttm_device.h>
 
+#include "xe_devcoredump_types.h"
 #include "xe_gt_types.h"
 #include "xe_platform_types.h"
 #include "xe_step_types.h"
@@ -55,6 +56,9 @@ struct xe_device {
 	/** @drm: drm device */
 	struct drm_device drm;
 
+	/** @devcoredump: device coredump */
+	struct xe_devcoredump devcoredump;
+
 	/** @info: device info */
 	struct intel_device_info {
 		/** @graphics_name: graphics IP name */
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index e857013070b9..231fb4145297 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -14,6 +14,7 @@
 #include <drm/drm_managed.h>
 
 #include "regs/xe_lrc_layout.h"
+#include "xe_devcoredump.h"
 #include "xe_device.h"
 #include "xe_engine.h"
 #include "xe_force_wake.h"
@@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
 		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
 			 xe_sched_job_seqno(job), e->guc->id, e->flags);
 		simple_error_capture(e);
+		xe_devcoredump(e);
 	} else {
 		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
 			 xe_sched_job_seqno(job), e->guc->id, e->flags);
diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
index e512e8b69831..1d496210b580 100644
--- a/drivers/gpu/drm/xe/xe_pci.c
+++ b/drivers/gpu/drm/xe/xe_pci.c
@@ -16,6 +16,7 @@
 
 #include "regs/xe_regs.h"
 #include "regs/xe_gt_regs.h"
+#include "xe_devcoredump.h"
 #include "xe_device.h"
 #include "xe_display.h"
 #include "xe_drv.h"
@@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		return err;
 	}
 
+	xe_devcoredump_init(xe);
 	xe_pm_runtime_init(xe);
 
 	return 0;
-- 
2.39.2

[Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Rodrigo Vivi]

The goal is to use devcoredump infrastructure to report error states
captured at the crash time.

The error state will contain useful information for GPU hang debug, such
as INSTDONE registers and the current buffers getting executed, as well
as any other information that helps user space and allow later replays of
the error.

The proposal here is to avoid a Xe only error_state like i915 and use
a standard dev_coredump infrastructure to expose the error state.

For our own case, the data is only useful if it is a snapshot of the
time when the GPU crash has happened, since we reset the GPU immediately
after and the registers might have changed. So the proposal here is to
have an internal snapshot to be printed out later.

Also, usually a subsequent GPU hang can be only a cause of the initial
one. So we only save the 'first' hang. The dev_coredump has a delayed
work queue where it remove the coredump and free all the data withing a
few moments of the error. When that happens we also reset our capture
state and allow further snapshots.

Right now this infra only print out the time of the hang. More information
will be migrated here on subsequent work. Also, in order to organize the
dump better, the goal is to propose dev_coredump changes itself to allow
multiple files and different controls. But for now we start Xe usage of
it without any dependency on dev_coredump core changes.

Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/Kconfig                |   1 +
 drivers/gpu/drm/xe/Makefile               |   1 +
 drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
 drivers/gpu/drm/xe/xe_device_types.h      |   4 +
 drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
 drivers/gpu/drm/xe/xe_pci.c               |   2 +
 8 files changed, 223 insertions(+)
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
 create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h

diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
index f6f3b491d162..d44794f99338 100644
--- a/drivers/gpu/drm/xe/Kconfig
+++ b/drivers/gpu/drm/xe/Kconfig
@@ -35,6 +35,7 @@ config DRM_XE
 	select DRM_TTM_HELPER
 	select DRM_SCHED
 	select MMU_NOTIFIER
+	select WANT_DEV_COREDUMP
 	help
 	  Experimental driver for Intel Xe series GPUs
 
diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
index ee4a95beec20..9d675f7c77aa 100644
--- a/drivers/gpu/drm/xe/Makefile
+++ b/drivers/gpu/drm/xe/Makefile
@@ -34,6 +34,7 @@ xe-y += xe_bb.o \
 	xe_bo.o \
 	xe_bo_evict.o \
 	xe_debugfs.o \
+	xe_devcoredump.o \
 	xe_device.o \
 	xe_dma_buf.o \
 	xe_engine.o \
diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
new file mode 100644
index 000000000000..d9531183f03a
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -0,0 +1,144 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include "xe_devcoredump.h"
+#include "xe_devcoredump_types.h"
+
+#include <linux/devcoredump.h>
+#include <generated/utsrelease.h>
+
+#include "xe_engine.h"
+#include "xe_gt.h"
+
+/**
+ * DOC: Xe device coredump
+ *
+ * Devices overview:
+ * Xe uses dev_coredump infrastructure for exposing the crash errors in a
+ * standardized way.
+ * devcoredump exposes a temporary device under /sys/class/devcoredump/
+ * which is linked with our card device directly.
+ * The core dump can be accessed either from
+ * /sys/class/drm/card<n>/device/devcoredump/ or from
+ * /sys/class/devcoredump/devcd<m> where
+ * /sys/class/devcoredump/devcd<m>/failing_device is a link to
+ * /sys/class/drm/card<n>/device/.
+ *
+ * Snapshot at hang:
+ * The 'data' file is printed with a drm_printer pointer at devcoredump read
+ * time. For this reason, we need to take snapshots from when the hang has
+ * happened, and not only when the user is reading the file. Otherwise the
+ * information is outdated since the resets might have happened in between.
+ *
+ * 'First' failure snapshot:
+ * In general, the first hang is the most critical one since the following hangs
+ * can be a consequence of the initial hang. For this reason we only take the
+ * snapshot of the 'first' failure and ignore subsequent calls of this function,
+ * at least while the coredump device is alive. Dev_coredump has a delayed work
+ * queue that will eventually delete the device and free all the dump
+ * information. At this time we also clear the faulty_engine and allow the next
+ * hang capture.
+ */
+
+static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
+				   size_t count, void *data, size_t datalen)
+{
+	struct xe_devcoredump *coredump = data;
+	struct xe_devcoredump_snapshot *ss;
+	struct drm_printer p;
+	struct drm_print_iterator iter;
+	struct timespec64 ts;
+
+	iter.data = buffer;
+	iter.offset = 0;
+	iter.start = offset;
+	iter.remain = count;
+
+	mutex_lock(&coredump->lock);
+
+	ss = &coredump->snapshot;
+	p = drm_coredump_printer(&iter);
+
+	drm_printf(&p, "**** Xe Device Coredump ****\n");
+	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
+	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
+
+	ts = ktime_to_timespec64(ss->snapshot_time);
+	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
+	ts = ktime_to_timespec64(ss->boot_time);
+	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
+	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
+	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
+
+	mutex_unlock(&coredump->lock);
+
+	return count - iter.remain;
+}
+
+static void xe_devcoredump_free(void *data)
+{
+	struct xe_devcoredump *coredump = data;
+	struct xe_device *xe = container_of(coredump, struct xe_device,
+					    devcoredump);
+	mutex_lock(&coredump->lock);
+
+	coredump->faulty_engine = NULL;
+	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
+
+	mutex_unlock(&coredump->lock);
+}
+
+static void devcoredump_snapshot(struct xe_devcoredump *coredump)
+{
+	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
+
+	lockdep_assert_held(&coredump->lock);
+	ss->snapshot_time = ktime_get_real();
+	ss->boot_time = ktime_get_boottime();
+}
+
+/**
+ * xe_devcoredump - Take the required snapshots and initialize coredump device.
+ * @e: The faulty xe_engine, where the issue was detected.
+ *
+ * This function should be called at the crash time. It is skipped if we still
+ * have the core dump device available with the information of the 'first'
+ * snapshot.
+ */
+void xe_devcoredump(struct xe_engine *e)
+{
+	struct xe_device *xe = gt_to_xe(e->gt);
+	struct xe_devcoredump *coredump = &xe->devcoredump;
+
+	mutex_lock(&coredump->lock);
+	if (coredump->faulty_engine) {
+		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
+		mutex_unlock(&coredump->lock);
+		return;
+	}
+	coredump->faulty_engine = e;
+	devcoredump_snapshot(coredump);
+	mutex_unlock(&coredump->lock);
+
+	drm_info(&xe->drm, "Xe device coredump has been created\n");
+	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
+
+	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
+		      xe_devcoredump_read, xe_devcoredump_free);
+}
+
+/**
+ * xe_devcoredump_init - Initialize xe_devcoredump.
+ * @xe: Xe device.
+ *
+ * This function should be called at the probe so the mutex lock can be
+ * initialized.
+ */
+void xe_devcoredump_init(struct xe_device *xe)
+{
+	struct xe_devcoredump *coredump = &xe->devcoredump;
+
+	mutex_init(&coredump->lock);
+}
diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
new file mode 100644
index 000000000000..30941d2e554b
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_devcoredump.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_DEVCOREDUMP_H_
+#define _XE_DEVCOREDUMP_H_
+
+struct xe_device;
+struct xe_engine;
+
+void xe_devcoredump_init(struct xe_device *xe);
+
+#ifdef CONFIG_DEV_COREDUMP
+void xe_devcoredump(struct xe_engine *e);
+#else
+static inline void xe_devcoredump(struct xe_engine *e)
+{
+}
+#endif
+
+#endif
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
new file mode 100644
index 000000000000..3f395fa9104e
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_DEVCOREDUMP_TYPES_H_
+#define _XE_DEVCOREDUMP_TYPES_H_
+
+#include <linux/ktime.h>
+#include <linux/mutex.h>
+
+struct xe_device;
+
+/**
+ * struct xe_devcoredump_snapshot - Crash snapshot
+ *
+ * This struct contains all the useful information quickly captured at the time
+ * of the crash. So, any subsequent reads of the coredump points to a data that
+ * shows the state of the GPU of when the issue has happened.
+ */
+struct xe_devcoredump_snapshot {
+	/** @snapshot_time:  Time of this capture. */
+	ktime_t snapshot_time;
+	/** @boot_time:  Relative boot time so the uptime can be calculated. */
+	ktime_t boot_time;
+};
+
+/**
+ * struct xe_devcoredump - Xe devcoredump main structure
+ *
+ * This struct represents the live and active dev_coredump node.
+ * It is created/populated at the time of a crash/error. Then it
+ * is read later when user access the device coredump data file
+ * for reading the information.
+ */
+struct xe_devcoredump {
+	/** @xe: Xe device. */
+	struct xe_device *xe;
+	/** @falty_engine: Engine where the crash/error happened. */
+	struct xe_engine *faulty_engine;
+	/** @lock: Protects data from races between capture and read out. */
+	struct mutex lock;
+	/** @snapshot: Snapshot is captured at time of the first crash */
+	struct xe_devcoredump_snapshot snapshot;
+};
+
+#endif
diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
index 1cb404e48aaa..2a0995824692 100644
--- a/drivers/gpu/drm/xe/xe_device_types.h
+++ b/drivers/gpu/drm/xe/xe_device_types.h
@@ -12,6 +12,7 @@
 #include <drm/drm_file.h>
 #include <drm/ttm/ttm_device.h>
 
+#include "xe_devcoredump_types.h"
 #include "xe_gt_types.h"
 #include "xe_platform_types.h"
 #include "xe_step_types.h"
@@ -55,6 +56,9 @@ struct xe_device {
 	/** @drm: drm device */
 	struct drm_device drm;
 
+	/** @devcoredump: device coredump */
+	struct xe_devcoredump devcoredump;
+
 	/** @info: device info */
 	struct intel_device_info {
 		/** @graphics_name: graphics IP name */
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index e857013070b9..231fb4145297 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -14,6 +14,7 @@
 #include <drm/drm_managed.h>
 
 #include "regs/xe_lrc_layout.h"
+#include "xe_devcoredump.h"
 #include "xe_device.h"
 #include "xe_engine.h"
 #include "xe_force_wake.h"
@@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
 		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
 			 xe_sched_job_seqno(job), e->guc->id, e->flags);
 		simple_error_capture(e);
+		xe_devcoredump(e);
 	} else {
 		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
 			 xe_sched_job_seqno(job), e->guc->id, e->flags);
diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
index e512e8b69831..1d496210b580 100644
--- a/drivers/gpu/drm/xe/xe_pci.c
+++ b/drivers/gpu/drm/xe/xe_pci.c
@@ -16,6 +16,7 @@
 
 #include "regs/xe_regs.h"
 #include "regs/xe_gt_regs.h"
+#include "xe_devcoredump.h"
 #include "xe_device.h"
 #include "xe_display.h"
 #include "xe_drv.h"
@@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		return err;
 	}
 
+	xe_devcoredump_init(xe);
 	xe_pm_runtime_init(xe);
 
 	return 0;
-- 
2.39.2

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Thomas Hellström]

On 4/26/23 22:57, Rodrigo Vivi wrote:
> The goal is to use devcoredump infrastructure to report error states
> captured at the crash time.
>
> The error state will contain useful information for GPU hang debug, such
> as INSTDONE registers and the current buffers getting executed, as well
> as any other information that helps user space and allow later replays of
> the error.
>
> The proposal here is to avoid a Xe only error_state like i915 and use
> a standard dev_coredump infrastructure to expose the error state.
>
> For our own case, the data is only useful if it is a snapshot of the
> time when the GPU crash has happened, since we reset the GPU immediately
> after and the registers might have changed. So the proposal here is to
> have an internal snapshot to be printed out later.
>
> Also, usually a subsequent GPU hang can be only a cause of the initial
> one. So we only save the 'first' hang. The dev_coredump has a delayed
> work queue where it remove the coredump and free all the data withing a
> few moments of the error. When that happens we also reset our capture
> state and allow further snapshots.
>
> Right now this infra only print out the time of the hang. More information
> will be migrated here on subsequent work. Also, in order to organize the
> dump better, the goal is to propose dev_coredump changes itself to allow
> multiple files and different controls. But for now we start Xe usage of
> it without any dependency on dev_coredump core changes.
>
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>   drivers/gpu/drm/xe/Kconfig                |   1 +
>   drivers/gpu/drm/xe/Makefile               |   1 +
>   drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
>   drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
>   drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
>   drivers/gpu/drm/xe/xe_device_types.h      |   4 +
>   drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
>   drivers/gpu/drm/xe/xe_pci.c               |   2 +
>   8 files changed, 223 insertions(+)
>   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
>   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
>   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
>
> diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> index f6f3b491d162..d44794f99338 100644
> --- a/drivers/gpu/drm/xe/Kconfig
> +++ b/drivers/gpu/drm/xe/Kconfig
> @@ -35,6 +35,7 @@ config DRM_XE
>   	select DRM_TTM_HELPER
>   	select DRM_SCHED
>   	select MMU_NOTIFIER
> +	select WANT_DEV_COREDUMP
>   	help
>   	  Experimental driver for Intel Xe series GPUs
>   
> diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> index ee4a95beec20..9d675f7c77aa 100644
> --- a/drivers/gpu/drm/xe/Makefile
> +++ b/drivers/gpu/drm/xe/Makefile
> @@ -34,6 +34,7 @@ xe-y += xe_bb.o \
>   	xe_bo.o \
>   	xe_bo_evict.o \
>   	xe_debugfs.o \
> +	xe_devcoredump.o \
>   	xe_device.o \
>   	xe_dma_buf.o \
>   	xe_engine.o \
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> new file mode 100644
> index 000000000000..d9531183f03a
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -0,0 +1,144 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2023 Intel Corporation
> + */
> +
> +#include "xe_devcoredump.h"
> +#include "xe_devcoredump_types.h"
> +
> +#include <linux/devcoredump.h>
> +#include <generated/utsrelease.h>
> +
> +#include "xe_engine.h"
> +#include "xe_gt.h"
> +
> +/**
> + * DOC: Xe device coredump
> + *
> + * Devices overview:
> + * Xe uses dev_coredump infrastructure for exposing the crash errors in a
> + * standardized way.
> + * devcoredump exposes a temporary device under /sys/class/devcoredump/
> + * which is linked with our card device directly.
> + * The core dump can be accessed either from
> + * /sys/class/drm/card<n>/device/devcoredump/ or from
> + * /sys/class/devcoredump/devcd<m> where
> + * /sys/class/devcoredump/devcd<m>/failing_device is a link to
> + * /sys/class/drm/card<n>/device/.
> + *
> + * Snapshot at hang:
> + * The 'data' file is printed with a drm_printer pointer at devcoredump read
> + * time. For this reason, we need to take snapshots from when the hang has
> + * happened, and not only when the user is reading the file. Otherwise the
> + * information is outdated since the resets might have happened in between.
> + *
> + * 'First' failure snapshot:
> + * In general, the first hang is the most critical one since the following hangs
> + * can be a consequence of the initial hang. For this reason we only take the
> + * snapshot of the 'first' failure and ignore subsequent calls of this function,
> + * at least while the coredump device is alive. Dev_coredump has a delayed work
> + * queue that will eventually delete the device and free all the dump
> + * information. At this time we also clear the faulty_engine and allow the next
> + * hang capture.
> + */
> +
> +static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> +				   size_t count, void *data, size_t datalen)
> +{
> +	struct xe_devcoredump *coredump = data;
> +	struct xe_devcoredump_snapshot *ss;
> +	struct drm_printer p;
> +	struct drm_print_iterator iter;
> +	struct timespec64 ts;
> +
> +	iter.data = buffer;
> +	iter.offset = 0;
> +	iter.start = offset;
> +	iter.remain = count;
> +
> +	mutex_lock(&coredump->lock);
> +
> +	ss = &coredump->snapshot;
> +	p = drm_coredump_printer(&iter);
> +
> +	drm_printf(&p, "**** Xe Device Coredump ****\n");
> +	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
> +	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
> +
> +	ts = ktime_to_timespec64(ss->snapshot_time);
> +	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> +	ts = ktime_to_timespec64(ss->boot_time);
> +	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> +	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
> +	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> +
> +	mutex_unlock(&coredump->lock);
> +
> +	return count - iter.remain;
> +}
> +
> +static void xe_devcoredump_free(void *data)
> +{
> +	struct xe_devcoredump *coredump = data;
> +	struct xe_device *xe = container_of(coredump, struct xe_device,
> +					    devcoredump);
> +	mutex_lock(&coredump->lock);
> +
> +	coredump->faulty_engine = NULL;
> +	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> +
> +	mutex_unlock(&coredump->lock);
> +}
> +
> +static void devcoredump_snapshot(struct xe_devcoredump *coredump)
> +{
> +	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> +
> +	lockdep_assert_held(&coredump->lock);
> +	ss->snapshot_time = ktime_get_real();
> +	ss->boot_time = ktime_get_boottime();
> +}
> +
> +/**
> + * xe_devcoredump - Take the required snapshots and initialize coredump device.
> + * @e: The faulty xe_engine, where the issue was detected.
> + *
> + * This function should be called at the crash time. It is skipped if we still
> + * have the core dump device available with the information of the 'first'
> + * snapshot.
> + */
> +void xe_devcoredump(struct xe_engine *e)
> +{
> +	struct xe_device *xe = gt_to_xe(e->gt);
> +	struct xe_devcoredump *coredump = &xe->devcoredump;

For !long running engines, this is the dma-fence signalling critical 
path, and since the drm_scheduler has not yet been properly annotated, 
we should probably annotate that here, to avoid seeing strange deadlocks 
during coredumps....

/Thomas



> +
> +	mutex_lock(&coredump->lock);
> +	if (coredump->faulty_engine) {
> +		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
> +		mutex_unlock(&coredump->lock);
> +		return;
> +	}
> +	coredump->faulty_engine = e;
> +	devcoredump_snapshot(coredump);
> +	mutex_unlock(&coredump->lock);
> +
> +	drm_info(&xe->drm, "Xe device coredump has been created\n");
> +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
> +
> +	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
> +		      xe_devcoredump_read, xe_devcoredump_free);
> +}
> +
> +/**
> + * xe_devcoredump_init - Initialize xe_devcoredump.
> + * @xe: Xe device.
> + *
> + * This function should be called at the probe so the mutex lock can be
> + * initialized.
> + */
> +void xe_devcoredump_init(struct xe_device *xe)
> +{
> +	struct xe_devcoredump *coredump = &xe->devcoredump;
> +
> +	mutex_init(&coredump->lock);
> +}
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
> new file mode 100644
> index 000000000000..30941d2e554b
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.h
> @@ -0,0 +1,22 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2023 Intel Corporation
> + */
> +
> +#ifndef _XE_DEVCOREDUMP_H_
> +#define _XE_DEVCOREDUMP_H_
> +
> +struct xe_device;
> +struct xe_engine;
> +
> +void xe_devcoredump_init(struct xe_device *xe);
> +
> +#ifdef CONFIG_DEV_COREDUMP
> +void xe_devcoredump(struct xe_engine *e);
> +#else
> +static inline void xe_devcoredump(struct xe_engine *e)
> +{
> +}
> +#endif
> +
> +#endif
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> new file mode 100644
> index 000000000000..3f395fa9104e
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> @@ -0,0 +1,47 @@
> +/* SPDX-License-Identifier: MIT */
> +/*
> + * Copyright © 2023 Intel Corporation
> + */
> +
> +#ifndef _XE_DEVCOREDUMP_TYPES_H_
> +#define _XE_DEVCOREDUMP_TYPES_H_
> +
> +#include <linux/ktime.h>
> +#include <linux/mutex.h>
> +
> +struct xe_device;
> +
> +/**
> + * struct xe_devcoredump_snapshot - Crash snapshot
> + *
> + * This struct contains all the useful information quickly captured at the time
> + * of the crash. So, any subsequent reads of the coredump points to a data that
> + * shows the state of the GPU of when the issue has happened.
> + */
> +struct xe_devcoredump_snapshot {
> +	/** @snapshot_time:  Time of this capture. */
> +	ktime_t snapshot_time;
> +	/** @boot_time:  Relative boot time so the uptime can be calculated. */
> +	ktime_t boot_time;
> +};
> +
> +/**
> + * struct xe_devcoredump - Xe devcoredump main structure
> + *
> + * This struct represents the live and active dev_coredump node.
> + * It is created/populated at the time of a crash/error. Then it
> + * is read later when user access the device coredump data file
> + * for reading the information.
> + */
> +struct xe_devcoredump {
> +	/** @xe: Xe device. */
> +	struct xe_device *xe;
> +	/** @falty_engine: Engine where the crash/error happened. */
> +	struct xe_engine *faulty_engine;
> +	/** @lock: Protects data from races between capture and read out. */
> +	struct mutex lock;
> +	/** @snapshot: Snapshot is captured at time of the first crash */
> +	struct xe_devcoredump_snapshot snapshot;
> +};
> +
> +#endif
> diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> index 1cb404e48aaa..2a0995824692 100644
> --- a/drivers/gpu/drm/xe/xe_device_types.h
> +++ b/drivers/gpu/drm/xe/xe_device_types.h
> @@ -12,6 +12,7 @@
>   #include <drm/drm_file.h>
>   #include <drm/ttm/ttm_device.h>
>   
> +#include "xe_devcoredump_types.h"
>   #include "xe_gt_types.h"
>   #include "xe_platform_types.h"
>   #include "xe_step_types.h"
> @@ -55,6 +56,9 @@ struct xe_device {
>   	/** @drm: drm device */
>   	struct drm_device drm;
>   
> +	/** @devcoredump: device coredump */
> +	struct xe_devcoredump devcoredump;
> +
>   	/** @info: device info */
>   	struct intel_device_info {
>   		/** @graphics_name: graphics IP name */
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> index e857013070b9..231fb4145297 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> @@ -14,6 +14,7 @@
>   #include <drm/drm_managed.h>
>   
>   #include "regs/xe_lrc_layout.h"
> +#include "xe_devcoredump.h"
>   #include "xe_device.h"
>   #include "xe_engine.h"
>   #include "xe_force_wake.h"
> @@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
>   		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
>   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
>   		simple_error_capture(e);
> +		xe_devcoredump(e);
>   	} else {
>   		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
>   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
> index e512e8b69831..1d496210b580 100644
> --- a/drivers/gpu/drm/xe/xe_pci.c
> +++ b/drivers/gpu/drm/xe/xe_pci.c
> @@ -16,6 +16,7 @@
>   
>   #include "regs/xe_regs.h"
>   #include "regs/xe_gt_regs.h"
> +#include "xe_devcoredump.h"
>   #include "xe_device.h"
>   #include "xe_display.h"
>   #include "xe_drv.h"
> @@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
>   		return err;
>   	}
>   
> +	xe_devcoredump_init(xe);
>   	xe_pm_runtime_init(xe);
>   
>   	return 0;

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Matthew Brost]

On Thu, Apr 27, 2023 at 10:28:13AM +0200, Thomas Hellström wrote:
> 
> On 4/26/23 22:57, Rodrigo Vivi wrote:
> > The goal is to use devcoredump infrastructure to report error states
> > captured at the crash time.
> > 
> > The error state will contain useful information for GPU hang debug, such
> > as INSTDONE registers and the current buffers getting executed, as well
> > as any other information that helps user space and allow later replays of
> > the error.
> > 
> > The proposal here is to avoid a Xe only error_state like i915 and use
> > a standard dev_coredump infrastructure to expose the error state.
> > 
> > For our own case, the data is only useful if it is a snapshot of the
> > time when the GPU crash has happened, since we reset the GPU immediately
> > after and the registers might have changed. So the proposal here is to
> > have an internal snapshot to be printed out later.
> > 
> > Also, usually a subsequent GPU hang can be only a cause of the initial
> > one. So we only save the 'first' hang. The dev_coredump has a delayed
> > work queue where it remove the coredump and free all the data withing a
> > few moments of the error. When that happens we also reset our capture
> > state and allow further snapshots.
> > 
> > Right now this infra only print out the time of the hang. More information
> > will be migrated here on subsequent work. Also, in order to organize the
> > dump better, the goal is to propose dev_coredump changes itself to allow
> > multiple files and different controls. But for now we start Xe usage of
> > it without any dependency on dev_coredump core changes.
> > 
> > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > ---
> >   drivers/gpu/drm/xe/Kconfig                |   1 +
> >   drivers/gpu/drm/xe/Makefile               |   1 +
> >   drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
> >   drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
> >   drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
> >   drivers/gpu/drm/xe/xe_device_types.h      |   4 +
> >   drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
> >   drivers/gpu/drm/xe/xe_pci.c               |   2 +
> >   8 files changed, 223 insertions(+)
> >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
> >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
> >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
> > 
> > diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> > index f6f3b491d162..d44794f99338 100644
> > --- a/drivers/gpu/drm/xe/Kconfig
> > +++ b/drivers/gpu/drm/xe/Kconfig
> > @@ -35,6 +35,7 @@ config DRM_XE
> >   	select DRM_TTM_HELPER
> >   	select DRM_SCHED
> >   	select MMU_NOTIFIER
> > +	select WANT_DEV_COREDUMP
> >   	help
> >   	  Experimental driver for Intel Xe series GPUs
> > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> > index ee4a95beec20..9d675f7c77aa 100644
> > --- a/drivers/gpu/drm/xe/Makefile
> > +++ b/drivers/gpu/drm/xe/Makefile
> > @@ -34,6 +34,7 @@ xe-y += xe_bb.o \
> >   	xe_bo.o \
> >   	xe_bo_evict.o \
> >   	xe_debugfs.o \
> > +	xe_devcoredump.o \
> >   	xe_device.o \
> >   	xe_dma_buf.o \
> >   	xe_engine.o \
> > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > new file mode 100644
> > index 000000000000..d9531183f03a
> > --- /dev/null
> > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > @@ -0,0 +1,144 @@
> > +// SPDX-License-Identifier: MIT
> > +/*
> > + * Copyright © 2023 Intel Corporation
> > + */
> > +
> > +#include "xe_devcoredump.h"
> > +#include "xe_devcoredump_types.h"
> > +
> > +#include <linux/devcoredump.h>
> > +#include <generated/utsrelease.h>
> > +
> > +#include "xe_engine.h"
> > +#include "xe_gt.h"
> > +
> > +/**
> > + * DOC: Xe device coredump
> > + *
> > + * Devices overview:
> > + * Xe uses dev_coredump infrastructure for exposing the crash errors in a
> > + * standardized way.
> > + * devcoredump exposes a temporary device under /sys/class/devcoredump/
> > + * which is linked with our card device directly.
> > + * The core dump can be accessed either from
> > + * /sys/class/drm/card<n>/device/devcoredump/ or from
> > + * /sys/class/devcoredump/devcd<m> where
> > + * /sys/class/devcoredump/devcd<m>/failing_device is a link to
> > + * /sys/class/drm/card<n>/device/.
> > + *
> > + * Snapshot at hang:
> > + * The 'data' file is printed with a drm_printer pointer at devcoredump read
> > + * time. For this reason, we need to take snapshots from when the hang has
> > + * happened, and not only when the user is reading the file. Otherwise the
> > + * information is outdated since the resets might have happened in between.
> > + *
> > + * 'First' failure snapshot:
> > + * In general, the first hang is the most critical one since the following hangs
> > + * can be a consequence of the initial hang. For this reason we only take the
> > + * snapshot of the 'first' failure and ignore subsequent calls of this function,
> > + * at least while the coredump device is alive. Dev_coredump has a delayed work
> > + * queue that will eventually delete the device and free all the dump
> > + * information. At this time we also clear the faulty_engine and allow the next
> > + * hang capture.
> > + */
> > +
> > +static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > +				   size_t count, void *data, size_t datalen)
> > +{
> > +	struct xe_devcoredump *coredump = data;
> > +	struct xe_devcoredump_snapshot *ss;
> > +	struct drm_printer p;
> > +	struct drm_print_iterator iter;
> > +	struct timespec64 ts;
> > +
> > +	iter.data = buffer;
> > +	iter.offset = 0;
> > +	iter.start = offset;
> > +	iter.remain = count;
> > +
> > +	mutex_lock(&coredump->lock);
> > +
> > +	ss = &coredump->snapshot;
> > +	p = drm_coredump_printer(&iter);
> > +
> > +	drm_printf(&p, "**** Xe Device Coredump ****\n");
> > +	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
> > +	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
> > +
> > +	ts = ktime_to_timespec64(ss->snapshot_time);
> > +	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > +	ts = ktime_to_timespec64(ss->boot_time);
> > +	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > +	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
> > +	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > +
> > +	mutex_unlock(&coredump->lock);
> > +
> > +	return count - iter.remain;
> > +}
> > +
> > +static void xe_devcoredump_free(void *data)
> > +{
> > +	struct xe_devcoredump *coredump = data;
> > +	struct xe_device *xe = container_of(coredump, struct xe_device,
> > +					    devcoredump);
> > +	mutex_lock(&coredump->lock);
> > +
> > +	coredump->faulty_engine = NULL;
> > +	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> > +
> > +	mutex_unlock(&coredump->lock);
> > +}
> > +
> > +static void devcoredump_snapshot(struct xe_devcoredump *coredump)
> > +{
> > +	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> > +
> > +	lockdep_assert_held(&coredump->lock);
> > +	ss->snapshot_time = ktime_get_real();
> > +	ss->boot_time = ktime_get_boottime();
> > +}
> > +
> > +/**
> > + * xe_devcoredump - Take the required snapshots and initialize coredump device.
> > + * @e: The faulty xe_engine, where the issue was detected.
> > + *
> > + * This function should be called at the crash time. It is skipped if we still
> > + * have the core dump device available with the information of the 'first'
> > + * snapshot.
> > + */
> > +void xe_devcoredump(struct xe_engine *e)
> > +{
> > +	struct xe_device *xe = gt_to_xe(e->gt);
> > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> 
> For !long running engines, this is the dma-fence signalling critical path,
> and since the drm_scheduler has not yet been properly annotated, we should
> probably annotate that here, to avoid seeing strange deadlocks during
> coredumps....
> 
> /Thomas
>

+1

Matt
 
> 
> 
> > +
> > +	mutex_lock(&coredump->lock);
> > +	if (coredump->faulty_engine) {
> > +		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
> > +		mutex_unlock(&coredump->lock);
> > +		return;
> > +	}
> > +	coredump->faulty_engine = e;
> > +	devcoredump_snapshot(coredump);
> > +	mutex_unlock(&coredump->lock);
> > +
> > +	drm_info(&xe->drm, "Xe device coredump has been created\n");
> > +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
> > +
> > +	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
> > +		      xe_devcoredump_read, xe_devcoredump_free);
> > +}
> > +
> > +/**
> > + * xe_devcoredump_init - Initialize xe_devcoredump.
> > + * @xe: Xe device.
> > + *
> > + * This function should be called at the probe so the mutex lock can be
> > + * initialized.
> > + */
> > +void xe_devcoredump_init(struct xe_device *xe)
> > +{
> > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > +
> > +	mutex_init(&coredump->lock);
> > +}
> > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
> > new file mode 100644
> > index 000000000000..30941d2e554b
> > --- /dev/null
> > +++ b/drivers/gpu/drm/xe/xe_devcoredump.h
> > @@ -0,0 +1,22 @@
> > +/* SPDX-License-Identifier: MIT */
> > +/*
> > + * Copyright © 2023 Intel Corporation
> > + */
> > +
> > +#ifndef _XE_DEVCOREDUMP_H_
> > +#define _XE_DEVCOREDUMP_H_
> > +
> > +struct xe_device;
> > +struct xe_engine;
> > +
> > +void xe_devcoredump_init(struct xe_device *xe);
> > +
> > +#ifdef CONFIG_DEV_COREDUMP
> > +void xe_devcoredump(struct xe_engine *e);
> > +#else
> > +static inline void xe_devcoredump(struct xe_engine *e)
> > +{
> > +}
> > +#endif
> > +
> > +#endif
> > diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > new file mode 100644
> > index 000000000000..3f395fa9104e
> > --- /dev/null
> > +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > @@ -0,0 +1,47 @@
> > +/* SPDX-License-Identifier: MIT */
> > +/*
> > + * Copyright © 2023 Intel Corporation
> > + */
> > +
> > +#ifndef _XE_DEVCOREDUMP_TYPES_H_
> > +#define _XE_DEVCOREDUMP_TYPES_H_
> > +
> > +#include <linux/ktime.h>
> > +#include <linux/mutex.h>
> > +
> > +struct xe_device;
> > +
> > +/**
> > + * struct xe_devcoredump_snapshot - Crash snapshot
> > + *
> > + * This struct contains all the useful information quickly captured at the time
> > + * of the crash. So, any subsequent reads of the coredump points to a data that
> > + * shows the state of the GPU of when the issue has happened.
> > + */
> > +struct xe_devcoredump_snapshot {
> > +	/** @snapshot_time:  Time of this capture. */
> > +	ktime_t snapshot_time;
> > +	/** @boot_time:  Relative boot time so the uptime can be calculated. */
> > +	ktime_t boot_time;
> > +};
> > +
> > +/**
> > + * struct xe_devcoredump - Xe devcoredump main structure
> > + *
> > + * This struct represents the live and active dev_coredump node.
> > + * It is created/populated at the time of a crash/error. Then it
> > + * is read later when user access the device coredump data file
> > + * for reading the information.
> > + */
> > +struct xe_devcoredump {
> > +	/** @xe: Xe device. */
> > +	struct xe_device *xe;
> > +	/** @falty_engine: Engine where the crash/error happened. */
> > +	struct xe_engine *faulty_engine;
> > +	/** @lock: Protects data from races between capture and read out. */
> > +	struct mutex lock;
> > +	/** @snapshot: Snapshot is captured at time of the first crash */
> > +	struct xe_devcoredump_snapshot snapshot;
> > +};
> > +
> > +#endif
> > diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> > index 1cb404e48aaa..2a0995824692 100644
> > --- a/drivers/gpu/drm/xe/xe_device_types.h
> > +++ b/drivers/gpu/drm/xe/xe_device_types.h
> > @@ -12,6 +12,7 @@
> >   #include <drm/drm_file.h>
> >   #include <drm/ttm/ttm_device.h>
> > +#include "xe_devcoredump_types.h"
> >   #include "xe_gt_types.h"
> >   #include "xe_platform_types.h"
> >   #include "xe_step_types.h"
> > @@ -55,6 +56,9 @@ struct xe_device {
> >   	/** @drm: drm device */
> >   	struct drm_device drm;
> > +	/** @devcoredump: device coredump */
> > +	struct xe_devcoredump devcoredump;
> > +
> >   	/** @info: device info */
> >   	struct intel_device_info {
> >   		/** @graphics_name: graphics IP name */
> > diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> > index e857013070b9..231fb4145297 100644
> > --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> > +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> > @@ -14,6 +14,7 @@
> >   #include <drm/drm_managed.h>
> >   #include "regs/xe_lrc_layout.h"
> > +#include "xe_devcoredump.h"
> >   #include "xe_device.h"
> >   #include "xe_engine.h"
> >   #include "xe_force_wake.h"
> > @@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
> >   		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
> >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> >   		simple_error_capture(e);
> > +		xe_devcoredump(e);
> >   	} else {
> >   		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
> >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
> > index e512e8b69831..1d496210b580 100644
> > --- a/drivers/gpu/drm/xe/xe_pci.c
> > +++ b/drivers/gpu/drm/xe/xe_pci.c
> > @@ -16,6 +16,7 @@
> >   #include "regs/xe_regs.h"
> >   #include "regs/xe_gt_regs.h"
> > +#include "xe_devcoredump.h"
> >   #include "xe_device.h"
> >   #include "xe_display.h"
> >   #include "xe_drv.h"
> > @@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
> >   		return err;
> >   	}
> > +	xe_devcoredump_init(xe);
> >   	xe_pm_runtime_init(xe);
> >   	return 0;

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Rodrigo Vivi]

On Tue, May 02, 2023 at 07:57:02AM +0000, Matthew Brost wrote:
> On Thu, Apr 27, 2023 at 10:28:13AM +0200, Thomas Hellström wrote:
> > 
> > On 4/26/23 22:57, Rodrigo Vivi wrote:
> > > The goal is to use devcoredump infrastructure to report error states
> > > captured at the crash time.
> > > 
> > > The error state will contain useful information for GPU hang debug, such
> > > as INSTDONE registers and the current buffers getting executed, as well
> > > as any other information that helps user space and allow later replays of
> > > the error.
> > > 
> > > The proposal here is to avoid a Xe only error_state like i915 and use
> > > a standard dev_coredump infrastructure to expose the error state.
> > > 
> > > For our own case, the data is only useful if it is a snapshot of the
> > > time when the GPU crash has happened, since we reset the GPU immediately
> > > after and the registers might have changed. So the proposal here is to
> > > have an internal snapshot to be printed out later.
> > > 
> > > Also, usually a subsequent GPU hang can be only a cause of the initial
> > > one. So we only save the 'first' hang. The dev_coredump has a delayed
> > > work queue where it remove the coredump and free all the data withing a
> > > few moments of the error. When that happens we also reset our capture
> > > state and allow further snapshots.
> > > 
> > > Right now this infra only print out the time of the hang. More information
> > > will be migrated here on subsequent work. Also, in order to organize the
> > > dump better, the goal is to propose dev_coredump changes itself to allow
> > > multiple files and different controls. But for now we start Xe usage of
> > > it without any dependency on dev_coredump core changes.
> > > 
> > > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > ---
> > >   drivers/gpu/drm/xe/Kconfig                |   1 +
> > >   drivers/gpu/drm/xe/Makefile               |   1 +
> > >   drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
> > >   drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
> > >   drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
> > >   drivers/gpu/drm/xe/xe_device_types.h      |   4 +
> > >   drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
> > >   drivers/gpu/drm/xe/xe_pci.c               |   2 +
> > >   8 files changed, 223 insertions(+)
> > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
> > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
> > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > 
> > > diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> > > index f6f3b491d162..d44794f99338 100644
> > > --- a/drivers/gpu/drm/xe/Kconfig
> > > +++ b/drivers/gpu/drm/xe/Kconfig
> > > @@ -35,6 +35,7 @@ config DRM_XE
> > >   	select DRM_TTM_HELPER
> > >   	select DRM_SCHED
> > >   	select MMU_NOTIFIER
> > > +	select WANT_DEV_COREDUMP
> > >   	help
> > >   	  Experimental driver for Intel Xe series GPUs
> > > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> > > index ee4a95beec20..9d675f7c77aa 100644
> > > --- a/drivers/gpu/drm/xe/Makefile
> > > +++ b/drivers/gpu/drm/xe/Makefile
> > > @@ -34,6 +34,7 @@ xe-y += xe_bb.o \
> > >   	xe_bo.o \
> > >   	xe_bo_evict.o \
> > >   	xe_debugfs.o \
> > > +	xe_devcoredump.o \
> > >   	xe_device.o \
> > >   	xe_dma_buf.o \
> > >   	xe_engine.o \
> > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > new file mode 100644
> > > index 000000000000..d9531183f03a
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > @@ -0,0 +1,144 @@
> > > +// SPDX-License-Identifier: MIT
> > > +/*
> > > + * Copyright © 2023 Intel Corporation
> > > + */
> > > +
> > > +#include "xe_devcoredump.h"
> > > +#include "xe_devcoredump_types.h"
> > > +
> > > +#include <linux/devcoredump.h>
> > > +#include <generated/utsrelease.h>
> > > +
> > > +#include "xe_engine.h"
> > > +#include "xe_gt.h"
> > > +
> > > +/**
> > > + * DOC: Xe device coredump
> > > + *
> > > + * Devices overview:
> > > + * Xe uses dev_coredump infrastructure for exposing the crash errors in a
> > > + * standardized way.
> > > + * devcoredump exposes a temporary device under /sys/class/devcoredump/
> > > + * which is linked with our card device directly.
> > > + * The core dump can be accessed either from
> > > + * /sys/class/drm/card<n>/device/devcoredump/ or from
> > > + * /sys/class/devcoredump/devcd<m> where
> > > + * /sys/class/devcoredump/devcd<m>/failing_device is a link to
> > > + * /sys/class/drm/card<n>/device/.
> > > + *
> > > + * Snapshot at hang:
> > > + * The 'data' file is printed with a drm_printer pointer at devcoredump read
> > > + * time. For this reason, we need to take snapshots from when the hang has
> > > + * happened, and not only when the user is reading the file. Otherwise the
> > > + * information is outdated since the resets might have happened in between.
> > > + *
> > > + * 'First' failure snapshot:
> > > + * In general, the first hang is the most critical one since the following hangs
> > > + * can be a consequence of the initial hang. For this reason we only take the
> > > + * snapshot of the 'first' failure and ignore subsequent calls of this function,
> > > + * at least while the coredump device is alive. Dev_coredump has a delayed work
> > > + * queue that will eventually delete the device and free all the dump
> > > + * information. At this time we also clear the faulty_engine and allow the next
> > > + * hang capture.
> > > + */
> > > +
> > > +static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > > +				   size_t count, void *data, size_t datalen)
> > > +{
> > > +	struct xe_devcoredump *coredump = data;
> > > +	struct xe_devcoredump_snapshot *ss;
> > > +	struct drm_printer p;
> > > +	struct drm_print_iterator iter;
> > > +	struct timespec64 ts;
> > > +
> > > +	iter.data = buffer;
> > > +	iter.offset = 0;
> > > +	iter.start = offset;
> > > +	iter.remain = count;
> > > +
> > > +	mutex_lock(&coredump->lock);
> > > +
> > > +	ss = &coredump->snapshot;
> > > +	p = drm_coredump_printer(&iter);
> > > +
> > > +	drm_printf(&p, "**** Xe Device Coredump ****\n");
> > > +	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
> > > +	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
> > > +
> > > +	ts = ktime_to_timespec64(ss->snapshot_time);
> > > +	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > +	ts = ktime_to_timespec64(ss->boot_time);
> > > +	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > +	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
> > > +	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > +
> > > +	mutex_unlock(&coredump->lock);
> > > +
> > > +	return count - iter.remain;
> > > +}
> > > +
> > > +static void xe_devcoredump_free(void *data)
> > > +{
> > > +	struct xe_devcoredump *coredump = data;
> > > +	struct xe_device *xe = container_of(coredump, struct xe_device,
> > > +					    devcoredump);
> > > +	mutex_lock(&coredump->lock);
> > > +
> > > +	coredump->faulty_engine = NULL;
> > > +	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> > > +
> > > +	mutex_unlock(&coredump->lock);
> > > +}
> > > +
> > > +static void devcoredump_snapshot(struct xe_devcoredump *coredump)
> > > +{
> > > +	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> > > +
> > > +	lockdep_assert_held(&coredump->lock);
> > > +	ss->snapshot_time = ktime_get_real();
> > > +	ss->boot_time = ktime_get_boottime();
> > > +}
> > > +
> > > +/**
> > > + * xe_devcoredump - Take the required snapshots and initialize coredump device.
> > > + * @e: The faulty xe_engine, where the issue was detected.
> > > + *
> > > + * This function should be called at the crash time. It is skipped if we still
> > > + * have the core dump device available with the information of the 'first'
> > > + * snapshot.
> > > + */
> > > +void xe_devcoredump(struct xe_engine *e)
> > > +{
> > > +	struct xe_device *xe = gt_to_xe(e->gt);
> > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > 
> > For !long running engines, this is the dma-fence signalling critical path,
> > and since the drm_scheduler has not yet been properly annotated, we should
> > probably annotate that here, to avoid seeing strange deadlocks during
> > coredumps....
> > 
> > /Thomas
> >
> 
> +1

Just to confirm we are in the same page here, do you guys mean move
dma_fence_begin_signalling() annotation here?

or which annotation am I missing?

> 
> Matt
>  
> > 
> > 
> > > +
> > > +	mutex_lock(&coredump->lock);
> > > +	if (coredump->faulty_engine) {
> > > +		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
> > > +		mutex_unlock(&coredump->lock);
> > > +		return;
> > > +	}
> > > +	coredump->faulty_engine = e;
> > > +	devcoredump_snapshot(coredump);
> > > +	mutex_unlock(&coredump->lock);
> > > +
> > > +	drm_info(&xe->drm, "Xe device coredump has been created\n");
> > > +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
> > > +
> > > +	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
> > > +		      xe_devcoredump_read, xe_devcoredump_free);
> > > +}
> > > +
> > > +/**
> > > + * xe_devcoredump_init - Initialize xe_devcoredump.
> > > + * @xe: Xe device.
> > > + *
> > > + * This function should be called at the probe so the mutex lock can be
> > > + * initialized.
> > > + */
> > > +void xe_devcoredump_init(struct xe_device *xe)
> > > +{
> > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > > +
> > > +	mutex_init(&coredump->lock);
> > > +}
> > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > new file mode 100644
> > > index 000000000000..30941d2e554b
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > @@ -0,0 +1,22 @@
> > > +/* SPDX-License-Identifier: MIT */
> > > +/*
> > > + * Copyright © 2023 Intel Corporation
> > > + */
> > > +
> > > +#ifndef _XE_DEVCOREDUMP_H_
> > > +#define _XE_DEVCOREDUMP_H_
> > > +
> > > +struct xe_device;
> > > +struct xe_engine;
> > > +
> > > +void xe_devcoredump_init(struct xe_device *xe);
> > > +
> > > +#ifdef CONFIG_DEV_COREDUMP
> > > +void xe_devcoredump(struct xe_engine *e);
> > > +#else
> > > +static inline void xe_devcoredump(struct xe_engine *e)
> > > +{
> > > +}
> > > +#endif
> > > +
> > > +#endif
> > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > new file mode 100644
> > > index 000000000000..3f395fa9104e
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > @@ -0,0 +1,47 @@
> > > +/* SPDX-License-Identifier: MIT */
> > > +/*
> > > + * Copyright © 2023 Intel Corporation
> > > + */
> > > +
> > > +#ifndef _XE_DEVCOREDUMP_TYPES_H_
> > > +#define _XE_DEVCOREDUMP_TYPES_H_
> > > +
> > > +#include <linux/ktime.h>
> > > +#include <linux/mutex.h>
> > > +
> > > +struct xe_device;
> > > +
> > > +/**
> > > + * struct xe_devcoredump_snapshot - Crash snapshot
> > > + *
> > > + * This struct contains all the useful information quickly captured at the time
> > > + * of the crash. So, any subsequent reads of the coredump points to a data that
> > > + * shows the state of the GPU of when the issue has happened.
> > > + */
> > > +struct xe_devcoredump_snapshot {
> > > +	/** @snapshot_time:  Time of this capture. */
> > > +	ktime_t snapshot_time;
> > > +	/** @boot_time:  Relative boot time so the uptime can be calculated. */
> > > +	ktime_t boot_time;
> > > +};
> > > +
> > > +/**
> > > + * struct xe_devcoredump - Xe devcoredump main structure
> > > + *
> > > + * This struct represents the live and active dev_coredump node.
> > > + * It is created/populated at the time of a crash/error. Then it
> > > + * is read later when user access the device coredump data file
> > > + * for reading the information.
> > > + */
> > > +struct xe_devcoredump {
> > > +	/** @xe: Xe device. */
> > > +	struct xe_device *xe;
> > > +	/** @falty_engine: Engine where the crash/error happened. */
> > > +	struct xe_engine *faulty_engine;
> > > +	/** @lock: Protects data from races between capture and read out. */
> > > +	struct mutex lock;
> > > +	/** @snapshot: Snapshot is captured at time of the first crash */
> > > +	struct xe_devcoredump_snapshot snapshot;
> > > +};
> > > +
> > > +#endif
> > > diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> > > index 1cb404e48aaa..2a0995824692 100644
> > > --- a/drivers/gpu/drm/xe/xe_device_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_device_types.h
> > > @@ -12,6 +12,7 @@
> > >   #include <drm/drm_file.h>
> > >   #include <drm/ttm/ttm_device.h>
> > > +#include "xe_devcoredump_types.h"
> > >   #include "xe_gt_types.h"
> > >   #include "xe_platform_types.h"
> > >   #include "xe_step_types.h"
> > > @@ -55,6 +56,9 @@ struct xe_device {
> > >   	/** @drm: drm device */
> > >   	struct drm_device drm;
> > > +	/** @devcoredump: device coredump */
> > > +	struct xe_devcoredump devcoredump;
> > > +
> > >   	/** @info: device info */
> > >   	struct intel_device_info {
> > >   		/** @graphics_name: graphics IP name */
> > > diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > index e857013070b9..231fb4145297 100644
> > > --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> > > +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > @@ -14,6 +14,7 @@
> > >   #include <drm/drm_managed.h>
> > >   #include "regs/xe_lrc_layout.h"
> > > +#include "xe_devcoredump.h"
> > >   #include "xe_device.h"
> > >   #include "xe_engine.h"
> > >   #include "xe_force_wake.h"
> > > @@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
> > >   		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
> > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > >   		simple_error_capture(e);
> > > +		xe_devcoredump(e);
> > >   	} else {
> > >   		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
> > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > > diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
> > > index e512e8b69831..1d496210b580 100644
> > > --- a/drivers/gpu/drm/xe/xe_pci.c
> > > +++ b/drivers/gpu/drm/xe/xe_pci.c
> > > @@ -16,6 +16,7 @@
> > >   #include "regs/xe_regs.h"
> > >   #include "regs/xe_gt_regs.h"
> > > +#include "xe_devcoredump.h"
> > >   #include "xe_device.h"
> > >   #include "xe_display.h"
> > >   #include "xe_drv.h"
> > > @@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
> > >   		return err;
> > >   	}
> > > +	xe_devcoredump_init(xe);
> > >   	xe_pm_runtime_init(xe);
> > >   	return 0;

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Rodrigo Vivi]

On Tue, May 02, 2023 at 07:57:02AM +0000, Matthew Brost wrote:
> On Thu, Apr 27, 2023 at 10:28:13AM +0200, Thomas Hellström wrote:
> > 
> > On 4/26/23 22:57, Rodrigo Vivi wrote:
> > > The goal is to use devcoredump infrastructure to report error states
> > > captured at the crash time.
> > > 
> > > The error state will contain useful information for GPU hang debug, such
> > > as INSTDONE registers and the current buffers getting executed, as well
> > > as any other information that helps user space and allow later replays of
> > > the error.
> > > 
> > > The proposal here is to avoid a Xe only error_state like i915 and use
> > > a standard dev_coredump infrastructure to expose the error state.
> > > 
> > > For our own case, the data is only useful if it is a snapshot of the
> > > time when the GPU crash has happened, since we reset the GPU immediately
> > > after and the registers might have changed. So the proposal here is to
> > > have an internal snapshot to be printed out later.
> > > 
> > > Also, usually a subsequent GPU hang can be only a cause of the initial
> > > one. So we only save the 'first' hang. The dev_coredump has a delayed
> > > work queue where it remove the coredump and free all the data withing a
> > > few moments of the error. When that happens we also reset our capture
> > > state and allow further snapshots.
> > > 
> > > Right now this infra only print out the time of the hang. More information
> > > will be migrated here on subsequent work. Also, in order to organize the
> > > dump better, the goal is to propose dev_coredump changes itself to allow
> > > multiple files and different controls. But for now we start Xe usage of
> > > it without any dependency on dev_coredump core changes.
> > > 
> > > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > ---
> > >   drivers/gpu/drm/xe/Kconfig                |   1 +
> > >   drivers/gpu/drm/xe/Makefile               |   1 +
> > >   drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
> > >   drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
> > >   drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
> > >   drivers/gpu/drm/xe/xe_device_types.h      |   4 +
> > >   drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
> > >   drivers/gpu/drm/xe/xe_pci.c               |   2 +
> > >   8 files changed, 223 insertions(+)
> > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
> > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
> > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > 
> > > diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> > > index f6f3b491d162..d44794f99338 100644
> > > --- a/drivers/gpu/drm/xe/Kconfig
> > > +++ b/drivers/gpu/drm/xe/Kconfig
> > > @@ -35,6 +35,7 @@ config DRM_XE
> > >   	select DRM_TTM_HELPER
> > >   	select DRM_SCHED
> > >   	select MMU_NOTIFIER
> > > +	select WANT_DEV_COREDUMP
> > >   	help
> > >   	  Experimental driver for Intel Xe series GPUs
> > > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> > > index ee4a95beec20..9d675f7c77aa 100644
> > > --- a/drivers/gpu/drm/xe/Makefile
> > > +++ b/drivers/gpu/drm/xe/Makefile
> > > @@ -34,6 +34,7 @@ xe-y += xe_bb.o \
> > >   	xe_bo.o \
> > >   	xe_bo_evict.o \
> > >   	xe_debugfs.o \
> > > +	xe_devcoredump.o \
> > >   	xe_device.o \
> > >   	xe_dma_buf.o \
> > >   	xe_engine.o \
> > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > new file mode 100644
> > > index 000000000000..d9531183f03a
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > @@ -0,0 +1,144 @@
> > > +// SPDX-License-Identifier: MIT
> > > +/*
> > > + * Copyright © 2023 Intel Corporation
> > > + */
> > > +
> > > +#include "xe_devcoredump.h"
> > > +#include "xe_devcoredump_types.h"
> > > +
> > > +#include <linux/devcoredump.h>
> > > +#include <generated/utsrelease.h>
> > > +
> > > +#include "xe_engine.h"
> > > +#include "xe_gt.h"
> > > +
> > > +/**
> > > + * DOC: Xe device coredump
> > > + *
> > > + * Devices overview:
> > > + * Xe uses dev_coredump infrastructure for exposing the crash errors in a
> > > + * standardized way.
> > > + * devcoredump exposes a temporary device under /sys/class/devcoredump/
> > > + * which is linked with our card device directly.
> > > + * The core dump can be accessed either from
> > > + * /sys/class/drm/card<n>/device/devcoredump/ or from
> > > + * /sys/class/devcoredump/devcd<m> where
> > > + * /sys/class/devcoredump/devcd<m>/failing_device is a link to
> > > + * /sys/class/drm/card<n>/device/.
> > > + *
> > > + * Snapshot at hang:
> > > + * The 'data' file is printed with a drm_printer pointer at devcoredump read
> > > + * time. For this reason, we need to take snapshots from when the hang has
> > > + * happened, and not only when the user is reading the file. Otherwise the
> > > + * information is outdated since the resets might have happened in between.
> > > + *
> > > + * 'First' failure snapshot:
> > > + * In general, the first hang is the most critical one since the following hangs
> > > + * can be a consequence of the initial hang. For this reason we only take the
> > > + * snapshot of the 'first' failure and ignore subsequent calls of this function,
> > > + * at least while the coredump device is alive. Dev_coredump has a delayed work
> > > + * queue that will eventually delete the device and free all the dump
> > > + * information. At this time we also clear the faulty_engine and allow the next
> > > + * hang capture.
> > > + */
> > > +
> > > +static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > > +				   size_t count, void *data, size_t datalen)
> > > +{
> > > +	struct xe_devcoredump *coredump = data;
> > > +	struct xe_devcoredump_snapshot *ss;
> > > +	struct drm_printer p;
> > > +	struct drm_print_iterator iter;
> > > +	struct timespec64 ts;
> > > +
> > > +	iter.data = buffer;
> > > +	iter.offset = 0;
> > > +	iter.start = offset;
> > > +	iter.remain = count;
> > > +
> > > +	mutex_lock(&coredump->lock);
> > > +
> > > +	ss = &coredump->snapshot;
> > > +	p = drm_coredump_printer(&iter);
> > > +
> > > +	drm_printf(&p, "**** Xe Device Coredump ****\n");
> > > +	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
> > > +	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
> > > +
> > > +	ts = ktime_to_timespec64(ss->snapshot_time);
> > > +	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > +	ts = ktime_to_timespec64(ss->boot_time);
> > > +	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > +	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
> > > +	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > +
> > > +	mutex_unlock(&coredump->lock);
> > > +
> > > +	return count - iter.remain;
> > > +}
> > > +
> > > +static void xe_devcoredump_free(void *data)
> > > +{
> > > +	struct xe_devcoredump *coredump = data;
> > > +	struct xe_device *xe = container_of(coredump, struct xe_device,
> > > +					    devcoredump);
> > > +	mutex_lock(&coredump->lock);
> > > +
> > > +	coredump->faulty_engine = NULL;
> > > +	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> > > +
> > > +	mutex_unlock(&coredump->lock);
> > > +}
> > > +
> > > +static void devcoredump_snapshot(struct xe_devcoredump *coredump)
> > > +{
> > > +	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> > > +
> > > +	lockdep_assert_held(&coredump->lock);
> > > +	ss->snapshot_time = ktime_get_real();
> > > +	ss->boot_time = ktime_get_boottime();
> > > +}
> > > +
> > > +/**
> > > + * xe_devcoredump - Take the required snapshots and initialize coredump device.
> > > + * @e: The faulty xe_engine, where the issue was detected.
> > > + *
> > > + * This function should be called at the crash time. It is skipped if we still
> > > + * have the core dump device available with the information of the 'first'
> > > + * snapshot.
> > > + */
> > > +void xe_devcoredump(struct xe_engine *e)
> > > +{
> > > +	struct xe_device *xe = gt_to_xe(e->gt);
> > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > 
> > For !long running engines, this is the dma-fence signalling critical path,
> > and since the drm_scheduler has not yet been properly annotated, we should
> > probably annotate that here, to avoid seeing strange deadlocks during
> > coredumps....
> > 
> > /Thomas
> >
> 
> +1

Just to confirm we are in the same page here, do you guys mean move
dma_fence_begin_signalling() annotation here?

or which annotation am I missing?

> 
> Matt
>  
> > 
> > 
> > > +
> > > +	mutex_lock(&coredump->lock);
> > > +	if (coredump->faulty_engine) {
> > > +		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
> > > +		mutex_unlock(&coredump->lock);
> > > +		return;
> > > +	}
> > > +	coredump->faulty_engine = e;
> > > +	devcoredump_snapshot(coredump);
> > > +	mutex_unlock(&coredump->lock);
> > > +
> > > +	drm_info(&xe->drm, "Xe device coredump has been created\n");
> > > +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
> > > +
> > > +	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
> > > +		      xe_devcoredump_read, xe_devcoredump_free);
> > > +}
> > > +
> > > +/**
> > > + * xe_devcoredump_init - Initialize xe_devcoredump.
> > > + * @xe: Xe device.
> > > + *
> > > + * This function should be called at the probe so the mutex lock can be
> > > + * initialized.
> > > + */
> > > +void xe_devcoredump_init(struct xe_device *xe)
> > > +{
> > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > > +
> > > +	mutex_init(&coredump->lock);
> > > +}
> > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > new file mode 100644
> > > index 000000000000..30941d2e554b
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > @@ -0,0 +1,22 @@
> > > +/* SPDX-License-Identifier: MIT */
> > > +/*
> > > + * Copyright © 2023 Intel Corporation
> > > + */
> > > +
> > > +#ifndef _XE_DEVCOREDUMP_H_
> > > +#define _XE_DEVCOREDUMP_H_
> > > +
> > > +struct xe_device;
> > > +struct xe_engine;
> > > +
> > > +void xe_devcoredump_init(struct xe_device *xe);
> > > +
> > > +#ifdef CONFIG_DEV_COREDUMP
> > > +void xe_devcoredump(struct xe_engine *e);
> > > +#else
> > > +static inline void xe_devcoredump(struct xe_engine *e)
> > > +{
> > > +}
> > > +#endif
> > > +
> > > +#endif
> > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > new file mode 100644
> > > index 000000000000..3f395fa9104e
> > > --- /dev/null
> > > +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > @@ -0,0 +1,47 @@
> > > +/* SPDX-License-Identifier: MIT */
> > > +/*
> > > + * Copyright © 2023 Intel Corporation
> > > + */
> > > +
> > > +#ifndef _XE_DEVCOREDUMP_TYPES_H_
> > > +#define _XE_DEVCOREDUMP_TYPES_H_
> > > +
> > > +#include <linux/ktime.h>
> > > +#include <linux/mutex.h>
> > > +
> > > +struct xe_device;
> > > +
> > > +/**
> > > + * struct xe_devcoredump_snapshot - Crash snapshot
> > > + *
> > > + * This struct contains all the useful information quickly captured at the time
> > > + * of the crash. So, any subsequent reads of the coredump points to a data that
> > > + * shows the state of the GPU of when the issue has happened.
> > > + */
> > > +struct xe_devcoredump_snapshot {
> > > +	/** @snapshot_time:  Time of this capture. */
> > > +	ktime_t snapshot_time;
> > > +	/** @boot_time:  Relative boot time so the uptime can be calculated. */
> > > +	ktime_t boot_time;
> > > +};
> > > +
> > > +/**
> > > + * struct xe_devcoredump - Xe devcoredump main structure
> > > + *
> > > + * This struct represents the live and active dev_coredump node.
> > > + * It is created/populated at the time of a crash/error. Then it
> > > + * is read later when user access the device coredump data file
> > > + * for reading the information.
> > > + */
> > > +struct xe_devcoredump {
> > > +	/** @xe: Xe device. */
> > > +	struct xe_device *xe;
> > > +	/** @falty_engine: Engine where the crash/error happened. */
> > > +	struct xe_engine *faulty_engine;
> > > +	/** @lock: Protects data from races between capture and read out. */
> > > +	struct mutex lock;
> > > +	/** @snapshot: Snapshot is captured at time of the first crash */
> > > +	struct xe_devcoredump_snapshot snapshot;
> > > +};
> > > +
> > > +#endif
> > > diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> > > index 1cb404e48aaa..2a0995824692 100644
> > > --- a/drivers/gpu/drm/xe/xe_device_types.h
> > > +++ b/drivers/gpu/drm/xe/xe_device_types.h
> > > @@ -12,6 +12,7 @@
> > >   #include <drm/drm_file.h>
> > >   #include <drm/ttm/ttm_device.h>
> > > +#include "xe_devcoredump_types.h"
> > >   #include "xe_gt_types.h"
> > >   #include "xe_platform_types.h"
> > >   #include "xe_step_types.h"
> > > @@ -55,6 +56,9 @@ struct xe_device {
> > >   	/** @drm: drm device */
> > >   	struct drm_device drm;
> > > +	/** @devcoredump: device coredump */
> > > +	struct xe_devcoredump devcoredump;
> > > +
> > >   	/** @info: device info */
> > >   	struct intel_device_info {
> > >   		/** @graphics_name: graphics IP name */
> > > diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > index e857013070b9..231fb4145297 100644
> > > --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> > > +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > @@ -14,6 +14,7 @@
> > >   #include <drm/drm_managed.h>
> > >   #include "regs/xe_lrc_layout.h"
> > > +#include "xe_devcoredump.h"
> > >   #include "xe_device.h"
> > >   #include "xe_engine.h"
> > >   #include "xe_force_wake.h"
> > > @@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
> > >   		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
> > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > >   		simple_error_capture(e);
> > > +		xe_devcoredump(e);
> > >   	} else {
> > >   		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
> > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > > diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
> > > index e512e8b69831..1d496210b580 100644
> > > --- a/drivers/gpu/drm/xe/xe_pci.c
> > > +++ b/drivers/gpu/drm/xe/xe_pci.c
> > > @@ -16,6 +16,7 @@
> > >   #include "regs/xe_regs.h"
> > >   #include "regs/xe_gt_regs.h"
> > > +#include "xe_devcoredump.h"
> > >   #include "xe_device.h"
> > >   #include "xe_display.h"
> > >   #include "xe_drv.h"
> > > @@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
> > >   		return err;
> > >   	}
> > > +	xe_devcoredump_init(xe);
> > >   	xe_pm_runtime_init(xe);
> > >   	return 0;

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Matthew Brost]

On Tue, May 02, 2023 at 02:06:55PM -0400, Rodrigo Vivi wrote:
> On Tue, May 02, 2023 at 07:57:02AM +0000, Matthew Brost wrote:
> > On Thu, Apr 27, 2023 at 10:28:13AM +0200, Thomas Hellström wrote:
> > > 
> > > On 4/26/23 22:57, Rodrigo Vivi wrote:
> > > > The goal is to use devcoredump infrastructure to report error states
> > > > captured at the crash time.
> > > > 
> > > > The error state will contain useful information for GPU hang debug, such
> > > > as INSTDONE registers and the current buffers getting executed, as well
> > > > as any other information that helps user space and allow later replays of
> > > > the error.
> > > > 
> > > > The proposal here is to avoid a Xe only error_state like i915 and use
> > > > a standard dev_coredump infrastructure to expose the error state.
> > > > 
> > > > For our own case, the data is only useful if it is a snapshot of the
> > > > time when the GPU crash has happened, since we reset the GPU immediately
> > > > after and the registers might have changed. So the proposal here is to
> > > > have an internal snapshot to be printed out later.
> > > > 
> > > > Also, usually a subsequent GPU hang can be only a cause of the initial
> > > > one. So we only save the 'first' hang. The dev_coredump has a delayed
> > > > work queue where it remove the coredump and free all the data withing a
> > > > few moments of the error. When that happens we also reset our capture
> > > > state and allow further snapshots.
> > > > 
> > > > Right now this infra only print out the time of the hang. More information
> > > > will be migrated here on subsequent work. Also, in order to organize the
> > > > dump better, the goal is to propose dev_coredump changes itself to allow
> > > > multiple files and different controls. But for now we start Xe usage of
> > > > it without any dependency on dev_coredump core changes.
> > > > 
> > > > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > ---
> > > >   drivers/gpu/drm/xe/Kconfig                |   1 +
> > > >   drivers/gpu/drm/xe/Makefile               |   1 +
> > > >   drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
> > > >   drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
> > > >   drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
> > > >   drivers/gpu/drm/xe/xe_device_types.h      |   4 +
> > > >   drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
> > > >   drivers/gpu/drm/xe/xe_pci.c               |   2 +
> > > >   8 files changed, 223 insertions(+)
> > > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
> > > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
> > > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > > 
> > > > diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> > > > index f6f3b491d162..d44794f99338 100644
> > > > --- a/drivers/gpu/drm/xe/Kconfig
> > > > +++ b/drivers/gpu/drm/xe/Kconfig
> > > > @@ -35,6 +35,7 @@ config DRM_XE
> > > >   	select DRM_TTM_HELPER
> > > >   	select DRM_SCHED
> > > >   	select MMU_NOTIFIER
> > > > +	select WANT_DEV_COREDUMP
> > > >   	help
> > > >   	  Experimental driver for Intel Xe series GPUs
> > > > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> > > > index ee4a95beec20..9d675f7c77aa 100644
> > > > --- a/drivers/gpu/drm/xe/Makefile
> > > > +++ b/drivers/gpu/drm/xe/Makefile
> > > > @@ -34,6 +34,7 @@ xe-y += xe_bb.o \
> > > >   	xe_bo.o \
> > > >   	xe_bo_evict.o \
> > > >   	xe_debugfs.o \
> > > > +	xe_devcoredump.o \
> > > >   	xe_device.o \
> > > >   	xe_dma_buf.o \
> > > >   	xe_engine.o \
> > > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > > new file mode 100644
> > > > index 000000000000..d9531183f03a
> > > > --- /dev/null
> > > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > > @@ -0,0 +1,144 @@
> > > > +// SPDX-License-Identifier: MIT
> > > > +/*
> > > > + * Copyright © 2023 Intel Corporation
> > > > + */
> > > > +
> > > > +#include "xe_devcoredump.h"
> > > > +#include "xe_devcoredump_types.h"
> > > > +
> > > > +#include <linux/devcoredump.h>
> > > > +#include <generated/utsrelease.h>
> > > > +
> > > > +#include "xe_engine.h"
> > > > +#include "xe_gt.h"
> > > > +
> > > > +/**
> > > > + * DOC: Xe device coredump
> > > > + *
> > > > + * Devices overview:
> > > > + * Xe uses dev_coredump infrastructure for exposing the crash errors in a
> > > > + * standardized way.
> > > > + * devcoredump exposes a temporary device under /sys/class/devcoredump/
> > > > + * which is linked with our card device directly.
> > > > + * The core dump can be accessed either from
> > > > + * /sys/class/drm/card<n>/device/devcoredump/ or from
> > > > + * /sys/class/devcoredump/devcd<m> where
> > > > + * /sys/class/devcoredump/devcd<m>/failing_device is a link to
> > > > + * /sys/class/drm/card<n>/device/.
> > > > + *
> > > > + * Snapshot at hang:
> > > > + * The 'data' file is printed with a drm_printer pointer at devcoredump read
> > > > + * time. For this reason, we need to take snapshots from when the hang has
> > > > + * happened, and not only when the user is reading the file. Otherwise the
> > > > + * information is outdated since the resets might have happened in between.
> > > > + *
> > > > + * 'First' failure snapshot:
> > > > + * In general, the first hang is the most critical one since the following hangs
> > > > + * can be a consequence of the initial hang. For this reason we only take the
> > > > + * snapshot of the 'first' failure and ignore subsequent calls of this function,
> > > > + * at least while the coredump device is alive. Dev_coredump has a delayed work
> > > > + * queue that will eventually delete the device and free all the dump
> > > > + * information. At this time we also clear the faulty_engine and allow the next
> > > > + * hang capture.
> > > > + */
> > > > +
> > > > +static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > > > +				   size_t count, void *data, size_t datalen)
> > > > +{
> > > > +	struct xe_devcoredump *coredump = data;
> > > > +	struct xe_devcoredump_snapshot *ss;
> > > > +	struct drm_printer p;
> > > > +	struct drm_print_iterator iter;
> > > > +	struct timespec64 ts;
> > > > +
> > > > +	iter.data = buffer;
> > > > +	iter.offset = 0;
> > > > +	iter.start = offset;
> > > > +	iter.remain = count;
> > > > +
> > > > +	mutex_lock(&coredump->lock);
> > > > +
> > > > +	ss = &coredump->snapshot;
> > > > +	p = drm_coredump_printer(&iter);
> > > > +
> > > > +	drm_printf(&p, "**** Xe Device Coredump ****\n");
> > > > +	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
> > > > +	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
> > > > +
> > > > +	ts = ktime_to_timespec64(ss->snapshot_time);
> > > > +	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > > +	ts = ktime_to_timespec64(ss->boot_time);
> > > > +	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > > +	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
> > > > +	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > > +
> > > > +	mutex_unlock(&coredump->lock);
> > > > +
> > > > +	return count - iter.remain;
> > > > +}
> > > > +
> > > > +static void xe_devcoredump_free(void *data)
> > > > +{
> > > > +	struct xe_devcoredump *coredump = data;
> > > > +	struct xe_device *xe = container_of(coredump, struct xe_device,
> > > > +					    devcoredump);
> > > > +	mutex_lock(&coredump->lock);
> > > > +
> > > > +	coredump->faulty_engine = NULL;
> > > > +	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> > > > +
> > > > +	mutex_unlock(&coredump->lock);
> > > > +}
> > > > +
> > > > +static void devcoredump_snapshot(struct xe_devcoredump *coredump)
> > > > +{
> > > > +	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> > > > +
> > > > +	lockdep_assert_held(&coredump->lock);
> > > > +	ss->snapshot_time = ktime_get_real();
> > > > +	ss->boot_time = ktime_get_boottime();
> > > > +}
> > > > +
> > > > +/**
> > > > + * xe_devcoredump - Take the required snapshots and initialize coredump device.
> > > > + * @e: The faulty xe_engine, where the issue was detected.
> > > > + *
> > > > + * This function should be called at the crash time. It is skipped if we still
> > > > + * have the core dump device available with the information of the 'first'
> > > > + * snapshot.
> > > > + */
> > > > +void xe_devcoredump(struct xe_engine *e)
> > > > +{
> > > > +	struct xe_device *xe = gt_to_xe(e->gt);
> > > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > > 
> > > For !long running engines, this is the dma-fence signalling critical path,
> > > and since the drm_scheduler has not yet been properly annotated, we should
> > > probably annotate that here, to avoid seeing strange deadlocks during
> > > coredumps....
> > > 
> > > /Thomas
> > >
> > 
> > +1
> 
> Just to confirm we are in the same page here, do you guys mean move
> dma_fence_begin_signalling() annotation here?
> 
> or which annotation am I missing?
> 

Yea dma_fence_begin_signalling at the beginning of function.

Matt 

> > 
> > Matt
> >  
> > > 
> > > 
> > > > +
> > > > +	mutex_lock(&coredump->lock);
> > > > +	if (coredump->faulty_engine) {
> > > > +		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
> > > > +		mutex_unlock(&coredump->lock);
> > > > +		return;
> > > > +	}
> > > > +	coredump->faulty_engine = e;
> > > > +	devcoredump_snapshot(coredump);
> > > > +	mutex_unlock(&coredump->lock);
> > > > +
> > > > +	drm_info(&xe->drm, "Xe device coredump has been created\n");
> > > > +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
> > > > +
> > > > +	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
> > > > +		      xe_devcoredump_read, xe_devcoredump_free);
> > > > +}
> > > > +
> > > > +/**
> > > > + * xe_devcoredump_init - Initialize xe_devcoredump.
> > > > + * @xe: Xe device.
> > > > + *
> > > > + * This function should be called at the probe so the mutex lock can be
> > > > + * initialized.
> > > > + */
> > > > +void xe_devcoredump_init(struct xe_device *xe)
> > > > +{
> > > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > > > +
> > > > +	mutex_init(&coredump->lock);
> > > > +}
> > > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > > new file mode 100644
> > > > index 000000000000..30941d2e554b
> > > > --- /dev/null
> > > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > > @@ -0,0 +1,22 @@
> > > > +/* SPDX-License-Identifier: MIT */
> > > > +/*
> > > > + * Copyright © 2023 Intel Corporation
> > > > + */
> > > > +
> > > > +#ifndef _XE_DEVCOREDUMP_H_
> > > > +#define _XE_DEVCOREDUMP_H_
> > > > +
> > > > +struct xe_device;
> > > > +struct xe_engine;
> > > > +
> > > > +void xe_devcoredump_init(struct xe_device *xe);
> > > > +
> > > > +#ifdef CONFIG_DEV_COREDUMP
> > > > +void xe_devcoredump(struct xe_engine *e);
> > > > +#else
> > > > +static inline void xe_devcoredump(struct xe_engine *e)
> > > > +{
> > > > +}
> > > > +#endif
> > > > +
> > > > +#endif
> > > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > > new file mode 100644
> > > > index 000000000000..3f395fa9104e
> > > > --- /dev/null
> > > > +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > > @@ -0,0 +1,47 @@
> > > > +/* SPDX-License-Identifier: MIT */
> > > > +/*
> > > > + * Copyright © 2023 Intel Corporation
> > > > + */
> > > > +
> > > > +#ifndef _XE_DEVCOREDUMP_TYPES_H_
> > > > +#define _XE_DEVCOREDUMP_TYPES_H_
> > > > +
> > > > +#include <linux/ktime.h>
> > > > +#include <linux/mutex.h>
> > > > +
> > > > +struct xe_device;
> > > > +
> > > > +/**
> > > > + * struct xe_devcoredump_snapshot - Crash snapshot
> > > > + *
> > > > + * This struct contains all the useful information quickly captured at the time
> > > > + * of the crash. So, any subsequent reads of the coredump points to a data that
> > > > + * shows the state of the GPU of when the issue has happened.
> > > > + */
> > > > +struct xe_devcoredump_snapshot {
> > > > +	/** @snapshot_time:  Time of this capture. */
> > > > +	ktime_t snapshot_time;
> > > > +	/** @boot_time:  Relative boot time so the uptime can be calculated. */
> > > > +	ktime_t boot_time;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct xe_devcoredump - Xe devcoredump main structure
> > > > + *
> > > > + * This struct represents the live and active dev_coredump node.
> > > > + * It is created/populated at the time of a crash/error. Then it
> > > > + * is read later when user access the device coredump data file
> > > > + * for reading the information.
> > > > + */
> > > > +struct xe_devcoredump {
> > > > +	/** @xe: Xe device. */
> > > > +	struct xe_device *xe;
> > > > +	/** @falty_engine: Engine where the crash/error happened. */
> > > > +	struct xe_engine *faulty_engine;
> > > > +	/** @lock: Protects data from races between capture and read out. */
> > > > +	struct mutex lock;
> > > > +	/** @snapshot: Snapshot is captured at time of the first crash */
> > > > +	struct xe_devcoredump_snapshot snapshot;
> > > > +};
> > > > +
> > > > +#endif
> > > > diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> > > > index 1cb404e48aaa..2a0995824692 100644
> > > > --- a/drivers/gpu/drm/xe/xe_device_types.h
> > > > +++ b/drivers/gpu/drm/xe/xe_device_types.h
> > > > @@ -12,6 +12,7 @@
> > > >   #include <drm/drm_file.h>
> > > >   #include <drm/ttm/ttm_device.h>
> > > > +#include "xe_devcoredump_types.h"
> > > >   #include "xe_gt_types.h"
> > > >   #include "xe_platform_types.h"
> > > >   #include "xe_step_types.h"
> > > > @@ -55,6 +56,9 @@ struct xe_device {
> > > >   	/** @drm: drm device */
> > > >   	struct drm_device drm;
> > > > +	/** @devcoredump: device coredump */
> > > > +	struct xe_devcoredump devcoredump;
> > > > +
> > > >   	/** @info: device info */
> > > >   	struct intel_device_info {
> > > >   		/** @graphics_name: graphics IP name */
> > > > diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > > index e857013070b9..231fb4145297 100644
> > > > --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> > > > +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > > @@ -14,6 +14,7 @@
> > > >   #include <drm/drm_managed.h>
> > > >   #include "regs/xe_lrc_layout.h"
> > > > +#include "xe_devcoredump.h"
> > > >   #include "xe_device.h"
> > > >   #include "xe_engine.h"
> > > >   #include "xe_force_wake.h"
> > > > @@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
> > > >   		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
> > > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > > >   		simple_error_capture(e);
> > > > +		xe_devcoredump(e);
> > > >   	} else {
> > > >   		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
> > > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > > > diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
> > > > index e512e8b69831..1d496210b580 100644
> > > > --- a/drivers/gpu/drm/xe/xe_pci.c
> > > > +++ b/drivers/gpu/drm/xe/xe_pci.c
> > > > @@ -16,6 +16,7 @@
> > > >   #include "regs/xe_regs.h"
> > > >   #include "regs/xe_gt_regs.h"
> > > > +#include "xe_devcoredump.h"
> > > >   #include "xe_device.h"
> > > >   #include "xe_display.h"
> > > >   #include "xe_drv.h"
> > > > @@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
> > > >   		return err;
> > > >   	}
> > > > +	xe_devcoredump_init(xe);
> > > >   	xe_pm_runtime_init(xe);
> > > >   	return 0;

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Matthew Brost]

On Tue, May 02, 2023 at 02:06:55PM -0400, Rodrigo Vivi wrote:
> On Tue, May 02, 2023 at 07:57:02AM +0000, Matthew Brost wrote:
> > On Thu, Apr 27, 2023 at 10:28:13AM +0200, Thomas Hellström wrote:
> > > 
> > > On 4/26/23 22:57, Rodrigo Vivi wrote:
> > > > The goal is to use devcoredump infrastructure to report error states
> > > > captured at the crash time.
> > > > 
> > > > The error state will contain useful information for GPU hang debug, such
> > > > as INSTDONE registers and the current buffers getting executed, as well
> > > > as any other information that helps user space and allow later replays of
> > > > the error.
> > > > 
> > > > The proposal here is to avoid a Xe only error_state like i915 and use
> > > > a standard dev_coredump infrastructure to expose the error state.
> > > > 
> > > > For our own case, the data is only useful if it is a snapshot of the
> > > > time when the GPU crash has happened, since we reset the GPU immediately
> > > > after and the registers might have changed. So the proposal here is to
> > > > have an internal snapshot to be printed out later.
> > > > 
> > > > Also, usually a subsequent GPU hang can be only a cause of the initial
> > > > one. So we only save the 'first' hang. The dev_coredump has a delayed
> > > > work queue where it remove the coredump and free all the data withing a
> > > > few moments of the error. When that happens we also reset our capture
> > > > state and allow further snapshots.
> > > > 
> > > > Right now this infra only print out the time of the hang. More information
> > > > will be migrated here on subsequent work. Also, in order to organize the
> > > > dump better, the goal is to propose dev_coredump changes itself to allow
> > > > multiple files and different controls. But for now we start Xe usage of
> > > > it without any dependency on dev_coredump core changes.
> > > > 
> > > > Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> > > > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > > ---
> > > >   drivers/gpu/drm/xe/Kconfig                |   1 +
> > > >   drivers/gpu/drm/xe/Makefile               |   1 +
> > > >   drivers/gpu/drm/xe/xe_devcoredump.c       | 144 ++++++++++++++++++++++
> > > >   drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++++
> > > >   drivers/gpu/drm/xe/xe_devcoredump_types.h |  47 +++++++
> > > >   drivers/gpu/drm/xe/xe_device_types.h      |   4 +
> > > >   drivers/gpu/drm/xe/xe_guc_submit.c        |   2 +
> > > >   drivers/gpu/drm/xe/xe_pci.c               |   2 +
> > > >   8 files changed, 223 insertions(+)
> > > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
> > > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
> > > >   create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > > 
> > > > diff --git a/drivers/gpu/drm/xe/Kconfig b/drivers/gpu/drm/xe/Kconfig
> > > > index f6f3b491d162..d44794f99338 100644
> > > > --- a/drivers/gpu/drm/xe/Kconfig
> > > > +++ b/drivers/gpu/drm/xe/Kconfig
> > > > @@ -35,6 +35,7 @@ config DRM_XE
> > > >   	select DRM_TTM_HELPER
> > > >   	select DRM_SCHED
> > > >   	select MMU_NOTIFIER
> > > > +	select WANT_DEV_COREDUMP
> > > >   	help
> > > >   	  Experimental driver for Intel Xe series GPUs
> > > > diff --git a/drivers/gpu/drm/xe/Makefile b/drivers/gpu/drm/xe/Makefile
> > > > index ee4a95beec20..9d675f7c77aa 100644
> > > > --- a/drivers/gpu/drm/xe/Makefile
> > > > +++ b/drivers/gpu/drm/xe/Makefile
> > > > @@ -34,6 +34,7 @@ xe-y += xe_bb.o \
> > > >   	xe_bo.o \
> > > >   	xe_bo_evict.o \
> > > >   	xe_debugfs.o \
> > > > +	xe_devcoredump.o \
> > > >   	xe_device.o \
> > > >   	xe_dma_buf.o \
> > > >   	xe_engine.o \
> > > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > > new file mode 100644
> > > > index 000000000000..d9531183f03a
> > > > --- /dev/null
> > > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > > @@ -0,0 +1,144 @@
> > > > +// SPDX-License-Identifier: MIT
> > > > +/*
> > > > + * Copyright © 2023 Intel Corporation
> > > > + */
> > > > +
> > > > +#include "xe_devcoredump.h"
> > > > +#include "xe_devcoredump_types.h"
> > > > +
> > > > +#include <linux/devcoredump.h>
> > > > +#include <generated/utsrelease.h>
> > > > +
> > > > +#include "xe_engine.h"
> > > > +#include "xe_gt.h"
> > > > +
> > > > +/**
> > > > + * DOC: Xe device coredump
> > > > + *
> > > > + * Devices overview:
> > > > + * Xe uses dev_coredump infrastructure for exposing the crash errors in a
> > > > + * standardized way.
> > > > + * devcoredump exposes a temporary device under /sys/class/devcoredump/
> > > > + * which is linked with our card device directly.
> > > > + * The core dump can be accessed either from
> > > > + * /sys/class/drm/card<n>/device/devcoredump/ or from
> > > > + * /sys/class/devcoredump/devcd<m> where
> > > > + * /sys/class/devcoredump/devcd<m>/failing_device is a link to
> > > > + * /sys/class/drm/card<n>/device/.
> > > > + *
> > > > + * Snapshot at hang:
> > > > + * The 'data' file is printed with a drm_printer pointer at devcoredump read
> > > > + * time. For this reason, we need to take snapshots from when the hang has
> > > > + * happened, and not only when the user is reading the file. Otherwise the
> > > > + * information is outdated since the resets might have happened in between.
> > > > + *
> > > > + * 'First' failure snapshot:
> > > > + * In general, the first hang is the most critical one since the following hangs
> > > > + * can be a consequence of the initial hang. For this reason we only take the
> > > > + * snapshot of the 'first' failure and ignore subsequent calls of this function,
> > > > + * at least while the coredump device is alive. Dev_coredump has a delayed work
> > > > + * queue that will eventually delete the device and free all the dump
> > > > + * information. At this time we also clear the faulty_engine and allow the next
> > > > + * hang capture.
> > > > + */
> > > > +
> > > > +static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > > > +				   size_t count, void *data, size_t datalen)
> > > > +{
> > > > +	struct xe_devcoredump *coredump = data;
> > > > +	struct xe_devcoredump_snapshot *ss;
> > > > +	struct drm_printer p;
> > > > +	struct drm_print_iterator iter;
> > > > +	struct timespec64 ts;
> > > > +
> > > > +	iter.data = buffer;
> > > > +	iter.offset = 0;
> > > > +	iter.start = offset;
> > > > +	iter.remain = count;
> > > > +
> > > > +	mutex_lock(&coredump->lock);
> > > > +
> > > > +	ss = &coredump->snapshot;
> > > > +	p = drm_coredump_printer(&iter);
> > > > +
> > > > +	drm_printf(&p, "**** Xe Device Coredump ****\n");
> > > > +	drm_printf(&p, "kernel: " UTS_RELEASE "\n");
> > > > +	drm_printf(&p, "module: " KBUILD_MODNAME "\n");
> > > > +
> > > > +	ts = ktime_to_timespec64(ss->snapshot_time);
> > > > +	drm_printf(&p, "Snapshot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > > +	ts = ktime_to_timespec64(ss->boot_time);
> > > > +	drm_printf(&p, "Boot time: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > > +	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
> > > > +	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
> > > > +
> > > > +	mutex_unlock(&coredump->lock);
> > > > +
> > > > +	return count - iter.remain;
> > > > +}
> > > > +
> > > > +static void xe_devcoredump_free(void *data)
> > > > +{
> > > > +	struct xe_devcoredump *coredump = data;
> > > > +	struct xe_device *xe = container_of(coredump, struct xe_device,
> > > > +					    devcoredump);
> > > > +	mutex_lock(&coredump->lock);
> > > > +
> > > > +	coredump->faulty_engine = NULL;
> > > > +	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> > > > +
> > > > +	mutex_unlock(&coredump->lock);
> > > > +}
> > > > +
> > > > +static void devcoredump_snapshot(struct xe_devcoredump *coredump)
> > > > +{
> > > > +	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> > > > +
> > > > +	lockdep_assert_held(&coredump->lock);
> > > > +	ss->snapshot_time = ktime_get_real();
> > > > +	ss->boot_time = ktime_get_boottime();
> > > > +}
> > > > +
> > > > +/**
> > > > + * xe_devcoredump - Take the required snapshots and initialize coredump device.
> > > > + * @e: The faulty xe_engine, where the issue was detected.
> > > > + *
> > > > + * This function should be called at the crash time. It is skipped if we still
> > > > + * have the core dump device available with the information of the 'first'
> > > > + * snapshot.
> > > > + */
> > > > +void xe_devcoredump(struct xe_engine *e)
> > > > +{
> > > > +	struct xe_device *xe = gt_to_xe(e->gt);
> > > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > > 
> > > For !long running engines, this is the dma-fence signalling critical path,
> > > and since the drm_scheduler has not yet been properly annotated, we should
> > > probably annotate that here, to avoid seeing strange deadlocks during
> > > coredumps....
> > > 
> > > /Thomas
> > >
> > 
> > +1
> 
> Just to confirm we are in the same page here, do you guys mean move
> dma_fence_begin_signalling() annotation here?
> 
> or which annotation am I missing?
> 

Yea dma_fence_begin_signalling at the beginning of function.

Matt 

> > 
> > Matt
> >  
> > > 
> > > 
> > > > +
> > > > +	mutex_lock(&coredump->lock);
> > > > +	if (coredump->faulty_engine) {
> > > > +		drm_dbg(&xe->drm, "Multiple hangs are occuring, but only the first snapshot was taken\n");
> > > > +		mutex_unlock(&coredump->lock);
> > > > +		return;
> > > > +	}
> > > > +	coredump->faulty_engine = e;
> > > > +	devcoredump_snapshot(coredump);
> > > > +	mutex_unlock(&coredump->lock);
> > > > +
> > > > +	drm_info(&xe->drm, "Xe device coredump has been created\n");
> > > > +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
> > > > +
> > > > +	dev_coredumpm(xe->drm.dev, THIS_MODULE, coredump, 0, GFP_KERNEL,
> > > > +		      xe_devcoredump_read, xe_devcoredump_free);
> > > > +}
> > > > +
> > > > +/**
> > > > + * xe_devcoredump_init - Initialize xe_devcoredump.
> > > > + * @xe: Xe device.
> > > > + *
> > > > + * This function should be called at the probe so the mutex lock can be
> > > > + * initialized.
> > > > + */
> > > > +void xe_devcoredump_init(struct xe_device *xe)
> > > > +{
> > > > +	struct xe_devcoredump *coredump = &xe->devcoredump;
> > > > +
> > > > +	mutex_init(&coredump->lock);
> > > > +}
> > > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.h b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > > new file mode 100644
> > > > index 000000000000..30941d2e554b
> > > > --- /dev/null
> > > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.h
> > > > @@ -0,0 +1,22 @@
> > > > +/* SPDX-License-Identifier: MIT */
> > > > +/*
> > > > + * Copyright © 2023 Intel Corporation
> > > > + */
> > > > +
> > > > +#ifndef _XE_DEVCOREDUMP_H_
> > > > +#define _XE_DEVCOREDUMP_H_
> > > > +
> > > > +struct xe_device;
> > > > +struct xe_engine;
> > > > +
> > > > +void xe_devcoredump_init(struct xe_device *xe);
> > > > +
> > > > +#ifdef CONFIG_DEV_COREDUMP
> > > > +void xe_devcoredump(struct xe_engine *e);
> > > > +#else
> > > > +static inline void xe_devcoredump(struct xe_engine *e)
> > > > +{
> > > > +}
> > > > +#endif
> > > > +
> > > > +#endif
> > > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > > new file mode 100644
> > > > index 000000000000..3f395fa9104e
> > > > --- /dev/null
> > > > +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> > > > @@ -0,0 +1,47 @@
> > > > +/* SPDX-License-Identifier: MIT */
> > > > +/*
> > > > + * Copyright © 2023 Intel Corporation
> > > > + */
> > > > +
> > > > +#ifndef _XE_DEVCOREDUMP_TYPES_H_
> > > > +#define _XE_DEVCOREDUMP_TYPES_H_
> > > > +
> > > > +#include <linux/ktime.h>
> > > > +#include <linux/mutex.h>
> > > > +
> > > > +struct xe_device;
> > > > +
> > > > +/**
> > > > + * struct xe_devcoredump_snapshot - Crash snapshot
> > > > + *
> > > > + * This struct contains all the useful information quickly captured at the time
> > > > + * of the crash. So, any subsequent reads of the coredump points to a data that
> > > > + * shows the state of the GPU of when the issue has happened.
> > > > + */
> > > > +struct xe_devcoredump_snapshot {
> > > > +	/** @snapshot_time:  Time of this capture. */
> > > > +	ktime_t snapshot_time;
> > > > +	/** @boot_time:  Relative boot time so the uptime can be calculated. */
> > > > +	ktime_t boot_time;
> > > > +};
> > > > +
> > > > +/**
> > > > + * struct xe_devcoredump - Xe devcoredump main structure
> > > > + *
> > > > + * This struct represents the live and active dev_coredump node.
> > > > + * It is created/populated at the time of a crash/error. Then it
> > > > + * is read later when user access the device coredump data file
> > > > + * for reading the information.
> > > > + */
> > > > +struct xe_devcoredump {
> > > > +	/** @xe: Xe device. */
> > > > +	struct xe_device *xe;
> > > > +	/** @falty_engine: Engine where the crash/error happened. */
> > > > +	struct xe_engine *faulty_engine;
> > > > +	/** @lock: Protects data from races between capture and read out. */
> > > > +	struct mutex lock;
> > > > +	/** @snapshot: Snapshot is captured at time of the first crash */
> > > > +	struct xe_devcoredump_snapshot snapshot;
> > > > +};
> > > > +
> > > > +#endif
> > > > diff --git a/drivers/gpu/drm/xe/xe_device_types.h b/drivers/gpu/drm/xe/xe_device_types.h
> > > > index 1cb404e48aaa..2a0995824692 100644
> > > > --- a/drivers/gpu/drm/xe/xe_device_types.h
> > > > +++ b/drivers/gpu/drm/xe/xe_device_types.h
> > > > @@ -12,6 +12,7 @@
> > > >   #include <drm/drm_file.h>
> > > >   #include <drm/ttm/ttm_device.h>
> > > > +#include "xe_devcoredump_types.h"
> > > >   #include "xe_gt_types.h"
> > > >   #include "xe_platform_types.h"
> > > >   #include "xe_step_types.h"
> > > > @@ -55,6 +56,9 @@ struct xe_device {
> > > >   	/** @drm: drm device */
> > > >   	struct drm_device drm;
> > > > +	/** @devcoredump: device coredump */
> > > > +	struct xe_devcoredump devcoredump;
> > > > +
> > > >   	/** @info: device info */
> > > >   	struct intel_device_info {
> > > >   		/** @graphics_name: graphics IP name */
> > > > diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > > index e857013070b9..231fb4145297 100644
> > > > --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> > > > +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> > > > @@ -14,6 +14,7 @@
> > > >   #include <drm/drm_managed.h>
> > > >   #include "regs/xe_lrc_layout.h"
> > > > +#include "xe_devcoredump.h"
> > > >   #include "xe_device.h"
> > > >   #include "xe_engine.h"
> > > >   #include "xe_force_wake.h"
> > > > @@ -800,6 +801,7 @@ guc_engine_timedout_job(struct drm_sched_job *drm_job)
> > > >   		drm_warn(&xe->drm, "Timedout job: seqno=%u, guc_id=%d, flags=0x%lx",
> > > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > > >   		simple_error_capture(e);
> > > > +		xe_devcoredump(e);
> > > >   	} else {
> > > >   		drm_dbg(&xe->drm, "Timedout signaled job: seqno=%u, guc_id=%d, flags=0x%lx",
> > > >   			 xe_sched_job_seqno(job), e->guc->id, e->flags);
> > > > diff --git a/drivers/gpu/drm/xe/xe_pci.c b/drivers/gpu/drm/xe/xe_pci.c
> > > > index e512e8b69831..1d496210b580 100644
> > > > --- a/drivers/gpu/drm/xe/xe_pci.c
> > > > +++ b/drivers/gpu/drm/xe/xe_pci.c
> > > > @@ -16,6 +16,7 @@
> > > >   #include "regs/xe_regs.h"
> > > >   #include "regs/xe_gt_regs.h"
> > > > +#include "xe_devcoredump.h"
> > > >   #include "xe_device.h"
> > > >   #include "xe_display.h"
> > > >   #include "xe_drv.h"
> > > > @@ -657,6 +658,7 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
> > > >   		return err;
> > > >   	}
> > > > +	xe_devcoredump_init(xe);
> > > >   	xe_pm_runtime_init(xe);
> > > >   	return 0;

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Jani Nikula]

On Wed, 26 Apr 2023, Rodrigo Vivi <rodrigo.vivi@intel.com> wrote:
> +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");

Drive-by comment, could use %d and xe->drm.primary->index instead of
<n>.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Graphics Center

Re: [Intel-xe] [PATCH 02/14] drm/xe: Introduce the dev_coredump infrastructure.

[Rodrigo Vivi]

On Tue, May 02, 2023 at 10:55:14AM +0300, Jani Nikula wrote:
> On Wed, 26 Apr 2023, Rodrigo Vivi <rodrigo.vivi@intel.com> wrote:
> > +	drm_info(&xe->drm, "Check your /sys/class/drm/card<n>/device/devcoredump/data\n");
> 
> Drive-by comment, could use %d and xe->drm.primary->index instead of

yeap, I wondered about that. I guess I will just add this for now, but
then work on all driver to stop using the drm.primary directly or update
the documentation that states the drivers shouldn't use that directly.

> <n>.
> 
> BR,
> Jani.
> 
> 
> -- 
> Jani Nikula, Intel Open Source Graphics Center

[PATCH 03/14] drm/xe: Do not take any action if our device was removed.

[Rodrigo Vivi]

Unfortunately devcoredump infrastructure does not provide and
interface for us to force the device removal upon the pci_remove
time of our device.

The devcoredump is linked at the device level, so when in use
it will prevent the module removal, but it doesn't prevent the
call of the pci_remove callback. This callback cannot fail
anyway and we end up clearing and freeing the entire pci device.

Hence, after we removed the pci device, we shouldn't allow any
read or free operations to avoid segmentation fault.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index d9531183f03a..a08929c01b75 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -42,6 +42,11 @@
  * hang capture.
  */
 
+static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
+{
+	return container_of(coredump, struct xe_device, devcoredump);
+}
+
 static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 				   size_t count, void *data, size_t datalen)
 {
@@ -51,6 +56,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	struct drm_print_iterator iter;
 	struct timespec64 ts;
 
+	/* Our device is gone already... */
+	if (!data || !coredump_to_xe(coredump))
+		return -ENODEV;
+
 	iter.data = buffer;
 	iter.offset = 0;
 	iter.start = offset;
@@ -80,12 +89,16 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 static void xe_devcoredump_free(void *data)
 {
 	struct xe_devcoredump *coredump = data;
-	struct xe_device *xe = container_of(coredump, struct xe_device,
-					    devcoredump);
+
+	/* Our device is gone. Nothing to do... */
+	if (!data || !coredump_to_xe(coredump))
+		return;
+
 	mutex_lock(&coredump->lock);
 
 	coredump->faulty_engine = NULL;
-	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
+	drm_info(&coredump_to_xe(coredump)->drm,
+		 "Xe device coredump has been deleted.\n");
 
 	mutex_unlock(&coredump->lock);
 }
-- 
2.39.2

[Intel-xe] [PATCH 03/14] drm/xe: Do not take any action if our device was removed.

[Rodrigo Vivi]

Unfortunately devcoredump infrastructure does not provide and
interface for us to force the device removal upon the pci_remove
time of our device.

The devcoredump is linked at the device level, so when in use
it will prevent the module removal, but it doesn't prevent the
call of the pci_remove callback. This callback cannot fail
anyway and we end up clearing and freeing the entire pci device.

Hence, after we removed the pci device, we shouldn't allow any
read or free operations to avoid segmentation fault.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index d9531183f03a..a08929c01b75 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -42,6 +42,11 @@
  * hang capture.
  */
 
+static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
+{
+	return container_of(coredump, struct xe_device, devcoredump);
+}
+
 static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 				   size_t count, void *data, size_t datalen)
 {
@@ -51,6 +56,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	struct drm_print_iterator iter;
 	struct timespec64 ts;
 
+	/* Our device is gone already... */
+	if (!data || !coredump_to_xe(coredump))
+		return -ENODEV;
+
 	iter.data = buffer;
 	iter.offset = 0;
 	iter.start = offset;
@@ -80,12 +89,16 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 static void xe_devcoredump_free(void *data)
 {
 	struct xe_devcoredump *coredump = data;
-	struct xe_device *xe = container_of(coredump, struct xe_device,
-					    devcoredump);
+
+	/* Our device is gone. Nothing to do... */
+	if (!data || !coredump_to_xe(coredump))
+		return;
+
 	mutex_lock(&coredump->lock);
 
 	coredump->faulty_engine = NULL;
-	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
+	drm_info(&coredump_to_xe(coredump)->drm,
+		 "Xe device coredump has been deleted.\n");
 
 	mutex_unlock(&coredump->lock);
 }
-- 
2.39.2

Re: [Intel-xe] [PATCH 03/14] drm/xe: Do not take any action if our device was removed.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:02PM -0400, Rodrigo Vivi wrote:
> Unfortunately devcoredump infrastructure does not provide and
> interface for us to force the device removal upon the pci_remove
> time of our device.
> 
> The devcoredump is linked at the device level, so when in use
> it will prevent the module removal, but it doesn't prevent the
> call of the pci_remove callback. This callback cannot fail
> anyway and we end up clearing and freeing the entire pci device.
> 
> Hence, after we removed the pci device, we shouldn't allow any
> read or free operations to avoid segmentation fault.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c | 19 ++++++++++++++++---
>  1 file changed, 16 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index d9531183f03a..a08929c01b75 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -42,6 +42,11 @@
>   * hang capture.
>   */
>  
> +static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
> +{
> +	return container_of(coredump, struct xe_device, devcoredump);

Confused how still would ever return NULL, can you explain?

Matt

> +}
> +
>  static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  				   size_t count, void *data, size_t datalen)
>  {
> @@ -51,6 +56,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	struct drm_print_iterator iter;
>  	struct timespec64 ts;
>  
> +	/* Our device is gone already... */
> +	if (!data || !coredump_to_xe(coredump))
> +		return -ENODEV;
> +
>  	iter.data = buffer;
>  	iter.offset = 0;
>  	iter.start = offset;
> @@ -80,12 +89,16 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  static void xe_devcoredump_free(void *data)
>  {
>  	struct xe_devcoredump *coredump = data;
> -	struct xe_device *xe = container_of(coredump, struct xe_device,
> -					    devcoredump);
> +
> +	/* Our device is gone. Nothing to do... */
> +	if (!data || !coredump_to_xe(coredump))
> +		return;
> +
>  	mutex_lock(&coredump->lock);
>  
>  	coredump->faulty_engine = NULL;
> -	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> +	drm_info(&coredump_to_xe(coredump)->drm,
> +		 "Xe device coredump has been deleted.\n");
>  
>  	mutex_unlock(&coredump->lock);
>  }
> -- 
> 2.39.2
> 

Re: [Intel-xe] [PATCH 03/14] drm/xe: Do not take any action if our device was removed.

[Rodrigo Vivi]

On Tue, May 02, 2023 at 03:40:50PM +0000, Matthew Brost wrote:
> On Wed, Apr 26, 2023 at 04:57:02PM -0400, Rodrigo Vivi wrote:
> > Unfortunately devcoredump infrastructure does not provide and
> > interface for us to force the device removal upon the pci_remove
> > time of our device.
> > 
> > The devcoredump is linked at the device level, so when in use
> > it will prevent the module removal, but it doesn't prevent the
> > call of the pci_remove callback. This callback cannot fail
> > anyway and we end up clearing and freeing the entire pci device.
> > 
> > Hence, after we removed the pci device, we shouldn't allow any
> > read or free operations to avoid segmentation fault.
> > 
> > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > ---
> >  drivers/gpu/drm/xe/xe_devcoredump.c | 19 ++++++++++++++++---
> >  1 file changed, 16 insertions(+), 3 deletions(-)
> > 
> > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > index d9531183f03a..a08929c01b75 100644
> > --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > @@ -42,6 +42,11 @@
> >   * hang capture.
> >   */
> >  
> > +static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
> > +{
> > +	return container_of(coredump, struct xe_device, devcoredump);
> 
> Confused how still would ever return NULL, can you explain?

Very good question! I'm honestly still confused myself.

There's something not quite right with the device relationship that
is getting created with the failling_device and the virtual coredump device.

Once failing_device is removed, the devcoredump should be removed as well,
or both removals blocked. However this is not what happens.

On rmmod xe, the device removal is called and we free all xe structs.
The pci device removal is a void function. We cannot fail. The module
removal ends up blocked because the relationship, but that doesn't
saves the day since the device itself is already gone, by the pci
removal function.

But the devcoredump device is there and active. There's no callback on
devcoredump infra that we could call to force the device removal. Then
any read function will hit a NULL xe device and BOOM!

This is one of the things I planned to tackle on the devcoredump side
after we get this basic infra in use in our driver. This patch allows
us to be protected from this scenario while we don't fix this at the
devcoredump side.

It's worth saying that the devcoredump virtual device is auto removed
after a time elapsed... couple minutes? (I can't remember by heart now).

> 
> Matt
> 
> > +}
> > +
> >  static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> >  				   size_t count, void *data, size_t datalen)
> >  {
> > @@ -51,6 +56,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> >  	struct drm_print_iterator iter;
> >  	struct timespec64 ts;
> >  
> > +	/* Our device is gone already... */
> > +	if (!data || !coredump_to_xe(coredump))
> > +		return -ENODEV;
> > +
> >  	iter.data = buffer;
> >  	iter.offset = 0;
> >  	iter.start = offset;
> > @@ -80,12 +89,16 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> >  static void xe_devcoredump_free(void *data)
> >  {
> >  	struct xe_devcoredump *coredump = data;
> > -	struct xe_device *xe = container_of(coredump, struct xe_device,
> > -					    devcoredump);
> > +
> > +	/* Our device is gone. Nothing to do... */
> > +	if (!data || !coredump_to_xe(coredump))
> > +		return;
> > +
> >  	mutex_lock(&coredump->lock);
> >  
> >  	coredump->faulty_engine = NULL;
> > -	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> > +	drm_info(&coredump_to_xe(coredump)->drm,
> > +		 "Xe device coredump has been deleted.\n");
> >  
> >  	mutex_unlock(&coredump->lock);
> >  }
> > -- 
> > 2.39.2
> > 

Re: [Intel-xe] [PATCH 03/14] drm/xe: Do not take any action if our device was removed.

[Matthew Brost]

On Tue, May 02, 2023 at 01:21:43PM -0400, Rodrigo Vivi wrote:
> On Tue, May 02, 2023 at 03:40:50PM +0000, Matthew Brost wrote:
> > On Wed, Apr 26, 2023 at 04:57:02PM -0400, Rodrigo Vivi wrote:
> > > Unfortunately devcoredump infrastructure does not provide and
> > > interface for us to force the device removal upon the pci_remove
> > > time of our device.
> > > 
> > > The devcoredump is linked at the device level, so when in use
> > > it will prevent the module removal, but it doesn't prevent the
> > > call of the pci_remove callback. This callback cannot fail
> > > anyway and we end up clearing and freeing the entire pci device.
> > > 
> > > Hence, after we removed the pci device, we shouldn't allow any
> > > read or free operations to avoid segmentation fault.
> > > 
> > > Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> > > ---
> > >  drivers/gpu/drm/xe/xe_devcoredump.c | 19 ++++++++++++++++---
> > >  1 file changed, 16 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > index d9531183f03a..a08929c01b75 100644
> > > --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> > > +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> > > @@ -42,6 +42,11 @@
> > >   * hang capture.
> > >   */
> > >  
> > > +static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
> > > +{
> > > +	return container_of(coredump, struct xe_device, devcoredump);
> > 
> > Confused how still would ever return NULL, can you explain?
> 
> Very good question! I'm honestly still confused myself.
> 
> There's something not quite right with the device relationship that
> is getting created with the failling_device and the virtual coredump device.
> 
> Once failing_device is removed, the devcoredump should be removed as well,
> or both removals blocked. However this is not what happens.
> 
> On rmmod xe, the device removal is called and we free all xe structs.
> The pci device removal is a void function. We cannot fail. The module
> removal ends up blocked because the relationship, but that doesn't
> saves the day since the device itself is already gone, by the pci
> removal function.
> 
> But the devcoredump device is there and active. There's no callback on
> devcoredump infra that we could call to force the device removal. Then
> any read function will hit a NULL xe device and BOOM!
> 
> This is one of the things I planned to tackle on the devcoredump side
> after we get this basic infra in use in our driver. This patch allows
> us to be protected from this scenario while we don't fix this at the
> devcoredump side.
> 
> It's worth saying that the devcoredump virtual device is auto removed
> after a time elapsed... couple minutes? (I can't remember by heart now).
> 

After some serious thought, I think I get this. With that:

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

But agree this goofy and try to fix this properly after this is merged.

> > 
> > Matt
> > 
> > > +}
> > > +
> > >  static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > >  				   size_t count, void *data, size_t datalen)
> > >  {
> > > @@ -51,6 +56,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > >  	struct drm_print_iterator iter;
> > >  	struct timespec64 ts;
> > >  
> > > +	/* Our device is gone already... */
> > > +	if (!data || !coredump_to_xe(coredump))
> > > +		return -ENODEV;
> > > +
> > >  	iter.data = buffer;
> > >  	iter.offset = 0;
> > >  	iter.start = offset;
> > > @@ -80,12 +89,16 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
> > >  static void xe_devcoredump_free(void *data)
> > >  {
> > >  	struct xe_devcoredump *coredump = data;
> > > -	struct xe_device *xe = container_of(coredump, struct xe_device,
> > > -					    devcoredump);
> > > +
> > > +	/* Our device is gone. Nothing to do... */
> > > +	if (!data || !coredump_to_xe(coredump))
> > > +		return;
> > > +
> > >  	mutex_lock(&coredump->lock);
> > >  
> > >  	coredump->faulty_engine = NULL;
> > > -	drm_info(&xe->drm, "Xe device coredump has been deleted.\n");
> > > +	drm_info(&coredump_to_xe(coredump)->drm,
> > > +		 "Xe device coredump has been deleted.\n");
> > >  
> > >  	mutex_unlock(&coredump->lock);
> > >  }
> > > -- 
> > > 2.39.2
> > > 

[PATCH 04/14] drm/xe: Extract non mapped regions out of GuC CTB into its own struct.

[Rodrigo Vivi]

No functional change here. The goal is to have a clear split between
the mapped portions of the CTB and the static information, so we can
easily capture snapshots that will be used for later read out with
the devcoredump infrastructure.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_ct.c       | 155 ++++++++++++++-------------
 drivers/gpu/drm/xe/xe_guc_ct_types.h |  20 ++--
 2 files changed, 95 insertions(+), 80 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
index 9055ff133a7c..e16e5fe37ed4 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct.c
+++ b/drivers/gpu/drm/xe/xe_guc_ct.c
@@ -172,13 +172,14 @@ int xe_guc_ct_init(struct xe_guc_ct *ct)
 static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g,
 				struct iosys_map *map)
 {
-	h2g->size = CTB_H2G_BUFFER_SIZE / sizeof(u32);
-	h2g->resv_space = 0;
-	h2g->tail = 0;
-	h2g->head = 0;
-	h2g->space = CIRC_SPACE(h2g->tail, h2g->head, h2g->size) -
-		h2g->resv_space;
-	h2g->broken = false;
+	h2g->info.size = CTB_H2G_BUFFER_SIZE / sizeof(u32);
+	h2g->info.resv_space = 0;
+	h2g->info.tail = 0;
+	h2g->info.head = 0;
+	h2g->info.space = CIRC_SPACE(h2g->info.tail, h2g->info.head,
+				     h2g->info.size) -
+			  h2g->info.resv_space;
+	h2g->info.broken = false;
 
 	h2g->desc = *map;
 	xe_map_memset(xe, &h2g->desc, 0, 0, sizeof(struct guc_ct_buffer_desc));
@@ -189,13 +190,14 @@ static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g,
 static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h,
 				struct iosys_map *map)
 {
-	g2h->size = CTB_G2H_BUFFER_SIZE / sizeof(u32);
-	g2h->resv_space = G2H_ROOM_BUFFER_SIZE / sizeof(u32);
-	g2h->head = 0;
-	g2h->tail = 0;
-	g2h->space = CIRC_SPACE(g2h->tail, g2h->head, g2h->size) -
-		g2h->resv_space;
-	g2h->broken = false;
+	g2h->info.size = CTB_G2H_BUFFER_SIZE / sizeof(u32);
+	g2h->info.resv_space = G2H_ROOM_BUFFER_SIZE / sizeof(u32);
+	g2h->info.head = 0;
+	g2h->info.tail = 0;
+	g2h->info.space = CIRC_SPACE(g2h->info.tail, g2h->info.head,
+				     g2h->info.size) -
+			  g2h->info.resv_space;
+	g2h->info.broken = false;
 
 	g2h->desc = IOSYS_MAP_INIT_OFFSET(map, CTB_DESC_SIZE);
 	xe_map_memset(xe, &g2h->desc, 0, 0, sizeof(struct guc_ct_buffer_desc));
@@ -212,7 +214,7 @@ static int guc_ct_ctb_h2g_register(struct xe_guc_ct *ct)
 
 	desc_addr = xe_bo_ggtt_addr(ct->bo);
 	ctb_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE * 2;
-	size = ct->ctbs.h2g.size * sizeof(u32);
+	size = ct->ctbs.h2g.info.size * sizeof(u32);
 
 	err = xe_guc_self_cfg64(guc,
 				GUC_KLV_SELF_CFG_H2G_CTB_DESCRIPTOR_ADDR_KEY,
@@ -240,7 +242,7 @@ static int guc_ct_ctb_g2h_register(struct xe_guc_ct *ct)
 	desc_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE;
 	ctb_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE * 2 +
 		CTB_H2G_BUFFER_SIZE;
-	size = ct->ctbs.g2h.size * sizeof(u32);
+	size = ct->ctbs.g2h.info.size * sizeof(u32);
 
 	err = xe_guc_self_cfg64(guc,
 				GUC_KLV_SELF_CFG_G2H_CTB_DESCRIPTOR_ADDR_KEY,
@@ -329,11 +331,12 @@ static bool h2g_has_room(struct xe_guc_ct *ct, u32 cmd_len)
 
 	lockdep_assert_held(&ct->lock);
 
-	if (cmd_len > h2g->space) {
-		h2g->head = desc_read(ct_to_xe(ct), h2g, head);
-		h2g->space = CIRC_SPACE(h2g->tail, h2g->head, h2g->size) -
-			h2g->resv_space;
-		if (cmd_len > h2g->space)
+	if (cmd_len > h2g->info.space) {
+		h2g->info.head = desc_read(ct_to_xe(ct), h2g, head);
+		h2g->info.space = CIRC_SPACE(h2g->info.tail, h2g->info.head,
+					     h2g->info.size) -
+				  h2g->info.resv_space;
+		if (cmd_len > h2g->info.space)
 			return false;
 	}
 
@@ -344,7 +347,7 @@ static bool g2h_has_room(struct xe_guc_ct *ct, u32 g2h_len)
 {
 	lockdep_assert_held(&ct->lock);
 
-	return ct->ctbs.g2h.space > g2h_len;
+	return ct->ctbs.g2h.info.space > g2h_len;
 }
 
 static int has_room(struct xe_guc_ct *ct, u32 cmd_len, u32 g2h_len)
@@ -360,16 +363,16 @@ static int has_room(struct xe_guc_ct *ct, u32 cmd_len, u32 g2h_len)
 static void h2g_reserve_space(struct xe_guc_ct *ct, u32 cmd_len)
 {
 	lockdep_assert_held(&ct->lock);
-	ct->ctbs.h2g.space -= cmd_len;
+	ct->ctbs.h2g.info.space -= cmd_len;
 }
 
 static void g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h)
 {
-	XE_BUG_ON(g2h_len > ct->ctbs.g2h.space);
+	XE_BUG_ON(g2h_len > ct->ctbs.g2h.info.space);
 
 	if (g2h_len) {
 		spin_lock_irq(&ct->fast_lock);
-		ct->ctbs.g2h.space -= g2h_len;
+		ct->ctbs.g2h.info.space -= g2h_len;
 		ct->g2h_outstanding += num_g2h;
 		spin_unlock_irq(&ct->fast_lock);
 	}
@@ -378,10 +381,10 @@ static void g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h)
 static void __g2h_release_space(struct xe_guc_ct *ct, u32 g2h_len)
 {
 	lockdep_assert_held(&ct->fast_lock);
-	XE_WARN_ON(ct->ctbs.g2h.space + g2h_len >
-		   ct->ctbs.g2h.size - ct->ctbs.g2h.resv_space);
+	XE_WARN_ON(ct->ctbs.g2h.info.space + g2h_len >
+		   ct->ctbs.g2h.info.size - ct->ctbs.g2h.info.resv_space);
 
-	ct->ctbs.g2h.space += g2h_len;
+	ct->ctbs.g2h.info.space += g2h_len;
 	--ct->g2h_outstanding;
 }
 
@@ -400,20 +403,21 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
 	u32 cmd[GUC_CTB_MSG_MAX_LEN / sizeof(u32)];
 	u32 cmd_len = len + GUC_CTB_HDR_LEN;
 	u32 cmd_idx = 0, i;
-	u32 tail = h2g->tail;
+	u32 tail = h2g->info.tail;
 	struct iosys_map map = IOSYS_MAP_INIT_OFFSET(&h2g->cmds,
 							 tail * sizeof(u32));
 
 	lockdep_assert_held(&ct->lock);
 	XE_BUG_ON(len * sizeof(u32) > GUC_CTB_MSG_MAX_LEN);
-	XE_BUG_ON(tail > h2g->size);
+	XE_BUG_ON(tail > h2g->info.size);
 
 	/* Command will wrap, zero fill (NOPs), return and check credits again */
-	if (tail + cmd_len > h2g->size) {
-		xe_map_memset(xe, &map, 0, 0, (h2g->size - tail) * sizeof(u32));
-		h2g_reserve_space(ct, (h2g->size - tail));
-		h2g->tail = 0;
-		desc_write(xe, h2g, tail, h2g->tail);
+	if (tail + cmd_len > h2g->info.size) {
+		xe_map_memset(xe, &map, 0, 0,
+			      (h2g->info.size - tail) * sizeof(u32));
+		h2g_reserve_space(ct, (h2g->info.size - tail));
+		h2g->info.tail = 0;
+		desc_write(xe, h2g, tail, h2g->info.tail);
 
 		return -EAGAIN;
 	}
@@ -445,11 +449,11 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
 	xe_device_wmb(ct_to_xe(ct));
 
 	/* Update local copies */
-	h2g->tail = (tail + cmd_len) % h2g->size;
+	h2g->info.tail = (tail + cmd_len) % h2g->info.size;
 	h2g_reserve_space(ct, cmd_len);
 
 	/* Update descriptor */
-	desc_write(xe, h2g, tail, h2g->tail);
+	desc_write(xe, h2g, tail, h2g->info.tail);
 
 	return 0;
 }
@@ -466,7 +470,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 	XE_BUG_ON(!g2h_len && num_g2h);
 	lockdep_assert_held(&ct->lock);
 
-	if (unlikely(ct->ctbs.h2g.broken)) {
+	if (unlikely(ct->ctbs.h2g.info.broken)) {
 		ret = -EPIPE;
 		goto out;
 	}
@@ -554,8 +558,9 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		if (sleep_period_ms == 1024)
 			goto broken;
 
-		trace_xe_guc_ct_h2g_flow_control(h2g->head, h2g->tail,
-						 h2g->size, h2g->space,
+		trace_xe_guc_ct_h2g_flow_control(h2g->info.head, h2g->info.tail,
+						 h2g->info.size,
+						 h2g->info.space,
 						 len + GUC_CTB_HDR_LEN);
 		msleep(sleep_period_ms);
 		sleep_period_ms <<= 1;
@@ -565,15 +570,16 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		struct xe_device *xe = ct_to_xe(ct);
 		struct guc_ctb *g2h = &ct->ctbs.g2h;
 
-		trace_xe_guc_ct_g2h_flow_control(g2h->head,
+		trace_xe_guc_ct_g2h_flow_control(g2h->info.head,
 						 desc_read(xe, g2h, tail),
-						 g2h->size, g2h->space,
+						 g2h->info.size,
+						 g2h->info.space,
 						 g2h_fence ?
 						 GUC_CTB_HXG_MSG_MAX_LEN :
 						 g2h_len);
 
 #define g2h_avail(ct)	\
-	(desc_read(ct_to_xe(ct), (&ct->ctbs.g2h), tail) != ct->ctbs.g2h.head)
+	(desc_read(ct_to_xe(ct), (&ct->ctbs.g2h), tail) != ct->ctbs.g2h.info.head)
 		if (!wait_event_timeout(ct->wq, !ct->g2h_outstanding ||
 					g2h_avail(ct), HZ))
 			goto broken;
@@ -590,7 +596,7 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
 broken:
 	drm_err(drm, "No forward process on H2G, reset required");
 	xe_guc_ct_print(ct, &p);
-	ct->ctbs.h2g.broken = true;
+	ct->ctbs.h2g.info.broken = true;
 
 	return -EDEADLK;
 }
@@ -656,7 +662,7 @@ static bool retry_failure(struct xe_guc_ct *ct, int ret)
 		return false;
 
 #define ct_alive(ct)	\
-	(ct->enabled && !ct->ctbs.h2g.broken && !ct->ctbs.g2h.broken)
+	(ct->enabled && !ct->ctbs.h2g.info.broken && !ct->ctbs.g2h.info.broken)
 	if (!wait_event_interruptible_timeout(ct->wq, ct_alive(ct),  HZ * 5))
 		return false;
 #undef ct_alive
@@ -821,7 +827,7 @@ static int parse_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len)
 		drm_err(&xe->drm,
 			"G2H channel broken on read, origin=%d, reset required\n",
 			origin);
-		ct->ctbs.g2h.broken = true;
+		ct->ctbs.g2h.info.broken = true;
 
 		return -EPROTO;
 	}
@@ -840,7 +846,7 @@ static int parse_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len)
 		drm_err(&xe->drm,
 			"G2H channel broken on read, type=%d, reset required\n",
 			type);
-		ct->ctbs.g2h.broken = true;
+		ct->ctbs.g2h.info.broken = true;
 
 		ret = -EOPNOTSUPP;
 	}
@@ -919,36 +925,37 @@ static int g2h_read(struct xe_guc_ct *ct, u32 *msg, bool fast_path)
 	if (!ct->enabled)
 		return -ENODEV;
 
-	if (g2h->broken)
+	if (g2h->info.broken)
 		return -EPIPE;
 
 	/* Calculate DW available to read */
 	tail = desc_read(xe, g2h, tail);
-	avail = tail - g2h->head;
+	avail = tail - g2h->info.head;
 	if (unlikely(avail == 0))
 		return 0;
 
 	if (avail < 0)
-		avail += g2h->size;
+		avail += g2h->info.size;
 
 	/* Read header */
-	xe_map_memcpy_from(xe, msg, &g2h->cmds, sizeof(u32) * g2h->head, sizeof(u32));
+	xe_map_memcpy_from(xe, msg, &g2h->cmds, sizeof(u32) * g2h->info.head,
+			   sizeof(u32));
 	len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN;
 	if (len > avail) {
 		drm_err(&xe->drm,
 			"G2H channel broken on read, avail=%d, len=%d, reset required\n",
 			avail, len);
-		g2h->broken = true;
+		g2h->info.broken = true;
 
 		return -EPROTO;
 	}
 
-	head = (g2h->head + 1) % g2h->size;
+	head = (g2h->info.head + 1) % g2h->info.size;
 	avail = len - 1;
 
 	/* Read G2H message */
-	if (avail + head > g2h->size) {
-		u32 avail_til_wrap = g2h->size - head;
+	if (avail + head > g2h->info.size) {
+		u32 avail_til_wrap = g2h->info.size - head;
 
 		xe_map_memcpy_from(xe, msg + 1,
 				   &g2h->cmds, sizeof(u32) * head,
@@ -983,8 +990,8 @@ static int g2h_read(struct xe_guc_ct *ct, u32 *msg, bool fast_path)
 	}
 
 	/* Update local / descriptor header */
-	g2h->head = (head + avail) % g2h->size;
-	desc_write(xe, g2h, head, g2h->head);
+	g2h->info.head = (head + avail) % g2h->info.size;
+	desc_write(xe, g2h, head, g2h->info.head);
 
 	return len;
 }
@@ -1093,12 +1100,12 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
 {
 	u32 head, tail;
 
-	drm_printf(p, "\tsize: %d\n", ctb->size);
-	drm_printf(p, "\tresv_space: %d\n", ctb->resv_space);
-	drm_printf(p, "\thead: %d\n", ctb->head);
-	drm_printf(p, "\ttail: %d\n", ctb->tail);
-	drm_printf(p, "\tspace: %d\n", ctb->space);
-	drm_printf(p, "\tbroken: %d\n", ctb->broken);
+	drm_printf(p, "\tsize: %d\n", ctb->info.size);
+	drm_printf(p, "\tresv_space: %d\n", ctb->info.resv_space);
+	drm_printf(p, "\thead: %d\n", ctb->info.head);
+	drm_printf(p, "\ttail: %d\n", ctb->info.tail);
+	drm_printf(p, "\tspace: %d\n", ctb->info.space);
+	drm_printf(p, "\tbroken: %d\n", ctb->info.broken);
 
 	head = desc_read(xe, ctb, head);
 	tail = desc_read(xe, ctb, tail);
@@ -1114,7 +1121,7 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
 			drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
 				   xe_map_rd(xe, &map, 0, u32));
 			++head;
-			if (head == ctb->size) {
+			if (head == ctb->info.size) {
 				head = 0;
 				map = ctb->cmds;
 			} else {
@@ -1168,12 +1175,12 @@ void xe_guc_ct_selftest(struct xe_guc_ct *ct, struct drm_printer *p)
 	if (!ret) {
 		xe_guc_ct_irq_handler(ct);
 		msleep(200);
-		if (g2h->space !=
-		    CIRC_SPACE(0, 0, g2h->size) - g2h->resv_space) {
+		if (g2h->info.space !=
+		    CIRC_SPACE(0, 0, g2h->info.size) - g2h->info.resv_space) {
 			drm_printf(p, "Mismatch on space %d, %d\n",
-				   g2h->space,
-				   CIRC_SPACE(0, 0, g2h->size) -
-				   g2h->resv_space);
+				   g2h->info.space,
+				   CIRC_SPACE(0, 0, g2h->info.size) -
+				   g2h->info.resv_space);
 			ret = -EIO;
 		}
 		if (ct->g2h_outstanding) {
@@ -1185,12 +1192,12 @@ void xe_guc_ct_selftest(struct xe_guc_ct *ct, struct drm_printer *p)
 
 	/* Check failure path for blocking CTs too */
 	xe_guc_ct_send_block(ct, bad_action, ARRAY_SIZE(bad_action));
-	if (g2h->space !=
-	    CIRC_SPACE(0, 0, g2h->size) - g2h->resv_space) {
+	if (g2h->info.space !=
+	    CIRC_SPACE(0, 0, g2h->info.size) - g2h->info.resv_space) {
 		drm_printf(p, "Mismatch on space %d, %d\n",
-			   g2h->space,
-			   CIRC_SPACE(0, 0, g2h->size) -
-			   g2h->resv_space);
+			   g2h->info.space,
+			   CIRC_SPACE(0, 0, g2h->info.size) -
+			   g2h->info.resv_space);
 		ret = -EIO;
 	}
 	if (ct->g2h_outstanding) {
diff --git a/drivers/gpu/drm/xe/xe_guc_ct_types.h b/drivers/gpu/drm/xe/xe_guc_ct_types.h
index fd27dacf00c5..64e3dd14d4b2 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct_types.h
+++ b/drivers/gpu/drm/xe/xe_guc_ct_types.h
@@ -19,13 +19,9 @@
 struct xe_bo;
 
 /**
- * struct guc_ctb - GuC command transport buffer (CTB)
+ * struct guc_ctb_info - GuC command transport buffer (CTB) info
  */
-struct guc_ctb {
-	/** @desc: dma buffer map for CTB descriptor */
-	struct iosys_map desc;
-	/** @cmds: dma buffer map for CTB commands */
-	struct iosys_map cmds;
+struct guc_ctb_info {
 	/** @size: size of CTB commands (DW) */
 	u32 size;
 	/** @resv_space: reserved space of CTB commands (DW) */
@@ -40,6 +36,18 @@ struct guc_ctb {
 	bool broken;
 };
 
+/**
+ * struct guc_ctb - GuC command transport buffer (CTB)
+ */
+struct guc_ctb {
+	/** @desc: dma buffer map for CTB descriptor */
+	struct iosys_map desc;
+	/** @cmds: dma buffer map for CTB commands */
+	struct iosys_map cmds;
+	/** @info: CTB info */
+	struct guc_ctb_info info;
+};
+
 /**
  * struct xe_guc_ct - GuC command transport (CT) layer
  *
-- 
2.39.2

[Intel-xe] [PATCH 04/14] drm/xe: Extract non mapped regions out of GuC CTB into its own struct.

[Rodrigo Vivi]

No functional change here. The goal is to have a clear split between
the mapped portions of the CTB and the static information, so we can
easily capture snapshots that will be used for later read out with
the devcoredump infrastructure.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_ct.c       | 155 ++++++++++++++-------------
 drivers/gpu/drm/xe/xe_guc_ct_types.h |  20 ++--
 2 files changed, 95 insertions(+), 80 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
index 9055ff133a7c..e16e5fe37ed4 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct.c
+++ b/drivers/gpu/drm/xe/xe_guc_ct.c
@@ -172,13 +172,14 @@ int xe_guc_ct_init(struct xe_guc_ct *ct)
 static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g,
 				struct iosys_map *map)
 {
-	h2g->size = CTB_H2G_BUFFER_SIZE / sizeof(u32);
-	h2g->resv_space = 0;
-	h2g->tail = 0;
-	h2g->head = 0;
-	h2g->space = CIRC_SPACE(h2g->tail, h2g->head, h2g->size) -
-		h2g->resv_space;
-	h2g->broken = false;
+	h2g->info.size = CTB_H2G_BUFFER_SIZE / sizeof(u32);
+	h2g->info.resv_space = 0;
+	h2g->info.tail = 0;
+	h2g->info.head = 0;
+	h2g->info.space = CIRC_SPACE(h2g->info.tail, h2g->info.head,
+				     h2g->info.size) -
+			  h2g->info.resv_space;
+	h2g->info.broken = false;
 
 	h2g->desc = *map;
 	xe_map_memset(xe, &h2g->desc, 0, 0, sizeof(struct guc_ct_buffer_desc));
@@ -189,13 +190,14 @@ static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g,
 static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h,
 				struct iosys_map *map)
 {
-	g2h->size = CTB_G2H_BUFFER_SIZE / sizeof(u32);
-	g2h->resv_space = G2H_ROOM_BUFFER_SIZE / sizeof(u32);
-	g2h->head = 0;
-	g2h->tail = 0;
-	g2h->space = CIRC_SPACE(g2h->tail, g2h->head, g2h->size) -
-		g2h->resv_space;
-	g2h->broken = false;
+	g2h->info.size = CTB_G2H_BUFFER_SIZE / sizeof(u32);
+	g2h->info.resv_space = G2H_ROOM_BUFFER_SIZE / sizeof(u32);
+	g2h->info.head = 0;
+	g2h->info.tail = 0;
+	g2h->info.space = CIRC_SPACE(g2h->info.tail, g2h->info.head,
+				     g2h->info.size) -
+			  g2h->info.resv_space;
+	g2h->info.broken = false;
 
 	g2h->desc = IOSYS_MAP_INIT_OFFSET(map, CTB_DESC_SIZE);
 	xe_map_memset(xe, &g2h->desc, 0, 0, sizeof(struct guc_ct_buffer_desc));
@@ -212,7 +214,7 @@ static int guc_ct_ctb_h2g_register(struct xe_guc_ct *ct)
 
 	desc_addr = xe_bo_ggtt_addr(ct->bo);
 	ctb_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE * 2;
-	size = ct->ctbs.h2g.size * sizeof(u32);
+	size = ct->ctbs.h2g.info.size * sizeof(u32);
 
 	err = xe_guc_self_cfg64(guc,
 				GUC_KLV_SELF_CFG_H2G_CTB_DESCRIPTOR_ADDR_KEY,
@@ -240,7 +242,7 @@ static int guc_ct_ctb_g2h_register(struct xe_guc_ct *ct)
 	desc_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE;
 	ctb_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE * 2 +
 		CTB_H2G_BUFFER_SIZE;
-	size = ct->ctbs.g2h.size * sizeof(u32);
+	size = ct->ctbs.g2h.info.size * sizeof(u32);
 
 	err = xe_guc_self_cfg64(guc,
 				GUC_KLV_SELF_CFG_G2H_CTB_DESCRIPTOR_ADDR_KEY,
@@ -329,11 +331,12 @@ static bool h2g_has_room(struct xe_guc_ct *ct, u32 cmd_len)
 
 	lockdep_assert_held(&ct->lock);
 
-	if (cmd_len > h2g->space) {
-		h2g->head = desc_read(ct_to_xe(ct), h2g, head);
-		h2g->space = CIRC_SPACE(h2g->tail, h2g->head, h2g->size) -
-			h2g->resv_space;
-		if (cmd_len > h2g->space)
+	if (cmd_len > h2g->info.space) {
+		h2g->info.head = desc_read(ct_to_xe(ct), h2g, head);
+		h2g->info.space = CIRC_SPACE(h2g->info.tail, h2g->info.head,
+					     h2g->info.size) -
+				  h2g->info.resv_space;
+		if (cmd_len > h2g->info.space)
 			return false;
 	}
 
@@ -344,7 +347,7 @@ static bool g2h_has_room(struct xe_guc_ct *ct, u32 g2h_len)
 {
 	lockdep_assert_held(&ct->lock);
 
-	return ct->ctbs.g2h.space > g2h_len;
+	return ct->ctbs.g2h.info.space > g2h_len;
 }
 
 static int has_room(struct xe_guc_ct *ct, u32 cmd_len, u32 g2h_len)
@@ -360,16 +363,16 @@ static int has_room(struct xe_guc_ct *ct, u32 cmd_len, u32 g2h_len)
 static void h2g_reserve_space(struct xe_guc_ct *ct, u32 cmd_len)
 {
 	lockdep_assert_held(&ct->lock);
-	ct->ctbs.h2g.space -= cmd_len;
+	ct->ctbs.h2g.info.space -= cmd_len;
 }
 
 static void g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h)
 {
-	XE_BUG_ON(g2h_len > ct->ctbs.g2h.space);
+	XE_BUG_ON(g2h_len > ct->ctbs.g2h.info.space);
 
 	if (g2h_len) {
 		spin_lock_irq(&ct->fast_lock);
-		ct->ctbs.g2h.space -= g2h_len;
+		ct->ctbs.g2h.info.space -= g2h_len;
 		ct->g2h_outstanding += num_g2h;
 		spin_unlock_irq(&ct->fast_lock);
 	}
@@ -378,10 +381,10 @@ static void g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h)
 static void __g2h_release_space(struct xe_guc_ct *ct, u32 g2h_len)
 {
 	lockdep_assert_held(&ct->fast_lock);
-	XE_WARN_ON(ct->ctbs.g2h.space + g2h_len >
-		   ct->ctbs.g2h.size - ct->ctbs.g2h.resv_space);
+	XE_WARN_ON(ct->ctbs.g2h.info.space + g2h_len >
+		   ct->ctbs.g2h.info.size - ct->ctbs.g2h.info.resv_space);
 
-	ct->ctbs.g2h.space += g2h_len;
+	ct->ctbs.g2h.info.space += g2h_len;
 	--ct->g2h_outstanding;
 }
 
@@ -400,20 +403,21 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
 	u32 cmd[GUC_CTB_MSG_MAX_LEN / sizeof(u32)];
 	u32 cmd_len = len + GUC_CTB_HDR_LEN;
 	u32 cmd_idx = 0, i;
-	u32 tail = h2g->tail;
+	u32 tail = h2g->info.tail;
 	struct iosys_map map = IOSYS_MAP_INIT_OFFSET(&h2g->cmds,
 							 tail * sizeof(u32));
 
 	lockdep_assert_held(&ct->lock);
 	XE_BUG_ON(len * sizeof(u32) > GUC_CTB_MSG_MAX_LEN);
-	XE_BUG_ON(tail > h2g->size);
+	XE_BUG_ON(tail > h2g->info.size);
 
 	/* Command will wrap, zero fill (NOPs), return and check credits again */
-	if (tail + cmd_len > h2g->size) {
-		xe_map_memset(xe, &map, 0, 0, (h2g->size - tail) * sizeof(u32));
-		h2g_reserve_space(ct, (h2g->size - tail));
-		h2g->tail = 0;
-		desc_write(xe, h2g, tail, h2g->tail);
+	if (tail + cmd_len > h2g->info.size) {
+		xe_map_memset(xe, &map, 0, 0,
+			      (h2g->info.size - tail) * sizeof(u32));
+		h2g_reserve_space(ct, (h2g->info.size - tail));
+		h2g->info.tail = 0;
+		desc_write(xe, h2g, tail, h2g->info.tail);
 
 		return -EAGAIN;
 	}
@@ -445,11 +449,11 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
 	xe_device_wmb(ct_to_xe(ct));
 
 	/* Update local copies */
-	h2g->tail = (tail + cmd_len) % h2g->size;
+	h2g->info.tail = (tail + cmd_len) % h2g->info.size;
 	h2g_reserve_space(ct, cmd_len);
 
 	/* Update descriptor */
-	desc_write(xe, h2g, tail, h2g->tail);
+	desc_write(xe, h2g, tail, h2g->info.tail);
 
 	return 0;
 }
@@ -466,7 +470,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 	XE_BUG_ON(!g2h_len && num_g2h);
 	lockdep_assert_held(&ct->lock);
 
-	if (unlikely(ct->ctbs.h2g.broken)) {
+	if (unlikely(ct->ctbs.h2g.info.broken)) {
 		ret = -EPIPE;
 		goto out;
 	}
@@ -554,8 +558,9 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		if (sleep_period_ms == 1024)
 			goto broken;
 
-		trace_xe_guc_ct_h2g_flow_control(h2g->head, h2g->tail,
-						 h2g->size, h2g->space,
+		trace_xe_guc_ct_h2g_flow_control(h2g->info.head, h2g->info.tail,
+						 h2g->info.size,
+						 h2g->info.space,
 						 len + GUC_CTB_HDR_LEN);
 		msleep(sleep_period_ms);
 		sleep_period_ms <<= 1;
@@ -565,15 +570,16 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		struct xe_device *xe = ct_to_xe(ct);
 		struct guc_ctb *g2h = &ct->ctbs.g2h;
 
-		trace_xe_guc_ct_g2h_flow_control(g2h->head,
+		trace_xe_guc_ct_g2h_flow_control(g2h->info.head,
 						 desc_read(xe, g2h, tail),
-						 g2h->size, g2h->space,
+						 g2h->info.size,
+						 g2h->info.space,
 						 g2h_fence ?
 						 GUC_CTB_HXG_MSG_MAX_LEN :
 						 g2h_len);
 
 #define g2h_avail(ct)	\
-	(desc_read(ct_to_xe(ct), (&ct->ctbs.g2h), tail) != ct->ctbs.g2h.head)
+	(desc_read(ct_to_xe(ct), (&ct->ctbs.g2h), tail) != ct->ctbs.g2h.info.head)
 		if (!wait_event_timeout(ct->wq, !ct->g2h_outstanding ||
 					g2h_avail(ct), HZ))
 			goto broken;
@@ -590,7 +596,7 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
 broken:
 	drm_err(drm, "No forward process on H2G, reset required");
 	xe_guc_ct_print(ct, &p);
-	ct->ctbs.h2g.broken = true;
+	ct->ctbs.h2g.info.broken = true;
 
 	return -EDEADLK;
 }
@@ -656,7 +662,7 @@ static bool retry_failure(struct xe_guc_ct *ct, int ret)
 		return false;
 
 #define ct_alive(ct)	\
-	(ct->enabled && !ct->ctbs.h2g.broken && !ct->ctbs.g2h.broken)
+	(ct->enabled && !ct->ctbs.h2g.info.broken && !ct->ctbs.g2h.info.broken)
 	if (!wait_event_interruptible_timeout(ct->wq, ct_alive(ct),  HZ * 5))
 		return false;
 #undef ct_alive
@@ -821,7 +827,7 @@ static int parse_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len)
 		drm_err(&xe->drm,
 			"G2H channel broken on read, origin=%d, reset required\n",
 			origin);
-		ct->ctbs.g2h.broken = true;
+		ct->ctbs.g2h.info.broken = true;
 
 		return -EPROTO;
 	}
@@ -840,7 +846,7 @@ static int parse_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len)
 		drm_err(&xe->drm,
 			"G2H channel broken on read, type=%d, reset required\n",
 			type);
-		ct->ctbs.g2h.broken = true;
+		ct->ctbs.g2h.info.broken = true;
 
 		ret = -EOPNOTSUPP;
 	}
@@ -919,36 +925,37 @@ static int g2h_read(struct xe_guc_ct *ct, u32 *msg, bool fast_path)
 	if (!ct->enabled)
 		return -ENODEV;
 
-	if (g2h->broken)
+	if (g2h->info.broken)
 		return -EPIPE;
 
 	/* Calculate DW available to read */
 	tail = desc_read(xe, g2h, tail);
-	avail = tail - g2h->head;
+	avail = tail - g2h->info.head;
 	if (unlikely(avail == 0))
 		return 0;
 
 	if (avail < 0)
-		avail += g2h->size;
+		avail += g2h->info.size;
 
 	/* Read header */
-	xe_map_memcpy_from(xe, msg, &g2h->cmds, sizeof(u32) * g2h->head, sizeof(u32));
+	xe_map_memcpy_from(xe, msg, &g2h->cmds, sizeof(u32) * g2h->info.head,
+			   sizeof(u32));
 	len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN;
 	if (len > avail) {
 		drm_err(&xe->drm,
 			"G2H channel broken on read, avail=%d, len=%d, reset required\n",
 			avail, len);
-		g2h->broken = true;
+		g2h->info.broken = true;
 
 		return -EPROTO;
 	}
 
-	head = (g2h->head + 1) % g2h->size;
+	head = (g2h->info.head + 1) % g2h->info.size;
 	avail = len - 1;
 
 	/* Read G2H message */
-	if (avail + head > g2h->size) {
-		u32 avail_til_wrap = g2h->size - head;
+	if (avail + head > g2h->info.size) {
+		u32 avail_til_wrap = g2h->info.size - head;
 
 		xe_map_memcpy_from(xe, msg + 1,
 				   &g2h->cmds, sizeof(u32) * head,
@@ -983,8 +990,8 @@ static int g2h_read(struct xe_guc_ct *ct, u32 *msg, bool fast_path)
 	}
 
 	/* Update local / descriptor header */
-	g2h->head = (head + avail) % g2h->size;
-	desc_write(xe, g2h, head, g2h->head);
+	g2h->info.head = (head + avail) % g2h->info.size;
+	desc_write(xe, g2h, head, g2h->info.head);
 
 	return len;
 }
@@ -1093,12 +1100,12 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
 {
 	u32 head, tail;
 
-	drm_printf(p, "\tsize: %d\n", ctb->size);
-	drm_printf(p, "\tresv_space: %d\n", ctb->resv_space);
-	drm_printf(p, "\thead: %d\n", ctb->head);
-	drm_printf(p, "\ttail: %d\n", ctb->tail);
-	drm_printf(p, "\tspace: %d\n", ctb->space);
-	drm_printf(p, "\tbroken: %d\n", ctb->broken);
+	drm_printf(p, "\tsize: %d\n", ctb->info.size);
+	drm_printf(p, "\tresv_space: %d\n", ctb->info.resv_space);
+	drm_printf(p, "\thead: %d\n", ctb->info.head);
+	drm_printf(p, "\ttail: %d\n", ctb->info.tail);
+	drm_printf(p, "\tspace: %d\n", ctb->info.space);
+	drm_printf(p, "\tbroken: %d\n", ctb->info.broken);
 
 	head = desc_read(xe, ctb, head);
 	tail = desc_read(xe, ctb, tail);
@@ -1114,7 +1121,7 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
 			drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
 				   xe_map_rd(xe, &map, 0, u32));
 			++head;
-			if (head == ctb->size) {
+			if (head == ctb->info.size) {
 				head = 0;
 				map = ctb->cmds;
 			} else {
@@ -1168,12 +1175,12 @@ void xe_guc_ct_selftest(struct xe_guc_ct *ct, struct drm_printer *p)
 	if (!ret) {
 		xe_guc_ct_irq_handler(ct);
 		msleep(200);
-		if (g2h->space !=
-		    CIRC_SPACE(0, 0, g2h->size) - g2h->resv_space) {
+		if (g2h->info.space !=
+		    CIRC_SPACE(0, 0, g2h->info.size) - g2h->info.resv_space) {
 			drm_printf(p, "Mismatch on space %d, %d\n",
-				   g2h->space,
-				   CIRC_SPACE(0, 0, g2h->size) -
-				   g2h->resv_space);
+				   g2h->info.space,
+				   CIRC_SPACE(0, 0, g2h->info.size) -
+				   g2h->info.resv_space);
 			ret = -EIO;
 		}
 		if (ct->g2h_outstanding) {
@@ -1185,12 +1192,12 @@ void xe_guc_ct_selftest(struct xe_guc_ct *ct, struct drm_printer *p)
 
 	/* Check failure path for blocking CTs too */
 	xe_guc_ct_send_block(ct, bad_action, ARRAY_SIZE(bad_action));
-	if (g2h->space !=
-	    CIRC_SPACE(0, 0, g2h->size) - g2h->resv_space) {
+	if (g2h->info.space !=
+	    CIRC_SPACE(0, 0, g2h->info.size) - g2h->info.resv_space) {
 		drm_printf(p, "Mismatch on space %d, %d\n",
-			   g2h->space,
-			   CIRC_SPACE(0, 0, g2h->size) -
-			   g2h->resv_space);
+			   g2h->info.space,
+			   CIRC_SPACE(0, 0, g2h->info.size) -
+			   g2h->info.resv_space);
 		ret = -EIO;
 	}
 	if (ct->g2h_outstanding) {
diff --git a/drivers/gpu/drm/xe/xe_guc_ct_types.h b/drivers/gpu/drm/xe/xe_guc_ct_types.h
index fd27dacf00c5..64e3dd14d4b2 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct_types.h
+++ b/drivers/gpu/drm/xe/xe_guc_ct_types.h
@@ -19,13 +19,9 @@
 struct xe_bo;
 
 /**
- * struct guc_ctb - GuC command transport buffer (CTB)
+ * struct guc_ctb_info - GuC command transport buffer (CTB) info
  */
-struct guc_ctb {
-	/** @desc: dma buffer map for CTB descriptor */
-	struct iosys_map desc;
-	/** @cmds: dma buffer map for CTB commands */
-	struct iosys_map cmds;
+struct guc_ctb_info {
 	/** @size: size of CTB commands (DW) */
 	u32 size;
 	/** @resv_space: reserved space of CTB commands (DW) */
@@ -40,6 +36,18 @@ struct guc_ctb {
 	bool broken;
 };
 
+/**
+ * struct guc_ctb - GuC command transport buffer (CTB)
+ */
+struct guc_ctb {
+	/** @desc: dma buffer map for CTB descriptor */
+	struct iosys_map desc;
+	/** @cmds: dma buffer map for CTB commands */
+	struct iosys_map cmds;
+	/** @info: CTB info */
+	struct guc_ctb_info info;
+};
+
 /**
  * struct xe_guc_ct - GuC command transport (CT) layer
  *
-- 
2.39.2

Re: [Intel-xe] [PATCH 04/14] drm/xe: Extract non mapped regions out of GuC CTB into its own struct.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:03PM -0400, Rodrigo Vivi wrote:
> No functional change here. The goal is to have a clear split between
> the mapped portions of the CTB and the static information, so we can
> easily capture snapshots that will be used for later read out with
> the devcoredump infrastructure.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Good clean up. Let's adopt this style everywhere in Xe going forward.

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_guc_ct.c       | 155 ++++++++++++++-------------
>  drivers/gpu/drm/xe/xe_guc_ct_types.h |  20 ++--
>  2 files changed, 95 insertions(+), 80 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
> index 9055ff133a7c..e16e5fe37ed4 100644
> --- a/drivers/gpu/drm/xe/xe_guc_ct.c
> +++ b/drivers/gpu/drm/xe/xe_guc_ct.c
> @@ -172,13 +172,14 @@ int xe_guc_ct_init(struct xe_guc_ct *ct)
>  static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g,
>  				struct iosys_map *map)
>  {
> -	h2g->size = CTB_H2G_BUFFER_SIZE / sizeof(u32);
> -	h2g->resv_space = 0;
> -	h2g->tail = 0;
> -	h2g->head = 0;
> -	h2g->space = CIRC_SPACE(h2g->tail, h2g->head, h2g->size) -
> -		h2g->resv_space;
> -	h2g->broken = false;
> +	h2g->info.size = CTB_H2G_BUFFER_SIZE / sizeof(u32);
> +	h2g->info.resv_space = 0;
> +	h2g->info.tail = 0;
> +	h2g->info.head = 0;
> +	h2g->info.space = CIRC_SPACE(h2g->info.tail, h2g->info.head,
> +				     h2g->info.size) -
> +			  h2g->info.resv_space;
> +	h2g->info.broken = false;
>  
>  	h2g->desc = *map;
>  	xe_map_memset(xe, &h2g->desc, 0, 0, sizeof(struct guc_ct_buffer_desc));
> @@ -189,13 +190,14 @@ static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g,
>  static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h,
>  				struct iosys_map *map)
>  {
> -	g2h->size = CTB_G2H_BUFFER_SIZE / sizeof(u32);
> -	g2h->resv_space = G2H_ROOM_BUFFER_SIZE / sizeof(u32);
> -	g2h->head = 0;
> -	g2h->tail = 0;
> -	g2h->space = CIRC_SPACE(g2h->tail, g2h->head, g2h->size) -
> -		g2h->resv_space;
> -	g2h->broken = false;
> +	g2h->info.size = CTB_G2H_BUFFER_SIZE / sizeof(u32);
> +	g2h->info.resv_space = G2H_ROOM_BUFFER_SIZE / sizeof(u32);
> +	g2h->info.head = 0;
> +	g2h->info.tail = 0;
> +	g2h->info.space = CIRC_SPACE(g2h->info.tail, g2h->info.head,
> +				     g2h->info.size) -
> +			  g2h->info.resv_space;
> +	g2h->info.broken = false;
>  
>  	g2h->desc = IOSYS_MAP_INIT_OFFSET(map, CTB_DESC_SIZE);
>  	xe_map_memset(xe, &g2h->desc, 0, 0, sizeof(struct guc_ct_buffer_desc));
> @@ -212,7 +214,7 @@ static int guc_ct_ctb_h2g_register(struct xe_guc_ct *ct)
>  
>  	desc_addr = xe_bo_ggtt_addr(ct->bo);
>  	ctb_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE * 2;
> -	size = ct->ctbs.h2g.size * sizeof(u32);
> +	size = ct->ctbs.h2g.info.size * sizeof(u32);
>  
>  	err = xe_guc_self_cfg64(guc,
>  				GUC_KLV_SELF_CFG_H2G_CTB_DESCRIPTOR_ADDR_KEY,
> @@ -240,7 +242,7 @@ static int guc_ct_ctb_g2h_register(struct xe_guc_ct *ct)
>  	desc_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE;
>  	ctb_addr = xe_bo_ggtt_addr(ct->bo) + CTB_DESC_SIZE * 2 +
>  		CTB_H2G_BUFFER_SIZE;
> -	size = ct->ctbs.g2h.size * sizeof(u32);
> +	size = ct->ctbs.g2h.info.size * sizeof(u32);
>  
>  	err = xe_guc_self_cfg64(guc,
>  				GUC_KLV_SELF_CFG_G2H_CTB_DESCRIPTOR_ADDR_KEY,
> @@ -329,11 +331,12 @@ static bool h2g_has_room(struct xe_guc_ct *ct, u32 cmd_len)
>  
>  	lockdep_assert_held(&ct->lock);
>  
> -	if (cmd_len > h2g->space) {
> -		h2g->head = desc_read(ct_to_xe(ct), h2g, head);
> -		h2g->space = CIRC_SPACE(h2g->tail, h2g->head, h2g->size) -
> -			h2g->resv_space;
> -		if (cmd_len > h2g->space)
> +	if (cmd_len > h2g->info.space) {
> +		h2g->info.head = desc_read(ct_to_xe(ct), h2g, head);
> +		h2g->info.space = CIRC_SPACE(h2g->info.tail, h2g->info.head,
> +					     h2g->info.size) -
> +				  h2g->info.resv_space;
> +		if (cmd_len > h2g->info.space)
>  			return false;
>  	}
>  
> @@ -344,7 +347,7 @@ static bool g2h_has_room(struct xe_guc_ct *ct, u32 g2h_len)
>  {
>  	lockdep_assert_held(&ct->lock);
>  
> -	return ct->ctbs.g2h.space > g2h_len;
> +	return ct->ctbs.g2h.info.space > g2h_len;
>  }
>  
>  static int has_room(struct xe_guc_ct *ct, u32 cmd_len, u32 g2h_len)
> @@ -360,16 +363,16 @@ static int has_room(struct xe_guc_ct *ct, u32 cmd_len, u32 g2h_len)
>  static void h2g_reserve_space(struct xe_guc_ct *ct, u32 cmd_len)
>  {
>  	lockdep_assert_held(&ct->lock);
> -	ct->ctbs.h2g.space -= cmd_len;
> +	ct->ctbs.h2g.info.space -= cmd_len;
>  }
>  
>  static void g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h)
>  {
> -	XE_BUG_ON(g2h_len > ct->ctbs.g2h.space);
> +	XE_BUG_ON(g2h_len > ct->ctbs.g2h.info.space);
>  
>  	if (g2h_len) {
>  		spin_lock_irq(&ct->fast_lock);
> -		ct->ctbs.g2h.space -= g2h_len;
> +		ct->ctbs.g2h.info.space -= g2h_len;
>  		ct->g2h_outstanding += num_g2h;
>  		spin_unlock_irq(&ct->fast_lock);
>  	}
> @@ -378,10 +381,10 @@ static void g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h)
>  static void __g2h_release_space(struct xe_guc_ct *ct, u32 g2h_len)
>  {
>  	lockdep_assert_held(&ct->fast_lock);
> -	XE_WARN_ON(ct->ctbs.g2h.space + g2h_len >
> -		   ct->ctbs.g2h.size - ct->ctbs.g2h.resv_space);
> +	XE_WARN_ON(ct->ctbs.g2h.info.space + g2h_len >
> +		   ct->ctbs.g2h.info.size - ct->ctbs.g2h.info.resv_space);
>  
> -	ct->ctbs.g2h.space += g2h_len;
> +	ct->ctbs.g2h.info.space += g2h_len;
>  	--ct->g2h_outstanding;
>  }
>  
> @@ -400,20 +403,21 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
>  	u32 cmd[GUC_CTB_MSG_MAX_LEN / sizeof(u32)];
>  	u32 cmd_len = len + GUC_CTB_HDR_LEN;
>  	u32 cmd_idx = 0, i;
> -	u32 tail = h2g->tail;
> +	u32 tail = h2g->info.tail;
>  	struct iosys_map map = IOSYS_MAP_INIT_OFFSET(&h2g->cmds,
>  							 tail * sizeof(u32));
>  
>  	lockdep_assert_held(&ct->lock);
>  	XE_BUG_ON(len * sizeof(u32) > GUC_CTB_MSG_MAX_LEN);
> -	XE_BUG_ON(tail > h2g->size);
> +	XE_BUG_ON(tail > h2g->info.size);
>  
>  	/* Command will wrap, zero fill (NOPs), return and check credits again */
> -	if (tail + cmd_len > h2g->size) {
> -		xe_map_memset(xe, &map, 0, 0, (h2g->size - tail) * sizeof(u32));
> -		h2g_reserve_space(ct, (h2g->size - tail));
> -		h2g->tail = 0;
> -		desc_write(xe, h2g, tail, h2g->tail);
> +	if (tail + cmd_len > h2g->info.size) {
> +		xe_map_memset(xe, &map, 0, 0,
> +			      (h2g->info.size - tail) * sizeof(u32));
> +		h2g_reserve_space(ct, (h2g->info.size - tail));
> +		h2g->info.tail = 0;
> +		desc_write(xe, h2g, tail, h2g->info.tail);
>  
>  		return -EAGAIN;
>  	}
> @@ -445,11 +449,11 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
>  	xe_device_wmb(ct_to_xe(ct));
>  
>  	/* Update local copies */
> -	h2g->tail = (tail + cmd_len) % h2g->size;
> +	h2g->info.tail = (tail + cmd_len) % h2g->info.size;
>  	h2g_reserve_space(ct, cmd_len);
>  
>  	/* Update descriptor */
> -	desc_write(xe, h2g, tail, h2g->tail);
> +	desc_write(xe, h2g, tail, h2g->info.tail);
>  
>  	return 0;
>  }
> @@ -466,7 +470,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
>  	XE_BUG_ON(!g2h_len && num_g2h);
>  	lockdep_assert_held(&ct->lock);
>  
> -	if (unlikely(ct->ctbs.h2g.broken)) {
> +	if (unlikely(ct->ctbs.h2g.info.broken)) {
>  		ret = -EPIPE;
>  		goto out;
>  	}
> @@ -554,8 +558,9 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
>  		if (sleep_period_ms == 1024)
>  			goto broken;
>  
> -		trace_xe_guc_ct_h2g_flow_control(h2g->head, h2g->tail,
> -						 h2g->size, h2g->space,
> +		trace_xe_guc_ct_h2g_flow_control(h2g->info.head, h2g->info.tail,
> +						 h2g->info.size,
> +						 h2g->info.space,
>  						 len + GUC_CTB_HDR_LEN);
>  		msleep(sleep_period_ms);
>  		sleep_period_ms <<= 1;
> @@ -565,15 +570,16 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
>  		struct xe_device *xe = ct_to_xe(ct);
>  		struct guc_ctb *g2h = &ct->ctbs.g2h;
>  
> -		trace_xe_guc_ct_g2h_flow_control(g2h->head,
> +		trace_xe_guc_ct_g2h_flow_control(g2h->info.head,
>  						 desc_read(xe, g2h, tail),
> -						 g2h->size, g2h->space,
> +						 g2h->info.size,
> +						 g2h->info.space,
>  						 g2h_fence ?
>  						 GUC_CTB_HXG_MSG_MAX_LEN :
>  						 g2h_len);
>  
>  #define g2h_avail(ct)	\
> -	(desc_read(ct_to_xe(ct), (&ct->ctbs.g2h), tail) != ct->ctbs.g2h.head)
> +	(desc_read(ct_to_xe(ct), (&ct->ctbs.g2h), tail) != ct->ctbs.g2h.info.head)
>  		if (!wait_event_timeout(ct->wq, !ct->g2h_outstanding ||
>  					g2h_avail(ct), HZ))
>  			goto broken;
> @@ -590,7 +596,7 @@ static int guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len,
>  broken:
>  	drm_err(drm, "No forward process on H2G, reset required");
>  	xe_guc_ct_print(ct, &p);
> -	ct->ctbs.h2g.broken = true;
> +	ct->ctbs.h2g.info.broken = true;
>  
>  	return -EDEADLK;
>  }
> @@ -656,7 +662,7 @@ static bool retry_failure(struct xe_guc_ct *ct, int ret)
>  		return false;
>  
>  #define ct_alive(ct)	\
> -	(ct->enabled && !ct->ctbs.h2g.broken && !ct->ctbs.g2h.broken)
> +	(ct->enabled && !ct->ctbs.h2g.info.broken && !ct->ctbs.g2h.info.broken)
>  	if (!wait_event_interruptible_timeout(ct->wq, ct_alive(ct),  HZ * 5))
>  		return false;
>  #undef ct_alive
> @@ -821,7 +827,7 @@ static int parse_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len)
>  		drm_err(&xe->drm,
>  			"G2H channel broken on read, origin=%d, reset required\n",
>  			origin);
> -		ct->ctbs.g2h.broken = true;
> +		ct->ctbs.g2h.info.broken = true;
>  
>  		return -EPROTO;
>  	}
> @@ -840,7 +846,7 @@ static int parse_g2h_msg(struct xe_guc_ct *ct, u32 *msg, u32 len)
>  		drm_err(&xe->drm,
>  			"G2H channel broken on read, type=%d, reset required\n",
>  			type);
> -		ct->ctbs.g2h.broken = true;
> +		ct->ctbs.g2h.info.broken = true;
>  
>  		ret = -EOPNOTSUPP;
>  	}
> @@ -919,36 +925,37 @@ static int g2h_read(struct xe_guc_ct *ct, u32 *msg, bool fast_path)
>  	if (!ct->enabled)
>  		return -ENODEV;
>  
> -	if (g2h->broken)
> +	if (g2h->info.broken)
>  		return -EPIPE;
>  
>  	/* Calculate DW available to read */
>  	tail = desc_read(xe, g2h, tail);
> -	avail = tail - g2h->head;
> +	avail = tail - g2h->info.head;
>  	if (unlikely(avail == 0))
>  		return 0;
>  
>  	if (avail < 0)
> -		avail += g2h->size;
> +		avail += g2h->info.size;
>  
>  	/* Read header */
> -	xe_map_memcpy_from(xe, msg, &g2h->cmds, sizeof(u32) * g2h->head, sizeof(u32));
> +	xe_map_memcpy_from(xe, msg, &g2h->cmds, sizeof(u32) * g2h->info.head,
> +			   sizeof(u32));
>  	len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN;
>  	if (len > avail) {
>  		drm_err(&xe->drm,
>  			"G2H channel broken on read, avail=%d, len=%d, reset required\n",
>  			avail, len);
> -		g2h->broken = true;
> +		g2h->info.broken = true;
>  
>  		return -EPROTO;
>  	}
>  
> -	head = (g2h->head + 1) % g2h->size;
> +	head = (g2h->info.head + 1) % g2h->info.size;
>  	avail = len - 1;
>  
>  	/* Read G2H message */
> -	if (avail + head > g2h->size) {
> -		u32 avail_til_wrap = g2h->size - head;
> +	if (avail + head > g2h->info.size) {
> +		u32 avail_til_wrap = g2h->info.size - head;
>  
>  		xe_map_memcpy_from(xe, msg + 1,
>  				   &g2h->cmds, sizeof(u32) * head,
> @@ -983,8 +990,8 @@ static int g2h_read(struct xe_guc_ct *ct, u32 *msg, bool fast_path)
>  	}
>  
>  	/* Update local / descriptor header */
> -	g2h->head = (head + avail) % g2h->size;
> -	desc_write(xe, g2h, head, g2h->head);
> +	g2h->info.head = (head + avail) % g2h->info.size;
> +	desc_write(xe, g2h, head, g2h->info.head);
>  
>  	return len;
>  }
> @@ -1093,12 +1100,12 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
>  {
>  	u32 head, tail;
>  
> -	drm_printf(p, "\tsize: %d\n", ctb->size);
> -	drm_printf(p, "\tresv_space: %d\n", ctb->resv_space);
> -	drm_printf(p, "\thead: %d\n", ctb->head);
> -	drm_printf(p, "\ttail: %d\n", ctb->tail);
> -	drm_printf(p, "\tspace: %d\n", ctb->space);
> -	drm_printf(p, "\tbroken: %d\n", ctb->broken);
> +	drm_printf(p, "\tsize: %d\n", ctb->info.size);
> +	drm_printf(p, "\tresv_space: %d\n", ctb->info.resv_space);
> +	drm_printf(p, "\thead: %d\n", ctb->info.head);
> +	drm_printf(p, "\ttail: %d\n", ctb->info.tail);
> +	drm_printf(p, "\tspace: %d\n", ctb->info.space);
> +	drm_printf(p, "\tbroken: %d\n", ctb->info.broken);
>  
>  	head = desc_read(xe, ctb, head);
>  	tail = desc_read(xe, ctb, tail);
> @@ -1114,7 +1121,7 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
>  			drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
>  				   xe_map_rd(xe, &map, 0, u32));
>  			++head;
> -			if (head == ctb->size) {
> +			if (head == ctb->info.size) {
>  				head = 0;
>  				map = ctb->cmds;
>  			} else {
> @@ -1168,12 +1175,12 @@ void xe_guc_ct_selftest(struct xe_guc_ct *ct, struct drm_printer *p)
>  	if (!ret) {
>  		xe_guc_ct_irq_handler(ct);
>  		msleep(200);
> -		if (g2h->space !=
> -		    CIRC_SPACE(0, 0, g2h->size) - g2h->resv_space) {
> +		if (g2h->info.space !=
> +		    CIRC_SPACE(0, 0, g2h->info.size) - g2h->info.resv_space) {
>  			drm_printf(p, "Mismatch on space %d, %d\n",
> -				   g2h->space,
> -				   CIRC_SPACE(0, 0, g2h->size) -
> -				   g2h->resv_space);
> +				   g2h->info.space,
> +				   CIRC_SPACE(0, 0, g2h->info.size) -
> +				   g2h->info.resv_space);
>  			ret = -EIO;
>  		}
>  		if (ct->g2h_outstanding) {
> @@ -1185,12 +1192,12 @@ void xe_guc_ct_selftest(struct xe_guc_ct *ct, struct drm_printer *p)
>  
>  	/* Check failure path for blocking CTs too */
>  	xe_guc_ct_send_block(ct, bad_action, ARRAY_SIZE(bad_action));
> -	if (g2h->space !=
> -	    CIRC_SPACE(0, 0, g2h->size) - g2h->resv_space) {
> +	if (g2h->info.space !=
> +	    CIRC_SPACE(0, 0, g2h->info.size) - g2h->info.resv_space) {
>  		drm_printf(p, "Mismatch on space %d, %d\n",
> -			   g2h->space,
> -			   CIRC_SPACE(0, 0, g2h->size) -
> -			   g2h->resv_space);
> +			   g2h->info.space,
> +			   CIRC_SPACE(0, 0, g2h->info.size) -
> +			   g2h->info.resv_space);
>  		ret = -EIO;
>  	}
>  	if (ct->g2h_outstanding) {
> diff --git a/drivers/gpu/drm/xe/xe_guc_ct_types.h b/drivers/gpu/drm/xe/xe_guc_ct_types.h
> index fd27dacf00c5..64e3dd14d4b2 100644
> --- a/drivers/gpu/drm/xe/xe_guc_ct_types.h
> +++ b/drivers/gpu/drm/xe/xe_guc_ct_types.h
> @@ -19,13 +19,9 @@
>  struct xe_bo;
>  
>  /**
> - * struct guc_ctb - GuC command transport buffer (CTB)
> + * struct guc_ctb_info - GuC command transport buffer (CTB) info
>   */
> -struct guc_ctb {
> -	/** @desc: dma buffer map for CTB descriptor */
> -	struct iosys_map desc;
> -	/** @cmds: dma buffer map for CTB commands */
> -	struct iosys_map cmds;
> +struct guc_ctb_info {
>  	/** @size: size of CTB commands (DW) */
>  	u32 size;
>  	/** @resv_space: reserved space of CTB commands (DW) */
> @@ -40,6 +36,18 @@ struct guc_ctb {
>  	bool broken;
>  };
>  
> +/**
> + * struct guc_ctb - GuC command transport buffer (CTB)
> + */
> +struct guc_ctb {
> +	/** @desc: dma buffer map for CTB descriptor */
> +	struct iosys_map desc;
> +	/** @cmds: dma buffer map for CTB commands */
> +	struct iosys_map cmds;
> +	/** @info: CTB info */
> +	struct guc_ctb_info info;
> +};
> +
>  /**
>   * struct xe_guc_ct - GuC command transport (CT) layer
>   *
> -- 
> 2.39.2
> 

[PATCH 05/14] drm/xe: Convert GuC CT print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_ct.c       | 132 +++++++++++++++++++++++----
 drivers/gpu/drm/xe/xe_guc_ct.h       |   7 +-
 drivers/gpu/drm/xe/xe_guc_ct_types.h |  26 ++++++
 3 files changed, 145 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
index e16e5fe37ed4..0b7b95dbd9be 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct.c
+++ b/drivers/gpu/drm/xe/xe_guc_ct.c
@@ -1095,31 +1095,26 @@ static void g2h_worker_func(struct work_struct *w)
 	xe_device_mem_access_put(ct_to_xe(ct));
 }
 
-static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
-			     struct drm_printer *p)
+static void guc_ctb_snapshot_capture(struct xe_device *xe, struct guc_ctb *ctb,
+				     struct guc_ctb_snapshot *snapshot)
 {
 	u32 head, tail;
 
-	drm_printf(p, "\tsize: %d\n", ctb->info.size);
-	drm_printf(p, "\tresv_space: %d\n", ctb->info.resv_space);
-	drm_printf(p, "\thead: %d\n", ctb->info.head);
-	drm_printf(p, "\ttail: %d\n", ctb->info.tail);
-	drm_printf(p, "\tspace: %d\n", ctb->info.space);
-	drm_printf(p, "\tbroken: %d\n", ctb->info.broken);
+	snapshot->cmds = kmalloc_array(ctb->info.size, sizeof(u32), GFP_ATOMIC);
 
-	head = desc_read(xe, ctb, head);
-	tail = desc_read(xe, ctb, tail);
-	drm_printf(p, "\thead (memory): %d\n", head);
-	drm_printf(p, "\ttail (memory): %d\n", tail);
-	drm_printf(p, "\tstatus (memory): 0x%x\n", desc_read(xe, ctb, status));
+	xe_map_memcpy_from(xe, &snapshot->desc, &ctb->desc, 0,
+			   sizeof(struct guc_ct_buffer_desc));
+	memcpy(&snapshot->info, &ctb->info, sizeof(struct guc_ctb_info));
+
+	head = snapshot->desc.head;
+	tail = snapshot->desc.tail;
 
 	if (head != tail) {
 		struct iosys_map map =
 			IOSYS_MAP_INIT_OFFSET(&ctb->cmds, head * sizeof(u32));
 
 		while (head != tail) {
-			drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
-				   xe_map_rd(xe, &map, 0, u32));
+			snapshot->cmds[head] = xe_map_rd(xe, &map, 0, u32);
 			++head;
 			if (head == ctb->info.size) {
 				head = 0;
@@ -1131,20 +1126,119 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
 	}
 }
 
-void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p)
+static void guc_ctb_snapshot_print(struct guc_ctb_snapshot *snapshot,
+				   struct drm_printer *p)
+{
+	u32 head, tail;
+
+	drm_printf(p, "\tsize: %d\n", snapshot->info.size);
+	drm_printf(p, "\tresv_space: %d\n", snapshot->info.space);
+	drm_printf(p, "\thead: %d\n", snapshot->info.head);
+	drm_printf(p, "\ttail: %d\n", snapshot->info.tail);
+	drm_printf(p, "\tspace: %d\n", snapshot->info.space);
+	drm_printf(p, "\tbroken: %d\n", snapshot->info.broken);
+	drm_printf(p, "\thead (memory): %d\n", snapshot->desc.head);
+	drm_printf(p, "\ttail (memory): %d\n", snapshot->desc.tail);
+	drm_printf(p, "\tstatus (memory): 0x%x\n", snapshot->desc.status);
+
+	head = snapshot->desc.head;
+	tail = snapshot->desc.tail;
+
+	while (head != tail) {
+		drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
+			   snapshot->cmds[head]);
+		++head;
+		if (head == snapshot->info.size)
+			head = 0;
+	}
+}
+
+static void guc_ctb_snapshot_free(struct guc_ctb_snapshot *snapshot)
+{
+	kfree(snapshot->cmds);
+}
+
+/**
+ * xe_guc_ct_snapshot_capture - Take a quick snapshot of the CT state.
+ * @ct: GuC CT object.
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a GuC CT snapshot object that must be freed by the caller
+ * 	    by using `xe_guc_ct_snapshot_free`.
+ */
+struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct)
 {
+	struct xe_guc_ct_snapshot *snapshot;
+
+	snapshot = kzalloc(sizeof(struct xe_guc_ct_snapshot), GFP_ATOMIC);
+
 	if (ct->enabled) {
+		snapshot->ct_enabled = true;
+		guc_ctb_snapshot_capture(ct_to_xe(ct), &ct->ctbs.h2g,
+					 &snapshot->h2g);
+		guc_ctb_snapshot_capture(ct_to_xe(ct), &ct->ctbs.g2h,
+					 &snapshot->g2h);
+	}
+
+	return snapshot;
+}
+
+/**
+ * xe_guc_ct_snapshot_print - Print out a given GuC CT snapshot.
+ * @snapshot: GuC CT snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given GuC CT snapshot object.
+ */
+void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot,
+			      struct drm_printer *p)
+{
+	if (snapshot->ct_enabled) {
 		drm_puts(p, "\nH2G CTB (all sizes in DW):\n");
-		guc_ct_ctb_print(ct_to_xe(ct), &ct->ctbs.h2g, p);
+		guc_ctb_snapshot_print(&snapshot->h2g, p);
 
 		drm_puts(p, "\nG2H CTB (all sizes in DW):\n");
-		guc_ct_ctb_print(ct_to_xe(ct), &ct->ctbs.g2h, p);
-		drm_printf(p, "\tg2h outstanding: %d\n", ct->g2h_outstanding);
+		guc_ctb_snapshot_print(&snapshot->g2h, p);
+
+		drm_printf(p, "\tg2h outstanding: %d\n",
+			   snapshot->g2h_outstanding);
 	} else {
 		drm_puts(p, "\nCT disabled\n");
 	}
 }
 
+/**
+ * xe_guc_ct_snapshot_free - Free all allocated objects for a given snapshot.
+ * @snapshot: GuC CT snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot)
+{
+	guc_ctb_snapshot_free(&snapshot->h2g);
+	guc_ctb_snapshot_free(&snapshot->g2h);
+	kfree(snapshot);
+}
+
+/**
+ * xe_guc_ct_print - GuC CT Print.
+ * @ct: GuC CT.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function quickly capture a snapshot and immediately print it out.
+ */
+void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p)
+{
+	struct xe_guc_ct_snapshot *snapshot;
+
+	snapshot = xe_guc_ct_snapshot_capture(ct);
+	xe_guc_ct_snapshot_print(snapshot, p);
+	xe_guc_ct_snapshot_free(snapshot);
+}
+
 #ifdef XE_GUC_CT_SELFTEST
 /*
  * Disable G2H processing in IRQ handler to force xe_guc_ct_send to enter flow
diff --git a/drivers/gpu/drm/xe/xe_guc_ct.h b/drivers/gpu/drm/xe/xe_guc_ct.h
index 49fb74f91e4d..29e0dff7ad9b 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct.h
+++ b/drivers/gpu/drm/xe/xe_guc_ct.h
@@ -13,9 +13,14 @@ struct drm_printer;
 int xe_guc_ct_init(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
-void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p);
 void xe_guc_ct_fast_path(struct xe_guc_ct *ct);
 
+struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct);
+void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot,
+			      struct drm_printer *p);
+void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot);
+void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p);
+
 static inline void xe_guc_ct_irq_handler(struct xe_guc_ct *ct)
 {
 	wake_up_all(&ct->wq);
diff --git a/drivers/gpu/drm/xe/xe_guc_ct_types.h b/drivers/gpu/drm/xe/xe_guc_ct_types.h
index 64e3dd14d4b2..93046d95b009 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct_types.h
+++ b/drivers/gpu/drm/xe/xe_guc_ct_types.h
@@ -48,6 +48,32 @@ struct guc_ctb {
 	struct guc_ctb_info info;
 };
 
+/**
+ * struct guc_ctb_snapshot - GuC command transport buffer (CTB) snapshot
+ */
+struct guc_ctb_snapshot {
+	/** @desc: snapshot of the CTB descriptor */
+	struct guc_ct_buffer_desc desc;
+	/** @cmds: snapshot of the CTB commands */
+	u32 *cmds;
+	/** @info: snapshot of the CTB info */
+	struct guc_ctb_info info;
+};
+
+/**
+ * struct xe_guc_ct_snapshot - GuC command transport (CT) snapshot
+ */
+struct xe_guc_ct_snapshot {
+	/** @ct_enabled: CT enabled info at capture time. */
+	bool ct_enabled;
+	/** @g2h_outstanding: G2H outstanding info at the capture time */
+	u32 g2h_outstanding;
+	/** @g2h: G2H CTB snapshot */
+	struct guc_ctb_snapshot g2h;
+	/** @h2g: H2G CTB snapshot */
+	struct guc_ctb_snapshot h2g;
+};
+
 /**
  * struct xe_guc_ct - GuC command transport (CT) layer
  *
-- 
2.39.2

[Intel-xe] [PATCH 05/14] drm/xe: Convert GuC CT print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_ct.c       | 132 +++++++++++++++++++++++----
 drivers/gpu/drm/xe/xe_guc_ct.h       |   7 +-
 drivers/gpu/drm/xe/xe_guc_ct_types.h |  26 ++++++
 3 files changed, 145 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
index e16e5fe37ed4..0b7b95dbd9be 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct.c
+++ b/drivers/gpu/drm/xe/xe_guc_ct.c
@@ -1095,31 +1095,26 @@ static void g2h_worker_func(struct work_struct *w)
 	xe_device_mem_access_put(ct_to_xe(ct));
 }
 
-static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
-			     struct drm_printer *p)
+static void guc_ctb_snapshot_capture(struct xe_device *xe, struct guc_ctb *ctb,
+				     struct guc_ctb_snapshot *snapshot)
 {
 	u32 head, tail;
 
-	drm_printf(p, "\tsize: %d\n", ctb->info.size);
-	drm_printf(p, "\tresv_space: %d\n", ctb->info.resv_space);
-	drm_printf(p, "\thead: %d\n", ctb->info.head);
-	drm_printf(p, "\ttail: %d\n", ctb->info.tail);
-	drm_printf(p, "\tspace: %d\n", ctb->info.space);
-	drm_printf(p, "\tbroken: %d\n", ctb->info.broken);
+	snapshot->cmds = kmalloc_array(ctb->info.size, sizeof(u32), GFP_ATOMIC);
 
-	head = desc_read(xe, ctb, head);
-	tail = desc_read(xe, ctb, tail);
-	drm_printf(p, "\thead (memory): %d\n", head);
-	drm_printf(p, "\ttail (memory): %d\n", tail);
-	drm_printf(p, "\tstatus (memory): 0x%x\n", desc_read(xe, ctb, status));
+	xe_map_memcpy_from(xe, &snapshot->desc, &ctb->desc, 0,
+			   sizeof(struct guc_ct_buffer_desc));
+	memcpy(&snapshot->info, &ctb->info, sizeof(struct guc_ctb_info));
+
+	head = snapshot->desc.head;
+	tail = snapshot->desc.tail;
 
 	if (head != tail) {
 		struct iosys_map map =
 			IOSYS_MAP_INIT_OFFSET(&ctb->cmds, head * sizeof(u32));
 
 		while (head != tail) {
-			drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
-				   xe_map_rd(xe, &map, 0, u32));
+			snapshot->cmds[head] = xe_map_rd(xe, &map, 0, u32);
 			++head;
 			if (head == ctb->info.size) {
 				head = 0;
@@ -1131,20 +1126,119 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
 	}
 }
 
-void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p)
+static void guc_ctb_snapshot_print(struct guc_ctb_snapshot *snapshot,
+				   struct drm_printer *p)
+{
+	u32 head, tail;
+
+	drm_printf(p, "\tsize: %d\n", snapshot->info.size);
+	drm_printf(p, "\tresv_space: %d\n", snapshot->info.space);
+	drm_printf(p, "\thead: %d\n", snapshot->info.head);
+	drm_printf(p, "\ttail: %d\n", snapshot->info.tail);
+	drm_printf(p, "\tspace: %d\n", snapshot->info.space);
+	drm_printf(p, "\tbroken: %d\n", snapshot->info.broken);
+	drm_printf(p, "\thead (memory): %d\n", snapshot->desc.head);
+	drm_printf(p, "\ttail (memory): %d\n", snapshot->desc.tail);
+	drm_printf(p, "\tstatus (memory): 0x%x\n", snapshot->desc.status);
+
+	head = snapshot->desc.head;
+	tail = snapshot->desc.tail;
+
+	while (head != tail) {
+		drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
+			   snapshot->cmds[head]);
+		++head;
+		if (head == snapshot->info.size)
+			head = 0;
+	}
+}
+
+static void guc_ctb_snapshot_free(struct guc_ctb_snapshot *snapshot)
+{
+	kfree(snapshot->cmds);
+}
+
+/**
+ * xe_guc_ct_snapshot_capture - Take a quick snapshot of the CT state.
+ * @ct: GuC CT object.
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a GuC CT snapshot object that must be freed by the caller
+ * 	    by using `xe_guc_ct_snapshot_free`.
+ */
+struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct)
 {
+	struct xe_guc_ct_snapshot *snapshot;
+
+	snapshot = kzalloc(sizeof(struct xe_guc_ct_snapshot), GFP_ATOMIC);
+
 	if (ct->enabled) {
+		snapshot->ct_enabled = true;
+		guc_ctb_snapshot_capture(ct_to_xe(ct), &ct->ctbs.h2g,
+					 &snapshot->h2g);
+		guc_ctb_snapshot_capture(ct_to_xe(ct), &ct->ctbs.g2h,
+					 &snapshot->g2h);
+	}
+
+	return snapshot;
+}
+
+/**
+ * xe_guc_ct_snapshot_print - Print out a given GuC CT snapshot.
+ * @snapshot: GuC CT snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given GuC CT snapshot object.
+ */
+void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot,
+			      struct drm_printer *p)
+{
+	if (snapshot->ct_enabled) {
 		drm_puts(p, "\nH2G CTB (all sizes in DW):\n");
-		guc_ct_ctb_print(ct_to_xe(ct), &ct->ctbs.h2g, p);
+		guc_ctb_snapshot_print(&snapshot->h2g, p);
 
 		drm_puts(p, "\nG2H CTB (all sizes in DW):\n");
-		guc_ct_ctb_print(ct_to_xe(ct), &ct->ctbs.g2h, p);
-		drm_printf(p, "\tg2h outstanding: %d\n", ct->g2h_outstanding);
+		guc_ctb_snapshot_print(&snapshot->g2h, p);
+
+		drm_printf(p, "\tg2h outstanding: %d\n",
+			   snapshot->g2h_outstanding);
 	} else {
 		drm_puts(p, "\nCT disabled\n");
 	}
 }
 
+/**
+ * xe_guc_ct_snapshot_free - Free all allocated objects for a given snapshot.
+ * @snapshot: GuC CT snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot)
+{
+	guc_ctb_snapshot_free(&snapshot->h2g);
+	guc_ctb_snapshot_free(&snapshot->g2h);
+	kfree(snapshot);
+}
+
+/**
+ * xe_guc_ct_print - GuC CT Print.
+ * @ct: GuC CT.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function quickly capture a snapshot and immediately print it out.
+ */
+void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p)
+{
+	struct xe_guc_ct_snapshot *snapshot;
+
+	snapshot = xe_guc_ct_snapshot_capture(ct);
+	xe_guc_ct_snapshot_print(snapshot, p);
+	xe_guc_ct_snapshot_free(snapshot);
+}
+
 #ifdef XE_GUC_CT_SELFTEST
 /*
  * Disable G2H processing in IRQ handler to force xe_guc_ct_send to enter flow
diff --git a/drivers/gpu/drm/xe/xe_guc_ct.h b/drivers/gpu/drm/xe/xe_guc_ct.h
index 49fb74f91e4d..29e0dff7ad9b 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct.h
+++ b/drivers/gpu/drm/xe/xe_guc_ct.h
@@ -13,9 +13,14 @@ struct drm_printer;
 int xe_guc_ct_init(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
-void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p);
 void xe_guc_ct_fast_path(struct xe_guc_ct *ct);
 
+struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct);
+void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot,
+			      struct drm_printer *p);
+void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot);
+void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p);
+
 static inline void xe_guc_ct_irq_handler(struct xe_guc_ct *ct)
 {
 	wake_up_all(&ct->wq);
diff --git a/drivers/gpu/drm/xe/xe_guc_ct_types.h b/drivers/gpu/drm/xe/xe_guc_ct_types.h
index 64e3dd14d4b2..93046d95b009 100644
--- a/drivers/gpu/drm/xe/xe_guc_ct_types.h
+++ b/drivers/gpu/drm/xe/xe_guc_ct_types.h
@@ -48,6 +48,32 @@ struct guc_ctb {
 	struct guc_ctb_info info;
 };
 
+/**
+ * struct guc_ctb_snapshot - GuC command transport buffer (CTB) snapshot
+ */
+struct guc_ctb_snapshot {
+	/** @desc: snapshot of the CTB descriptor */
+	struct guc_ct_buffer_desc desc;
+	/** @cmds: snapshot of the CTB commands */
+	u32 *cmds;
+	/** @info: snapshot of the CTB info */
+	struct guc_ctb_info info;
+};
+
+/**
+ * struct xe_guc_ct_snapshot - GuC command transport (CT) snapshot
+ */
+struct xe_guc_ct_snapshot {
+	/** @ct_enabled: CT enabled info at capture time. */
+	bool ct_enabled;
+	/** @g2h_outstanding: G2H outstanding info at the capture time */
+	u32 g2h_outstanding;
+	/** @g2h: G2H CTB snapshot */
+	struct guc_ctb_snapshot g2h;
+	/** @h2g: H2G CTB snapshot */
+	struct guc_ctb_snapshot h2g;
+};
+
 /**
  * struct xe_guc_ct - GuC command transport (CT) layer
  *
-- 
2.39.2

Re: [Intel-xe] [PATCH 05/14] drm/xe: Convert GuC CT print to snapshot capture and print.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:04PM -0400, Rodrigo Vivi wrote:
> The goal is to allow for a snapshot capture to be taken at the time
> of the crash, while the print out can happen at a later time through
> the exposed devcoredump virtual device.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_guc_ct.c       | 132 +++++++++++++++++++++++----
>  drivers/gpu/drm/xe/xe_guc_ct.h       |   7 +-
>  drivers/gpu/drm/xe/xe_guc_ct_types.h |  26 ++++++
>  3 files changed, 145 insertions(+), 20 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_guc_ct.c b/drivers/gpu/drm/xe/xe_guc_ct.c
> index e16e5fe37ed4..0b7b95dbd9be 100644
> --- a/drivers/gpu/drm/xe/xe_guc_ct.c
> +++ b/drivers/gpu/drm/xe/xe_guc_ct.c
> @@ -1095,31 +1095,26 @@ static void g2h_worker_func(struct work_struct *w)
>  	xe_device_mem_access_put(ct_to_xe(ct));
>  }
>  
> -static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
> -			     struct drm_printer *p)
> +static void guc_ctb_snapshot_capture(struct xe_device *xe, struct guc_ctb *ctb,
> +				     struct guc_ctb_snapshot *snapshot)
>  {
>  	u32 head, tail;
>  
> -	drm_printf(p, "\tsize: %d\n", ctb->info.size);
> -	drm_printf(p, "\tresv_space: %d\n", ctb->info.resv_space);
> -	drm_printf(p, "\thead: %d\n", ctb->info.head);
> -	drm_printf(p, "\ttail: %d\n", ctb->info.tail);
> -	drm_printf(p, "\tspace: %d\n", ctb->info.space);
> -	drm_printf(p, "\tbroken: %d\n", ctb->info.broken);
> +	snapshot->cmds = kmalloc_array(ctb->info.size, sizeof(u32), GFP_ATOMIC);

So since this GFP_ATOMIC I assume this so we can call this code from the
TDR or a CTB handler (dma fence signaling paths)? Also I don't see
where you check for this allocation failing.

>  
> -	head = desc_read(xe, ctb, head);
> -	tail = desc_read(xe, ctb, tail);
> -	drm_printf(p, "\thead (memory): %d\n", head);
> -	drm_printf(p, "\ttail (memory): %d\n", tail);
> -	drm_printf(p, "\tstatus (memory): 0x%x\n", desc_read(xe, ctb, status));
> +	xe_map_memcpy_from(xe, &snapshot->desc, &ctb->desc, 0,
> +			   sizeof(struct guc_ct_buffer_desc));
> +	memcpy(&snapshot->info, &ctb->info, sizeof(struct guc_ctb_info));
> +
> +	head = snapshot->desc.head;
> +	tail = snapshot->desc.tail;
>  
>  	if (head != tail) {
>  		struct iosys_map map =
>  			IOSYS_MAP_INIT_OFFSET(&ctb->cmds, head * sizeof(u32));
>  
>  		while (head != tail) {
> -			drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
> -				   xe_map_rd(xe, &map, 0, u32));
> +			snapshot->cmds[head] = xe_map_rd(xe, &map, 0, u32);
>  			++head;
>  			if (head == ctb->info.size) {
>  				head = 0;
> @@ -1131,20 +1126,119 @@ static void guc_ct_ctb_print(struct xe_device *xe, struct guc_ctb *ctb,
>  	}
>  }
>  
> -void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p)
> +static void guc_ctb_snapshot_print(struct guc_ctb_snapshot *snapshot,
> +				   struct drm_printer *p)
> +{
> +	u32 head, tail;
> +
> +	drm_printf(p, "\tsize: %d\n", snapshot->info.size);
> +	drm_printf(p, "\tresv_space: %d\n", snapshot->info.space);
> +	drm_printf(p, "\thead: %d\n", snapshot->info.head);
> +	drm_printf(p, "\ttail: %d\n", snapshot->info.tail);
> +	drm_printf(p, "\tspace: %d\n", snapshot->info.space);
> +	drm_printf(p, "\tbroken: %d\n", snapshot->info.broken);
> +	drm_printf(p, "\thead (memory): %d\n", snapshot->desc.head);
> +	drm_printf(p, "\ttail (memory): %d\n", snapshot->desc.tail);
> +	drm_printf(p, "\tstatus (memory): 0x%x\n", snapshot->desc.status);
> +
> +	head = snapshot->desc.head;
> +	tail = snapshot->desc.tail;
> +
> +	while (head != tail) {
> +		drm_printf(p, "\tcmd[%d]: 0x%08x\n", head,
> +			   snapshot->cmds[head]);
> +		++head;
> +		if (head == snapshot->info.size)
> +			head = 0;
> +	}
> +}
> +
> +static void guc_ctb_snapshot_free(struct guc_ctb_snapshot *snapshot)
> +{
> +	kfree(snapshot->cmds);
> +}
> +
> +/**
> + * xe_guc_ct_snapshot_capture - Take a quick snapshot of the CT state.
> + * @ct: GuC CT object.
> + *
> + * This can be printed out in a later stage like during dev_coredump
> + * analysis.
> + *
> + * Returns: a GuC CT snapshot object that must be freed by the caller
> + * 	    by using `xe_guc_ct_snapshot_free`.
> + */
> +struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct)
>  {
> +	struct xe_guc_ct_snapshot *snapshot;
> +
> +	snapshot = kzalloc(sizeof(struct xe_guc_ct_snapshot), GFP_ATOMIC);
> +

Same here, need to check for an alloc failure.

Also maybe we should a flag to switch between GFP_ATOMIC (signaling
path) and GFP_KERNEL (debugfs). In the case above, CMDs might be huge as
deadlock workaround (like 16 MBs or something) so atomic seems risky.

Aside from these comments, I do rather like what you have done here.

Matt

>  	if (ct->enabled) {
> +		snapshot->ct_enabled = true;
> +		guc_ctb_snapshot_capture(ct_to_xe(ct), &ct->ctbs.h2g,
> +					 &snapshot->h2g);
> +		guc_ctb_snapshot_capture(ct_to_xe(ct), &ct->ctbs.g2h,
> +					 &snapshot->g2h);
> +	}
> +
> +	return snapshot;
> +}
> +
> +/**
> + * xe_guc_ct_snapshot_print - Print out a given GuC CT snapshot.
> + * @snapshot: GuC CT snapshot object.
> + * @p: drm_printer where it will be printed out.
> + *
> + * This function prints out a given GuC CT snapshot object.
> + */
> +void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot,
> +			      struct drm_printer *p)
> +{
> +	if (snapshot->ct_enabled) {
>  		drm_puts(p, "\nH2G CTB (all sizes in DW):\n");
> -		guc_ct_ctb_print(ct_to_xe(ct), &ct->ctbs.h2g, p);
> +		guc_ctb_snapshot_print(&snapshot->h2g, p);
>  
>  		drm_puts(p, "\nG2H CTB (all sizes in DW):\n");
> -		guc_ct_ctb_print(ct_to_xe(ct), &ct->ctbs.g2h, p);
> -		drm_printf(p, "\tg2h outstanding: %d\n", ct->g2h_outstanding);
> +		guc_ctb_snapshot_print(&snapshot->g2h, p);
> +
> +		drm_printf(p, "\tg2h outstanding: %d\n",
> +			   snapshot->g2h_outstanding);
>  	} else {
>  		drm_puts(p, "\nCT disabled\n");
>  	}
>  }
>  
> +/**
> + * xe_guc_ct_snapshot_free - Free all allocated objects for a given snapshot.
> + * @snapshot: GuC CT snapshot object.
> + *
> + * This function free all the memory that needed to be allocated at capture
> + * time.
> + */
> +void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot)
> +{
> +	guc_ctb_snapshot_free(&snapshot->h2g);
> +	guc_ctb_snapshot_free(&snapshot->g2h);
> +	kfree(snapshot);
> +}
> +
> +/**
> + * xe_guc_ct_print - GuC CT Print.
> + * @ct: GuC CT.
> + * @p: drm_printer where it will be printed out.
> + *
> + * This function quickly capture a snapshot and immediately print it out.
> + */
> +void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p)
> +{
> +	struct xe_guc_ct_snapshot *snapshot;
> +
> +	snapshot = xe_guc_ct_snapshot_capture(ct);
> +	xe_guc_ct_snapshot_print(snapshot, p);
> +	xe_guc_ct_snapshot_free(snapshot);
> +}
> +
>  #ifdef XE_GUC_CT_SELFTEST
>  /*
>   * Disable G2H processing in IRQ handler to force xe_guc_ct_send to enter flow
> diff --git a/drivers/gpu/drm/xe/xe_guc_ct.h b/drivers/gpu/drm/xe/xe_guc_ct.h
> index 49fb74f91e4d..29e0dff7ad9b 100644
> --- a/drivers/gpu/drm/xe/xe_guc_ct.h
> +++ b/drivers/gpu/drm/xe/xe_guc_ct.h
> @@ -13,9 +13,14 @@ struct drm_printer;
>  int xe_guc_ct_init(struct xe_guc_ct *ct);
>  int xe_guc_ct_enable(struct xe_guc_ct *ct);
>  void xe_guc_ct_disable(struct xe_guc_ct *ct);
> -void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p);
>  void xe_guc_ct_fast_path(struct xe_guc_ct *ct);
>  
> +struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct);
> +void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot,
> +			      struct drm_printer *p);
> +void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot);
> +void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p);
> +
>  static inline void xe_guc_ct_irq_handler(struct xe_guc_ct *ct)
>  {
>  	wake_up_all(&ct->wq);
> diff --git a/drivers/gpu/drm/xe/xe_guc_ct_types.h b/drivers/gpu/drm/xe/xe_guc_ct_types.h
> index 64e3dd14d4b2..93046d95b009 100644
> --- a/drivers/gpu/drm/xe/xe_guc_ct_types.h
> +++ b/drivers/gpu/drm/xe/xe_guc_ct_types.h
> @@ -48,6 +48,32 @@ struct guc_ctb {
>  	struct guc_ctb_info info;
>  };
>  
> +/**
> + * struct guc_ctb_snapshot - GuC command transport buffer (CTB) snapshot
> + */
> +struct guc_ctb_snapshot {
> +	/** @desc: snapshot of the CTB descriptor */
> +	struct guc_ct_buffer_desc desc;
> +	/** @cmds: snapshot of the CTB commands */
> +	u32 *cmds;
> +	/** @info: snapshot of the CTB info */
> +	struct guc_ctb_info info;
> +};
> +
> +/**
> + * struct xe_guc_ct_snapshot - GuC command transport (CT) snapshot
> + */
> +struct xe_guc_ct_snapshot {
> +	/** @ct_enabled: CT enabled info at capture time. */
> +	bool ct_enabled;
> +	/** @g2h_outstanding: G2H outstanding info at the capture time */
> +	u32 g2h_outstanding;
> +	/** @g2h: G2H CTB snapshot */
> +	struct guc_ctb_snapshot g2h;
> +	/** @h2g: H2G CTB snapshot */
> +	struct guc_ctb_snapshot h2g;
> +};
> +
>  /**
>   * struct xe_guc_ct - GuC command transport (CT) layer
>   *
> -- 
> 2.39.2
> 

[PATCH 06/14] drm/xe: Add GuC CT snapshot to xe_devcoredump.

[Rodrigo Vivi]

Let's start to move our existent logs to devcoredump one by
one. Any format change should come on follow-up work.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 14 ++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  4 ++++
 2 files changed, 18 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index a08929c01b75..795581c58d90 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -11,6 +11,7 @@
 
 #include "xe_engine.h"
 #include "xe_gt.h"
+#include "xe_guc_ct.h"
 
 /**
  * DOC: Xe device coredump
@@ -47,6 +48,11 @@ static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
 	return container_of(coredump, struct xe_device, devcoredump);
 }
 
+static struct xe_guc *engine_to_guc(struct xe_engine *e)
+{
+	return &e->gt->uc.guc;
+}
+
 static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 				   size_t count, void *data, size_t datalen)
 {
@@ -81,6 +87,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
 	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
 
+	drm_printf(&p, "\n**** GuC CT ****\n");
+	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
+
 	mutex_unlock(&coredump->lock);
 
 	return count - iter.remain;
@@ -96,6 +105,8 @@ static void xe_devcoredump_free(void *data)
 
 	mutex_lock(&coredump->lock);
 
+	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
+
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
 		 "Xe device coredump has been deleted.\n");
@@ -106,10 +117,13 @@ static void xe_devcoredump_free(void *data)
 static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 {
 	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
+	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
 
 	lockdep_assert_held(&coredump->lock);
 	ss->snapshot_time = ktime_get_real();
 	ss->boot_time = ktime_get_boottime();
+
+	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
 }
 
 /**
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index 3f395fa9104e..1e44d5346364 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -23,6 +23,10 @@ struct xe_devcoredump_snapshot {
 	ktime_t snapshot_time;
 	/** @boot_time:  Relative boot time so the uptime can be calculated. */
 	ktime_t boot_time;
+
+	/* GuC snapshots */
+	/** @ct_snapshot: GuC CT snapshot */
+	struct xe_guc_ct_snapshot *ct;
 };
 
 /**
-- 
2.39.2

[Intel-xe] [PATCH 06/14] drm/xe: Add GuC CT snapshot to xe_devcoredump.

[Rodrigo Vivi]

Let's start to move our existent logs to devcoredump one by
one. Any format change should come on follow-up work.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 14 ++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  4 ++++
 2 files changed, 18 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index a08929c01b75..795581c58d90 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -11,6 +11,7 @@
 
 #include "xe_engine.h"
 #include "xe_gt.h"
+#include "xe_guc_ct.h"
 
 /**
  * DOC: Xe device coredump
@@ -47,6 +48,11 @@ static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
 	return container_of(coredump, struct xe_device, devcoredump);
 }
 
+static struct xe_guc *engine_to_guc(struct xe_engine *e)
+{
+	return &e->gt->uc.guc;
+}
+
 static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 				   size_t count, void *data, size_t datalen)
 {
@@ -81,6 +87,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
 	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
 
+	drm_printf(&p, "\n**** GuC CT ****\n");
+	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
+
 	mutex_unlock(&coredump->lock);
 
 	return count - iter.remain;
@@ -96,6 +105,8 @@ static void xe_devcoredump_free(void *data)
 
 	mutex_lock(&coredump->lock);
 
+	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
+
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
 		 "Xe device coredump has been deleted.\n");
@@ -106,10 +117,13 @@ static void xe_devcoredump_free(void *data)
 static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 {
 	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
+	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
 
 	lockdep_assert_held(&coredump->lock);
 	ss->snapshot_time = ktime_get_real();
 	ss->boot_time = ktime_get_boottime();
+
+	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
 }
 
 /**
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index 3f395fa9104e..1e44d5346364 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -23,6 +23,10 @@ struct xe_devcoredump_snapshot {
 	ktime_t snapshot_time;
 	/** @boot_time:  Relative boot time so the uptime can be calculated. */
 	ktime_t boot_time;
+
+	/* GuC snapshots */
+	/** @ct_snapshot: GuC CT snapshot */
+	struct xe_guc_ct_snapshot *ct;
 };
 
 /**
-- 
2.39.2

Re: [PATCH 06/14] drm/xe: Add GuC CT snapshot to xe_devcoredump.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:05PM -0400, Rodrigo Vivi wrote:
> Let's start to move our existent logs to devcoredump one by
> one. Any format change should come on follow-up work.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c       | 14 ++++++++++++++
>  drivers/gpu/drm/xe/xe_devcoredump_types.h |  4 ++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index a08929c01b75..795581c58d90 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -11,6 +11,7 @@
>  
>  #include "xe_engine.h"
>  #include "xe_gt.h"
> +#include "xe_guc_ct.h"
>  
>  /**
>   * DOC: Xe device coredump
> @@ -47,6 +48,11 @@ static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
>  	return container_of(coredump, struct xe_device, devcoredump);
>  }
>  
> +static struct xe_guc *engine_to_guc(struct xe_engine *e)
> +{
> +	return &e->gt->uc.guc;
> +}
> +
>  static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  				   size_t count, void *data, size_t datalen)
>  {
> @@ -81,6 +87,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
>  	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
>  
> +	drm_printf(&p, "\n**** GuC CT ****\n");
> +	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
> +
>  	mutex_unlock(&coredump->lock);
>  
>  	return count - iter.remain;
> @@ -96,6 +105,8 @@ static void xe_devcoredump_free(void *data)
>  
>  	mutex_lock(&coredump->lock);
>  
> +	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
> +
>  	coredump->faulty_engine = NULL;
>  	drm_info(&coredump_to_xe(coredump)->drm,
>  		 "Xe device coredump has been deleted.\n");
> @@ -106,10 +117,13 @@ static void xe_devcoredump_free(void *data)
>  static void devcoredump_snapshot(struct xe_devcoredump *coredump)
>  {
>  	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> +	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
>  
>  	lockdep_assert_held(&coredump->lock);
>  	ss->snapshot_time = ktime_get_real();
>  	ss->boot_time = ktime_get_boottime();
> +
> +	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
>  }
>  
>  /**
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> index 3f395fa9104e..1e44d5346364 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
> +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> @@ -23,6 +23,10 @@ struct xe_devcoredump_snapshot {
>  	ktime_t snapshot_time;
>  	/** @boot_time:  Relative boot time so the uptime can be calculated. */
>  	ktime_t boot_time;
> +
> +	/* GuC snapshots */
> +	/** @ct_snapshot: GuC CT snapshot */
> +	struct xe_guc_ct_snapshot *ct;
>  };
>  
>  /**
> -- 
> 2.39.2
> 

Re: [Intel-xe] [PATCH 06/14] drm/xe: Add GuC CT snapshot to xe_devcoredump.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:05PM -0400, Rodrigo Vivi wrote:
> Let's start to move our existent logs to devcoredump one by
> one. Any format change should come on follow-up work.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c       | 14 ++++++++++++++
>  drivers/gpu/drm/xe/xe_devcoredump_types.h |  4 ++++
>  2 files changed, 18 insertions(+)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index a08929c01b75..795581c58d90 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -11,6 +11,7 @@
>  
>  #include "xe_engine.h"
>  #include "xe_gt.h"
> +#include "xe_guc_ct.h"
>  
>  /**
>   * DOC: Xe device coredump
> @@ -47,6 +48,11 @@ static struct xe_device *coredump_to_xe(const struct xe_devcoredump *coredump)
>  	return container_of(coredump, struct xe_device, devcoredump);
>  }
>  
> +static struct xe_guc *engine_to_guc(struct xe_engine *e)
> +{
> +	return &e->gt->uc.guc;
> +}
> +
>  static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  				   size_t count, void *data, size_t datalen)
>  {
> @@ -81,6 +87,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	ts = ktime_to_timespec64(ktime_sub(ss->snapshot_time, ss->boot_time));
>  	drm_printf(&p, "Uptime: %lld.%09ld\n", ts.tv_sec, ts.tv_nsec);
>  
> +	drm_printf(&p, "\n**** GuC CT ****\n");
> +	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
> +
>  	mutex_unlock(&coredump->lock);
>  
>  	return count - iter.remain;
> @@ -96,6 +105,8 @@ static void xe_devcoredump_free(void *data)
>  
>  	mutex_lock(&coredump->lock);
>  
> +	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
> +
>  	coredump->faulty_engine = NULL;
>  	drm_info(&coredump_to_xe(coredump)->drm,
>  		 "Xe device coredump has been deleted.\n");
> @@ -106,10 +117,13 @@ static void xe_devcoredump_free(void *data)
>  static void devcoredump_snapshot(struct xe_devcoredump *coredump)
>  {
>  	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> +	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
>  
>  	lockdep_assert_held(&coredump->lock);
>  	ss->snapshot_time = ktime_get_real();
>  	ss->boot_time = ktime_get_boottime();
> +
> +	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
>  }
>  
>  /**
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> index 3f395fa9104e..1e44d5346364 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
> +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> @@ -23,6 +23,10 @@ struct xe_devcoredump_snapshot {
>  	ktime_t snapshot_time;
>  	/** @boot_time:  Relative boot time so the uptime can be calculated. */
>  	ktime_t boot_time;
> +
> +	/* GuC snapshots */
> +	/** @ct_snapshot: GuC CT snapshot */
> +	struct xe_guc_ct_snapshot *ct;
>  };
>  
>  /**
> -- 
> 2.39.2
> 

[PATCH 07/14] drm/xe: Introduce guc_submit_types.h with relevant structs.

[Rodrigo Vivi]

These structs and definitions are only used for the guc_submit
and they were added specifically for the parallel submission.

While doing that also delete the unused struct guc_wq_item.

Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_fwif.h         | 29 -----------
 drivers/gpu/drm/xe/xe_guc_submit.c       | 40 ++++-----------
 drivers/gpu/drm/xe/xe_guc_submit_types.h | 64 ++++++++++++++++++++++++
 3 files changed, 75 insertions(+), 58 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_guc_submit_types.h

diff --git a/drivers/gpu/drm/xe/xe_guc_fwif.h b/drivers/gpu/drm/xe/xe_guc_fwif.h
index 20155ba4ef07..27d132ce2087 100644
--- a/drivers/gpu/drm/xe/xe_guc_fwif.h
+++ b/drivers/gpu/drm/xe/xe_guc_fwif.h
@@ -46,35 +46,6 @@
 #define GUC_MAX_ENGINE_CLASSES		16
 #define GUC_MAX_INSTANCES_PER_CLASS	32
 
-/* Work item for submitting workloads into work queue of GuC. */
-#define WQ_STATUS_ACTIVE		1
-#define WQ_STATUS_SUSPENDED		2
-#define WQ_STATUS_CMD_ERROR		3
-#define WQ_STATUS_ENGINE_ID_NOT_USED	4
-#define WQ_STATUS_SUSPENDED_FROM_RESET	5
-#define WQ_TYPE_NOOP			0x4
-#define WQ_TYPE_MULTI_LRC		0x5
-#define WQ_TYPE_MASK			GENMASK(7, 0)
-#define WQ_LEN_MASK			GENMASK(26, 16)
-
-#define WQ_GUC_ID_MASK			GENMASK(15, 0)
-#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
-
-struct guc_wq_item {
-	u32 header;
-	u32 context_desc;
-	u32 submit_element_info;
-	u32 fence_id;
-} __packed;
-
-struct guc_sched_wq_desc {
-	u32 head;
-	u32 tail;
-	u32 error_offset;
-	u32 wq_status;
-	u32 reserved[28];
-} __packed;
-
 /* Helper for context registration H2G */
 struct guc_ctxt_registration_info {
 	u32 flags;
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index 231fb4145297..a5fe7755ce4c 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -22,6 +22,7 @@
 #include "xe_guc.h"
 #include "xe_guc_ct.h"
 #include "xe_guc_engine_types.h"
+#include "xe_guc_submit_types.h"
 #include "xe_hw_engine.h"
 #include "xe_hw_fence.h"
 #include "xe_lrc.h"
@@ -378,32 +379,12 @@ static void set_min_preemption_timeout(struct xe_guc *guc, struct xe_engine *e)
 		       __guc_engine_policy_action_size(&policy), 0, 0);
 }
 
-#define PARALLEL_SCRATCH_SIZE	2048
-#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
-#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
-#define CACHELINE_BYTES		64
-
-struct sync_semaphore {
-	u32 semaphore;
-	u8 unused[CACHELINE_BYTES - sizeof(u32)];
-};
-
-struct parallel_scratch {
-	struct guc_sched_wq_desc wq_desc;
-
-	struct sync_semaphore go;
-	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
-
-	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
-		sizeof(struct sync_semaphore) * (XE_HW_ENGINE_MAX_INSTANCE + 1)];
-
-	u32 wq[WQ_SIZE / sizeof(u32)];
-};
-
 #define parallel_read(xe_, map_, field_) \
-	xe_map_rd_field(xe_, &map_, 0, struct parallel_scratch, field_)
+	xe_map_rd_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
+			field_)
 #define parallel_write(xe_, map_, field_, val_) \
-	xe_map_wr_field(xe_, &map_, 0, struct parallel_scratch, field_, val_)
+	xe_map_wr_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
+			field_, val_)
 
 static void __register_mlrc_engine(struct xe_guc *guc,
 				   struct xe_engine *e,
@@ -486,13 +467,13 @@ static void register_engine(struct xe_engine *e)
 		struct iosys_map map = xe_lrc_parallel_map(lrc);
 
 		info.wq_desc_lo = lower_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq_desc));
+			offsetof(struct guc_submit_parallel_scratch, wq_desc));
 		info.wq_desc_hi = upper_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq_desc));
+			offsetof(struct guc_submit_parallel_scratch, wq_desc));
 		info.wq_base_lo = lower_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq[0]));
+			offsetof(struct guc_submit_parallel_scratch, wq[0]));
 		info.wq_base_hi = upper_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq[0]));
+			offsetof(struct guc_submit_parallel_scratch, wq[0]));
 		info.wq_size = WQ_SIZE;
 
 		e->guc->wqi_head = 0;
@@ -594,7 +575,7 @@ static void wq_item_append(struct xe_engine *e)
 
 	XE_BUG_ON(i != wqi_size / sizeof(u32));
 
-	iosys_map_incr(&map, offsetof(struct parallel_scratch,
+	iosys_map_incr(&map, offsetof(struct guc_submit_parallel_scratch,
 					wq[e->guc->wqi_tail / sizeof(u32)]));
 	xe_map_memcpy_to(xe, &map, 0, wqi, wqi_size);
 	e->guc->wqi_tail += wqi_size;
@@ -1674,6 +1655,7 @@ static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
 		guc_engine_wq_print(e, p);
 
 	spin_lock(&sched->job_list_lock);
+
 	list_for_each_entry(job, &sched->pending_list, drm.list)
 		drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n",
 			   xe_sched_job_seqno(job),
diff --git a/drivers/gpu/drm/xe/xe_guc_submit_types.h b/drivers/gpu/drm/xe/xe_guc_submit_types.h
new file mode 100644
index 000000000000..d369ea0bad60
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_guc_submit_types.h
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_GUC_SUBMIT_TYPES_H_
+#define _XE_GUC_SUBMIT_TYPES_H_
+
+#include "xe_hw_engine_types.h"
+
+/* Work item for submitting workloads into work queue of GuC. */
+#define WQ_STATUS_ACTIVE		1
+#define WQ_STATUS_SUSPENDED		2
+#define WQ_STATUS_CMD_ERROR		3
+#define WQ_STATUS_ENGINE_ID_NOT_USED	4
+#define WQ_STATUS_SUSPENDED_FROM_RESET	5
+#define WQ_TYPE_NOOP			0x4
+#define WQ_TYPE_MULTI_LRC		0x5
+#define WQ_TYPE_MASK			GENMASK(7, 0)
+#define WQ_LEN_MASK			GENMASK(26, 16)
+
+#define WQ_GUC_ID_MASK			GENMASK(15, 0)
+#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
+
+#define PARALLEL_SCRATCH_SIZE	2048
+#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
+#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
+#define CACHELINE_BYTES		64
+
+struct guc_sched_wq_desc {
+	u32 head;
+	u32 tail;
+	u32 error_offset;
+	u32 wq_status;
+	u32 reserved[28];
+} __packed;
+
+struct sync_semaphore {
+	u32 semaphore;
+	u8 unused[CACHELINE_BYTES - sizeof(u32)];
+};
+
+/**
+ * Struct guc_submit_parallel_scratch - A scratch shared mapped buffer.
+ */
+struct guc_submit_parallel_scratch {
+	/** @wq_desc: Guc scheduler workqueue descriptor */
+	struct guc_sched_wq_desc wq_desc;
+
+	/** @go: Go Semaphore */
+	struct sync_semaphore go;
+	/** @join: Joined semaphore for the relevant hw engine instances */
+	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
+
+	/** @unused: Unused/Reserved memory space */
+	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
+		  sizeof(struct sync_semaphore) *
+		  (XE_HW_ENGINE_MAX_INSTANCE + 1)];
+
+	/** @wq: Workqueue info */
+	u32 wq[WQ_SIZE / sizeof(u32)];
+};
+
+#endif
-- 
2.39.2

[Intel-xe] [PATCH 07/14] drm/xe: Introduce guc_submit_types.h with relevant structs.

[Rodrigo Vivi]

These structs and definitions are only used for the guc_submit
and they were added specifically for the parallel submission.

While doing that also delete the unused struct guc_wq_item.

Cc: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_fwif.h         | 29 -----------
 drivers/gpu/drm/xe/xe_guc_submit.c       | 40 ++++-----------
 drivers/gpu/drm/xe/xe_guc_submit_types.h | 64 ++++++++++++++++++++++++
 3 files changed, 75 insertions(+), 58 deletions(-)
 create mode 100644 drivers/gpu/drm/xe/xe_guc_submit_types.h

diff --git a/drivers/gpu/drm/xe/xe_guc_fwif.h b/drivers/gpu/drm/xe/xe_guc_fwif.h
index 20155ba4ef07..27d132ce2087 100644
--- a/drivers/gpu/drm/xe/xe_guc_fwif.h
+++ b/drivers/gpu/drm/xe/xe_guc_fwif.h
@@ -46,35 +46,6 @@
 #define GUC_MAX_ENGINE_CLASSES		16
 #define GUC_MAX_INSTANCES_PER_CLASS	32
 
-/* Work item for submitting workloads into work queue of GuC. */
-#define WQ_STATUS_ACTIVE		1
-#define WQ_STATUS_SUSPENDED		2
-#define WQ_STATUS_CMD_ERROR		3
-#define WQ_STATUS_ENGINE_ID_NOT_USED	4
-#define WQ_STATUS_SUSPENDED_FROM_RESET	5
-#define WQ_TYPE_NOOP			0x4
-#define WQ_TYPE_MULTI_LRC		0x5
-#define WQ_TYPE_MASK			GENMASK(7, 0)
-#define WQ_LEN_MASK			GENMASK(26, 16)
-
-#define WQ_GUC_ID_MASK			GENMASK(15, 0)
-#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
-
-struct guc_wq_item {
-	u32 header;
-	u32 context_desc;
-	u32 submit_element_info;
-	u32 fence_id;
-} __packed;
-
-struct guc_sched_wq_desc {
-	u32 head;
-	u32 tail;
-	u32 error_offset;
-	u32 wq_status;
-	u32 reserved[28];
-} __packed;
-
 /* Helper for context registration H2G */
 struct guc_ctxt_registration_info {
 	u32 flags;
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index 231fb4145297..a5fe7755ce4c 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -22,6 +22,7 @@
 #include "xe_guc.h"
 #include "xe_guc_ct.h"
 #include "xe_guc_engine_types.h"
+#include "xe_guc_submit_types.h"
 #include "xe_hw_engine.h"
 #include "xe_hw_fence.h"
 #include "xe_lrc.h"
@@ -378,32 +379,12 @@ static void set_min_preemption_timeout(struct xe_guc *guc, struct xe_engine *e)
 		       __guc_engine_policy_action_size(&policy), 0, 0);
 }
 
-#define PARALLEL_SCRATCH_SIZE	2048
-#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
-#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
-#define CACHELINE_BYTES		64
-
-struct sync_semaphore {
-	u32 semaphore;
-	u8 unused[CACHELINE_BYTES - sizeof(u32)];
-};
-
-struct parallel_scratch {
-	struct guc_sched_wq_desc wq_desc;
-
-	struct sync_semaphore go;
-	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
-
-	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
-		sizeof(struct sync_semaphore) * (XE_HW_ENGINE_MAX_INSTANCE + 1)];
-
-	u32 wq[WQ_SIZE / sizeof(u32)];
-};
-
 #define parallel_read(xe_, map_, field_) \
-	xe_map_rd_field(xe_, &map_, 0, struct parallel_scratch, field_)
+	xe_map_rd_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
+			field_)
 #define parallel_write(xe_, map_, field_, val_) \
-	xe_map_wr_field(xe_, &map_, 0, struct parallel_scratch, field_, val_)
+	xe_map_wr_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
+			field_, val_)
 
 static void __register_mlrc_engine(struct xe_guc *guc,
 				   struct xe_engine *e,
@@ -486,13 +467,13 @@ static void register_engine(struct xe_engine *e)
 		struct iosys_map map = xe_lrc_parallel_map(lrc);
 
 		info.wq_desc_lo = lower_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq_desc));
+			offsetof(struct guc_submit_parallel_scratch, wq_desc));
 		info.wq_desc_hi = upper_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq_desc));
+			offsetof(struct guc_submit_parallel_scratch, wq_desc));
 		info.wq_base_lo = lower_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq[0]));
+			offsetof(struct guc_submit_parallel_scratch, wq[0]));
 		info.wq_base_hi = upper_32_bits(ggtt_addr +
-			offsetof(struct parallel_scratch, wq[0]));
+			offsetof(struct guc_submit_parallel_scratch, wq[0]));
 		info.wq_size = WQ_SIZE;
 
 		e->guc->wqi_head = 0;
@@ -594,7 +575,7 @@ static void wq_item_append(struct xe_engine *e)
 
 	XE_BUG_ON(i != wqi_size / sizeof(u32));
 
-	iosys_map_incr(&map, offsetof(struct parallel_scratch,
+	iosys_map_incr(&map, offsetof(struct guc_submit_parallel_scratch,
 					wq[e->guc->wqi_tail / sizeof(u32)]));
 	xe_map_memcpy_to(xe, &map, 0, wqi, wqi_size);
 	e->guc->wqi_tail += wqi_size;
@@ -1674,6 +1655,7 @@ static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
 		guc_engine_wq_print(e, p);
 
 	spin_lock(&sched->job_list_lock);
+
 	list_for_each_entry(job, &sched->pending_list, drm.list)
 		drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n",
 			   xe_sched_job_seqno(job),
diff --git a/drivers/gpu/drm/xe/xe_guc_submit_types.h b/drivers/gpu/drm/xe/xe_guc_submit_types.h
new file mode 100644
index 000000000000..d369ea0bad60
--- /dev/null
+++ b/drivers/gpu/drm/xe/xe_guc_submit_types.h
@@ -0,0 +1,64 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_GUC_SUBMIT_TYPES_H_
+#define _XE_GUC_SUBMIT_TYPES_H_
+
+#include "xe_hw_engine_types.h"
+
+/* Work item for submitting workloads into work queue of GuC. */
+#define WQ_STATUS_ACTIVE		1
+#define WQ_STATUS_SUSPENDED		2
+#define WQ_STATUS_CMD_ERROR		3
+#define WQ_STATUS_ENGINE_ID_NOT_USED	4
+#define WQ_STATUS_SUSPENDED_FROM_RESET	5
+#define WQ_TYPE_NOOP			0x4
+#define WQ_TYPE_MULTI_LRC		0x5
+#define WQ_TYPE_MASK			GENMASK(7, 0)
+#define WQ_LEN_MASK			GENMASK(26, 16)
+
+#define WQ_GUC_ID_MASK			GENMASK(15, 0)
+#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
+
+#define PARALLEL_SCRATCH_SIZE	2048
+#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
+#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
+#define CACHELINE_BYTES		64
+
+struct guc_sched_wq_desc {
+	u32 head;
+	u32 tail;
+	u32 error_offset;
+	u32 wq_status;
+	u32 reserved[28];
+} __packed;
+
+struct sync_semaphore {
+	u32 semaphore;
+	u8 unused[CACHELINE_BYTES - sizeof(u32)];
+};
+
+/**
+ * Struct guc_submit_parallel_scratch - A scratch shared mapped buffer.
+ */
+struct guc_submit_parallel_scratch {
+	/** @wq_desc: Guc scheduler workqueue descriptor */
+	struct guc_sched_wq_desc wq_desc;
+
+	/** @go: Go Semaphore */
+	struct sync_semaphore go;
+	/** @join: Joined semaphore for the relevant hw engine instances */
+	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
+
+	/** @unused: Unused/Reserved memory space */
+	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
+		  sizeof(struct sync_semaphore) *
+		  (XE_HW_ENGINE_MAX_INSTANCE + 1)];
+
+	/** @wq: Workqueue info */
+	u32 wq[WQ_SIZE / sizeof(u32)];
+};
+
+#endif
-- 
2.39.2

Re: [PATCH 07/14] drm/xe: Introduce guc_submit_types.h with relevant structs.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:06PM -0400, Rodrigo Vivi wrote:
> These structs and definitions are only used for the guc_submit
> and they were added specifically for the parallel submission.
> 
> While doing that also delete the unused struct guc_wq_item.
> 
> Cc: Matthew Brost <matthew.brost@intel.com>
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

Side note this reminds me we need to implement to parallel submission
handsake in xe_ring_ops.c if we want parallel submission to truly work.
Should be a straight port from the i915... Also should write an IGT for
that too. 

> ---
>  drivers/gpu/drm/xe/xe_guc_fwif.h         | 29 -----------
>  drivers/gpu/drm/xe/xe_guc_submit.c       | 40 ++++-----------
>  drivers/gpu/drm/xe/xe_guc_submit_types.h | 64 ++++++++++++++++++++++++
>  3 files changed, 75 insertions(+), 58 deletions(-)
>  create mode 100644 drivers/gpu/drm/xe/xe_guc_submit_types.h
> 
> diff --git a/drivers/gpu/drm/xe/xe_guc_fwif.h b/drivers/gpu/drm/xe/xe_guc_fwif.h
> index 20155ba4ef07..27d132ce2087 100644
> --- a/drivers/gpu/drm/xe/xe_guc_fwif.h
> +++ b/drivers/gpu/drm/xe/xe_guc_fwif.h
> @@ -46,35 +46,6 @@
>  #define GUC_MAX_ENGINE_CLASSES		16
>  #define GUC_MAX_INSTANCES_PER_CLASS	32
>  
> -/* Work item for submitting workloads into work queue of GuC. */
> -#define WQ_STATUS_ACTIVE		1
> -#define WQ_STATUS_SUSPENDED		2
> -#define WQ_STATUS_CMD_ERROR		3
> -#define WQ_STATUS_ENGINE_ID_NOT_USED	4
> -#define WQ_STATUS_SUSPENDED_FROM_RESET	5
> -#define WQ_TYPE_NOOP			0x4
> -#define WQ_TYPE_MULTI_LRC		0x5
> -#define WQ_TYPE_MASK			GENMASK(7, 0)
> -#define WQ_LEN_MASK			GENMASK(26, 16)
> -
> -#define WQ_GUC_ID_MASK			GENMASK(15, 0)
> -#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
> -
> -struct guc_wq_item {
> -	u32 header;
> -	u32 context_desc;
> -	u32 submit_element_info;
> -	u32 fence_id;
> -} __packed;
> -
> -struct guc_sched_wq_desc {
> -	u32 head;
> -	u32 tail;
> -	u32 error_offset;
> -	u32 wq_status;
> -	u32 reserved[28];
> -} __packed;
> -
>  /* Helper for context registration H2G */
>  struct guc_ctxt_registration_info {
>  	u32 flags;
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> index 231fb4145297..a5fe7755ce4c 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> @@ -22,6 +22,7 @@
>  #include "xe_guc.h"
>  #include "xe_guc_ct.h"
>  #include "xe_guc_engine_types.h"
> +#include "xe_guc_submit_types.h"
>  #include "xe_hw_engine.h"
>  #include "xe_hw_fence.h"
>  #include "xe_lrc.h"
> @@ -378,32 +379,12 @@ static void set_min_preemption_timeout(struct xe_guc *guc, struct xe_engine *e)
>  		       __guc_engine_policy_action_size(&policy), 0, 0);
>  }
>  
> -#define PARALLEL_SCRATCH_SIZE	2048
> -#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
> -#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
> -#define CACHELINE_BYTES		64
> -
> -struct sync_semaphore {
> -	u32 semaphore;
> -	u8 unused[CACHELINE_BYTES - sizeof(u32)];
> -};
> -
> -struct parallel_scratch {
> -	struct guc_sched_wq_desc wq_desc;
> -
> -	struct sync_semaphore go;
> -	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
> -
> -	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
> -		sizeof(struct sync_semaphore) * (XE_HW_ENGINE_MAX_INSTANCE + 1)];
> -
> -	u32 wq[WQ_SIZE / sizeof(u32)];
> -};
> -
>  #define parallel_read(xe_, map_, field_) \
> -	xe_map_rd_field(xe_, &map_, 0, struct parallel_scratch, field_)
> +	xe_map_rd_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
> +			field_)
>  #define parallel_write(xe_, map_, field_, val_) \
> -	xe_map_wr_field(xe_, &map_, 0, struct parallel_scratch, field_, val_)
> +	xe_map_wr_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
> +			field_, val_)
>  
>  static void __register_mlrc_engine(struct xe_guc *guc,
>  				   struct xe_engine *e,
> @@ -486,13 +467,13 @@ static void register_engine(struct xe_engine *e)
>  		struct iosys_map map = xe_lrc_parallel_map(lrc);
>  
>  		info.wq_desc_lo = lower_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq_desc));
> +			offsetof(struct guc_submit_parallel_scratch, wq_desc));
>  		info.wq_desc_hi = upper_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq_desc));
> +			offsetof(struct guc_submit_parallel_scratch, wq_desc));
>  		info.wq_base_lo = lower_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq[0]));
> +			offsetof(struct guc_submit_parallel_scratch, wq[0]));
>  		info.wq_base_hi = upper_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq[0]));
> +			offsetof(struct guc_submit_parallel_scratch, wq[0]));
>  		info.wq_size = WQ_SIZE;
>  
>  		e->guc->wqi_head = 0;
> @@ -594,7 +575,7 @@ static void wq_item_append(struct xe_engine *e)
>  
>  	XE_BUG_ON(i != wqi_size / sizeof(u32));
>  
> -	iosys_map_incr(&map, offsetof(struct parallel_scratch,
> +	iosys_map_incr(&map, offsetof(struct guc_submit_parallel_scratch,
>  					wq[e->guc->wqi_tail / sizeof(u32)]));
>  	xe_map_memcpy_to(xe, &map, 0, wqi, wqi_size);
>  	e->guc->wqi_tail += wqi_size;
> @@ -1674,6 +1655,7 @@ static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
>  		guc_engine_wq_print(e, p);
>  
>  	spin_lock(&sched->job_list_lock);
> +
>  	list_for_each_entry(job, &sched->pending_list, drm.list)
>  		drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n",
>  			   xe_sched_job_seqno(job),
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit_types.h b/drivers/gpu/drm/xe/xe_guc_submit_types.h
> new file mode 100644
> index 000000000000..d369ea0bad60
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_guc_submit_types.h
> @@ -0,0 +1,64 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2023 Intel Corporation
> + */
> +
> +#ifndef _XE_GUC_SUBMIT_TYPES_H_
> +#define _XE_GUC_SUBMIT_TYPES_H_
> +
> +#include "xe_hw_engine_types.h"
> +
> +/* Work item for submitting workloads into work queue of GuC. */
> +#define WQ_STATUS_ACTIVE		1
> +#define WQ_STATUS_SUSPENDED		2
> +#define WQ_STATUS_CMD_ERROR		3
> +#define WQ_STATUS_ENGINE_ID_NOT_USED	4
> +#define WQ_STATUS_SUSPENDED_FROM_RESET	5
> +#define WQ_TYPE_NOOP			0x4
> +#define WQ_TYPE_MULTI_LRC		0x5
> +#define WQ_TYPE_MASK			GENMASK(7, 0)
> +#define WQ_LEN_MASK			GENMASK(26, 16)
> +
> +#define WQ_GUC_ID_MASK			GENMASK(15, 0)
> +#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
> +
> +#define PARALLEL_SCRATCH_SIZE	2048
> +#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
> +#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
> +#define CACHELINE_BYTES		64
> +
> +struct guc_sched_wq_desc {
> +	u32 head;
> +	u32 tail;
> +	u32 error_offset;
> +	u32 wq_status;
> +	u32 reserved[28];
> +} __packed;
> +
> +struct sync_semaphore {
> +	u32 semaphore;
> +	u8 unused[CACHELINE_BYTES - sizeof(u32)];
> +};
> +
> +/**
> + * Struct guc_submit_parallel_scratch - A scratch shared mapped buffer.
> + */
> +struct guc_submit_parallel_scratch {
> +	/** @wq_desc: Guc scheduler workqueue descriptor */
> +	struct guc_sched_wq_desc wq_desc;
> +
> +	/** @go: Go Semaphore */
> +	struct sync_semaphore go;
> +	/** @join: Joined semaphore for the relevant hw engine instances */
> +	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
> +
> +	/** @unused: Unused/Reserved memory space */
> +	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
> +		  sizeof(struct sync_semaphore) *
> +		  (XE_HW_ENGINE_MAX_INSTANCE + 1)];
> +
> +	/** @wq: Workqueue info */
> +	u32 wq[WQ_SIZE / sizeof(u32)];
> +};
> +
> +#endif
> -- 
> 2.39.2
> 

Re: [Intel-xe] [PATCH 07/14] drm/xe: Introduce guc_submit_types.h with relevant structs.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:06PM -0400, Rodrigo Vivi wrote:
> These structs and definitions are only used for the guc_submit
> and they were added specifically for the parallel submission.
> 
> While doing that also delete the unused struct guc_wq_item.
> 
> Cc: Matthew Brost <matthew.brost@intel.com>
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

Side note this reminds me we need to implement to parallel submission
handsake in xe_ring_ops.c if we want parallel submission to truly work.
Should be a straight port from the i915... Also should write an IGT for
that too. 

> ---
>  drivers/gpu/drm/xe/xe_guc_fwif.h         | 29 -----------
>  drivers/gpu/drm/xe/xe_guc_submit.c       | 40 ++++-----------
>  drivers/gpu/drm/xe/xe_guc_submit_types.h | 64 ++++++++++++++++++++++++
>  3 files changed, 75 insertions(+), 58 deletions(-)
>  create mode 100644 drivers/gpu/drm/xe/xe_guc_submit_types.h
> 
> diff --git a/drivers/gpu/drm/xe/xe_guc_fwif.h b/drivers/gpu/drm/xe/xe_guc_fwif.h
> index 20155ba4ef07..27d132ce2087 100644
> --- a/drivers/gpu/drm/xe/xe_guc_fwif.h
> +++ b/drivers/gpu/drm/xe/xe_guc_fwif.h
> @@ -46,35 +46,6 @@
>  #define GUC_MAX_ENGINE_CLASSES		16
>  #define GUC_MAX_INSTANCES_PER_CLASS	32
>  
> -/* Work item for submitting workloads into work queue of GuC. */
> -#define WQ_STATUS_ACTIVE		1
> -#define WQ_STATUS_SUSPENDED		2
> -#define WQ_STATUS_CMD_ERROR		3
> -#define WQ_STATUS_ENGINE_ID_NOT_USED	4
> -#define WQ_STATUS_SUSPENDED_FROM_RESET	5
> -#define WQ_TYPE_NOOP			0x4
> -#define WQ_TYPE_MULTI_LRC		0x5
> -#define WQ_TYPE_MASK			GENMASK(7, 0)
> -#define WQ_LEN_MASK			GENMASK(26, 16)
> -
> -#define WQ_GUC_ID_MASK			GENMASK(15, 0)
> -#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
> -
> -struct guc_wq_item {
> -	u32 header;
> -	u32 context_desc;
> -	u32 submit_element_info;
> -	u32 fence_id;
> -} __packed;
> -
> -struct guc_sched_wq_desc {
> -	u32 head;
> -	u32 tail;
> -	u32 error_offset;
> -	u32 wq_status;
> -	u32 reserved[28];
> -} __packed;
> -
>  /* Helper for context registration H2G */
>  struct guc_ctxt_registration_info {
>  	u32 flags;
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> index 231fb4145297..a5fe7755ce4c 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> @@ -22,6 +22,7 @@
>  #include "xe_guc.h"
>  #include "xe_guc_ct.h"
>  #include "xe_guc_engine_types.h"
> +#include "xe_guc_submit_types.h"
>  #include "xe_hw_engine.h"
>  #include "xe_hw_fence.h"
>  #include "xe_lrc.h"
> @@ -378,32 +379,12 @@ static void set_min_preemption_timeout(struct xe_guc *guc, struct xe_engine *e)
>  		       __guc_engine_policy_action_size(&policy), 0, 0);
>  }
>  
> -#define PARALLEL_SCRATCH_SIZE	2048
> -#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
> -#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
> -#define CACHELINE_BYTES		64
> -
> -struct sync_semaphore {
> -	u32 semaphore;
> -	u8 unused[CACHELINE_BYTES - sizeof(u32)];
> -};
> -
> -struct parallel_scratch {
> -	struct guc_sched_wq_desc wq_desc;
> -
> -	struct sync_semaphore go;
> -	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
> -
> -	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
> -		sizeof(struct sync_semaphore) * (XE_HW_ENGINE_MAX_INSTANCE + 1)];
> -
> -	u32 wq[WQ_SIZE / sizeof(u32)];
> -};
> -
>  #define parallel_read(xe_, map_, field_) \
> -	xe_map_rd_field(xe_, &map_, 0, struct parallel_scratch, field_)
> +	xe_map_rd_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
> +			field_)
>  #define parallel_write(xe_, map_, field_, val_) \
> -	xe_map_wr_field(xe_, &map_, 0, struct parallel_scratch, field_, val_)
> +	xe_map_wr_field(xe_, &map_, 0, struct guc_submit_parallel_scratch, \
> +			field_, val_)
>  
>  static void __register_mlrc_engine(struct xe_guc *guc,
>  				   struct xe_engine *e,
> @@ -486,13 +467,13 @@ static void register_engine(struct xe_engine *e)
>  		struct iosys_map map = xe_lrc_parallel_map(lrc);
>  
>  		info.wq_desc_lo = lower_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq_desc));
> +			offsetof(struct guc_submit_parallel_scratch, wq_desc));
>  		info.wq_desc_hi = upper_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq_desc));
> +			offsetof(struct guc_submit_parallel_scratch, wq_desc));
>  		info.wq_base_lo = lower_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq[0]));
> +			offsetof(struct guc_submit_parallel_scratch, wq[0]));
>  		info.wq_base_hi = upper_32_bits(ggtt_addr +
> -			offsetof(struct parallel_scratch, wq[0]));
> +			offsetof(struct guc_submit_parallel_scratch, wq[0]));
>  		info.wq_size = WQ_SIZE;
>  
>  		e->guc->wqi_head = 0;
> @@ -594,7 +575,7 @@ static void wq_item_append(struct xe_engine *e)
>  
>  	XE_BUG_ON(i != wqi_size / sizeof(u32));
>  
> -	iosys_map_incr(&map, offsetof(struct parallel_scratch,
> +	iosys_map_incr(&map, offsetof(struct guc_submit_parallel_scratch,
>  					wq[e->guc->wqi_tail / sizeof(u32)]));
>  	xe_map_memcpy_to(xe, &map, 0, wqi, wqi_size);
>  	e->guc->wqi_tail += wqi_size;
> @@ -1674,6 +1655,7 @@ static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
>  		guc_engine_wq_print(e, p);
>  
>  	spin_lock(&sched->job_list_lock);
> +
>  	list_for_each_entry(job, &sched->pending_list, drm.list)
>  		drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n",
>  			   xe_sched_job_seqno(job),
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit_types.h b/drivers/gpu/drm/xe/xe_guc_submit_types.h
> new file mode 100644
> index 000000000000..d369ea0bad60
> --- /dev/null
> +++ b/drivers/gpu/drm/xe/xe_guc_submit_types.h
> @@ -0,0 +1,64 @@
> +// SPDX-License-Identifier: MIT
> +/*
> + * Copyright © 2023 Intel Corporation
> + */
> +
> +#ifndef _XE_GUC_SUBMIT_TYPES_H_
> +#define _XE_GUC_SUBMIT_TYPES_H_
> +
> +#include "xe_hw_engine_types.h"
> +
> +/* Work item for submitting workloads into work queue of GuC. */
> +#define WQ_STATUS_ACTIVE		1
> +#define WQ_STATUS_SUSPENDED		2
> +#define WQ_STATUS_CMD_ERROR		3
> +#define WQ_STATUS_ENGINE_ID_NOT_USED	4
> +#define WQ_STATUS_SUSPENDED_FROM_RESET	5
> +#define WQ_TYPE_NOOP			0x4
> +#define WQ_TYPE_MULTI_LRC		0x5
> +#define WQ_TYPE_MASK			GENMASK(7, 0)
> +#define WQ_LEN_MASK			GENMASK(26, 16)
> +
> +#define WQ_GUC_ID_MASK			GENMASK(15, 0)
> +#define WQ_RING_TAIL_MASK		GENMASK(28, 18)
> +
> +#define PARALLEL_SCRATCH_SIZE	2048
> +#define WQ_SIZE			(PARALLEL_SCRATCH_SIZE / 2)
> +#define WQ_OFFSET		(PARALLEL_SCRATCH_SIZE - WQ_SIZE)
> +#define CACHELINE_BYTES		64
> +
> +struct guc_sched_wq_desc {
> +	u32 head;
> +	u32 tail;
> +	u32 error_offset;
> +	u32 wq_status;
> +	u32 reserved[28];
> +} __packed;
> +
> +struct sync_semaphore {
> +	u32 semaphore;
> +	u8 unused[CACHELINE_BYTES - sizeof(u32)];
> +};
> +
> +/**
> + * Struct guc_submit_parallel_scratch - A scratch shared mapped buffer.
> + */
> +struct guc_submit_parallel_scratch {
> +	/** @wq_desc: Guc scheduler workqueue descriptor */
> +	struct guc_sched_wq_desc wq_desc;
> +
> +	/** @go: Go Semaphore */
> +	struct sync_semaphore go;
> +	/** @join: Joined semaphore for the relevant hw engine instances */
> +	struct sync_semaphore join[XE_HW_ENGINE_MAX_INSTANCE];
> +
> +	/** @unused: Unused/Reserved memory space */
> +	u8 unused[WQ_OFFSET - sizeof(struct guc_sched_wq_desc) -
> +		  sizeof(struct sync_semaphore) *
> +		  (XE_HW_ENGINE_MAX_INSTANCE + 1)];
> +
> +	/** @wq: Workqueue info */
> +	u32 wq[WQ_SIZE / sizeof(u32)];
> +};
> +
> +#endif
> -- 
> 2.39.2
> 

[Intel-xe] [PATCH 08/14] drm/xe: Convert GuC Engine print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_submit.c       | 212 +++++++++++++++++++----
 drivers/gpu/drm/xe/xe_guc_submit.h       |  10 +-
 drivers/gpu/drm/xe/xe_guc_submit_types.h |  91 ++++++++++
 3 files changed, 274 insertions(+), 39 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index a5fe7755ce4c..9c06411f857f 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -1596,75 +1596,211 @@ int xe_guc_engine_reset_failure_handler(struct xe_guc *guc, u32 *msg, u32 len)
 	return 0;
 }
 
-static void guc_engine_wq_print(struct xe_engine *e, struct drm_printer *p)
+static void
+guc_engine_wq_snapshot_capture(struct xe_engine *e,
+			       struct xe_guc_submit_engine_snapshot *snapshot)
 {
 	struct xe_guc *guc = engine_to_guc(e);
 	struct xe_device *xe = guc_to_xe(guc);
 	struct iosys_map map = xe_lrc_parallel_map(e->lrc);
 	int i;
 
+	snapshot->guc.wqi_head = e->guc->wqi_head;
+	snapshot->guc.wqi_tail = e->guc->wqi_tail;
+	snapshot->parallel.wq_desc.head = parallel_read(xe, map, wq_desc.head);
+	snapshot->parallel.wq_desc.tail = parallel_read(xe, map, wq_desc.tail);
+	snapshot->parallel.wq_desc.status = parallel_read(xe, map,
+							  wq_desc.wq_status);
+
+	if (snapshot->parallel.wq_desc.head !=
+	    snapshot->parallel.wq_desc.tail) {
+		for (i = snapshot->parallel.wq_desc.head;
+		     i != snapshot->parallel.wq_desc.tail;
+		     i = (i + sizeof(u32)) % WQ_SIZE)
+			snapshot->parallel.wq[i / sizeof(u32)] =
+				parallel_read(xe, map, wq[i / sizeof(u32)]);
+	}
+}
+
+static void
+guc_engine_wq_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
+			     struct drm_printer *p)
+{
+	int i;
+
 	drm_printf(p, "\tWQ head: %u (internal), %d (memory)\n",
-		   e->guc->wqi_head, parallel_read(xe, map, wq_desc.head));
+		   snapshot->guc.wqi_head, snapshot->parallel.wq_desc.head);
 	drm_printf(p, "\tWQ tail: %u (internal), %d (memory)\n",
-		   e->guc->wqi_tail, parallel_read(xe, map, wq_desc.tail));
-	drm_printf(p, "\tWQ status: %u\n",
-		   parallel_read(xe, map, wq_desc.wq_status));
-	if (parallel_read(xe, map, wq_desc.head) !=
-	    parallel_read(xe, map, wq_desc.tail)) {
-		for (i = parallel_read(xe, map, wq_desc.head);
-		     i != parallel_read(xe, map, wq_desc.tail);
+		   snapshot->guc.wqi_tail, snapshot->parallel.wq_desc.tail);
+	drm_printf(p, "\tWQ status: %u\n", snapshot->parallel.wq_desc.status);
+
+	if (snapshot->parallel.wq_desc.head !=
+	    snapshot->parallel.wq_desc.tail) {
+		for (i = snapshot->parallel.wq_desc.head;
+		     i != snapshot->parallel.wq_desc.tail;
 		     i = (i + sizeof(u32)) % WQ_SIZE)
 			drm_printf(p, "\tWQ[%zu]: 0x%08x\n", i / sizeof(u32),
-				   parallel_read(xe, map, wq[i / sizeof(u32)]));
+				   snapshot->parallel.wq[i / sizeof(u32)]);
 	}
 }
 
-static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
+/**
+ * xe_guc_engine_snapshot_capture - Take a quick snapshot of the GuC Engine.
+ * @e: Xe Engine.
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a GuC Submit Engine snapshot object that must be freed by the
+ * 	    caller, using `xe_guc_engine_snapshot_free`.
+ */
+struct xe_guc_submit_engine_snapshot *
+xe_guc_engine_snapshot_capture(struct xe_engine *e)
 {
 	struct drm_gpu_scheduler *sched = &e->guc->sched;
 	struct xe_sched_job *job;
+	struct xe_guc_submit_engine_snapshot *snapshot;
 	int i;
 
-	drm_printf(p, "\nGuC ID: %d\n", e->guc->id);
-	drm_printf(p, "\tName: %s\n", e->name);
-	drm_printf(p, "\tClass: %d\n", e->class);
-	drm_printf(p, "\tLogical mask: 0x%x\n", e->logical_mask);
-	drm_printf(p, "\tWidth: %d\n", e->width);
-	drm_printf(p, "\tRef: %d\n", kref_read(&e->refcount));
-	drm_printf(p, "\tTimeout: %ld (ms)\n", sched->timeout);
-	drm_printf(p, "\tTimeslice: %u (us)\n", e->sched_props.timeslice_us);
-	drm_printf(p, "\tPreempt timeout: %u (us)\n",
-		   e->sched_props.preempt_timeout_us);
+	snapshot = kzalloc(sizeof(struct xe_guc_submit_engine_snapshot),
+			   GFP_ATOMIC);
+
+	snapshot->guc.id = e->guc->id;
+	memcpy(&snapshot->name, &e->name, sizeof(snapshot->name));
+	snapshot->class = e->class;
+	snapshot->logical_mask = e->logical_mask;
+	snapshot->width = e->width;
+	snapshot->refcount = kref_read(&e->refcount);
+	snapshot->sched_timeout = sched->timeout;
+	snapshot->sched_props.timeslice_us = e->sched_props.timeslice_us;
+	snapshot->sched_props.preempt_timeout_us =
+		e->sched_props.preempt_timeout_us;
+
+	snapshot->lrc = kmalloc_array(e->width, sizeof(struct lrc_snapshot),
+				GFP_ATOMIC);
+
 	for (i = 0; i < e->width; ++i ) {
 		struct xe_lrc *lrc = e->lrc + i;
 
+		snapshot->lrc[i].context_desc =
+			lower_32_bits(xe_lrc_ggtt_addr(lrc));
+		snapshot->lrc[i].head = xe_lrc_ring_head(lrc);
+		snapshot->lrc[i].tail.internal = lrc->ring.tail;
+		snapshot->lrc[i].tail.memory =
+			xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL);
+		snapshot->lrc[i].start_seqno = xe_lrc_start_seqno(lrc);
+		snapshot->lrc[i].seqno = xe_lrc_seqno(lrc);
+
+	}
+
+	snapshot->schedule_state = atomic_read(&e->guc->state);
+	snapshot->engine_flags = e->flags;
+
+	snapshot->parallel_execution = xe_engine_is_parallel(e);
+	if (snapshot->parallel_execution)
+		guc_engine_wq_snapshot_capture(e, snapshot);
+
+	spin_lock(&sched->job_list_lock);
+	snapshot->pending_list_size = list_count_nodes(&sched->pending_list);
+	snapshot->pending_list = kmalloc_array(snapshot->pending_list_size,
+					 sizeof(struct pending_list_snapshot),
+					 GFP_ATOMIC);
+	i = 0;
+	list_for_each_entry(job, &sched->pending_list, drm.list) {
+		snapshot->pending_list[i].seqno = xe_sched_job_seqno(job);
+		snapshot->pending_list[i].fence =
+			dma_fence_is_signaled(job->fence) ? 1 : 0;
+		snapshot->pending_list[i].finished =
+			dma_fence_is_signaled(&job->drm.s_fence->finished)
+			? 1 : 0;
+		i++;
+	}
+	spin_unlock(&sched->job_list_lock);
+
+	return snapshot;
+}
+
+/**
+ * xe_guc_engine_snapshot_print - Print out a given GuC Engine snapshot.
+ * @snapshot: GuC Submit Engine snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given GuC Submit Engine snapshot object.
+ */
+void
+xe_guc_engine_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
+			     struct drm_printer *p)
+{
+	int i;
+
+	drm_printf(p, "\nGuC ID: %d\n", snapshot->guc.id);
+	drm_printf(p, "\tName: %s\n", snapshot->name);
+	drm_printf(p, "\tClass: %d\n", snapshot->class);
+	drm_printf(p, "\tLogical mask: 0x%x\n", snapshot->logical_mask);
+	drm_printf(p, "\tWidth: %d\n", snapshot->width);
+	drm_printf(p, "\tRef: %d\n", snapshot->refcount);
+	drm_printf(p, "\tTimeout: %ld (ms)\n", snapshot->sched_timeout);
+	drm_printf(p, "\tTimeslice: %u (us)\n",
+		   snapshot->sched_props.timeslice_us);
+	drm_printf(p, "\tPreempt timeout: %u (us)\n",
+		   snapshot->sched_props.preempt_timeout_us);
+
+	for (i = 0; i < snapshot->width; ++i ) {
 		drm_printf(p, "\tHW Context Desc: 0x%08x\n",
-			   lower_32_bits(xe_lrc_ggtt_addr(lrc)));
+			   snapshot->lrc[i].context_desc);
 		drm_printf(p, "\tLRC Head: (memory) %u\n",
-			   xe_lrc_ring_head(lrc));
+			   snapshot->lrc[i].head);
 		drm_printf(p, "\tLRC Tail: (internal) %u, (memory) %u\n",
-			   lrc->ring.tail,
-			   xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL));
+			   snapshot->lrc[i].tail.internal,
+			   snapshot->lrc[i].tail.memory);
 		drm_printf(p, "\tStart seqno: (memory) %d\n",
-			   xe_lrc_start_seqno(lrc));
-		drm_printf(p, "\tSeqno: (memory) %d\n", xe_lrc_seqno(lrc));
+			   snapshot->lrc[i].start_seqno);
+		drm_printf(p, "\tSeqno: (memory) %d\n", snapshot->lrc[i].seqno);
 	}
-	drm_printf(p, "\tSchedule State: 0x%x\n", atomic_read(&e->guc->state));
-	drm_printf(p, "\tFlags: 0x%lx\n", e->flags);
-	if (xe_engine_is_parallel(e))
-		guc_engine_wq_print(e, p);
+	drm_printf(p, "\tSchedule State: 0x%x\n", snapshot->schedule_state);
+	drm_printf(p, "\tFlags: 0x%lx\n", snapshot->engine_flags);
 
-	spin_lock(&sched->job_list_lock);
+	if (snapshot->parallel_execution)
+		guc_engine_wq_snapshot_print(snapshot, p);
 
-	list_for_each_entry(job, &sched->pending_list, drm.list)
+	for(i = 0; i < snapshot->pending_list_size; i++)
 		drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n",
-			   xe_sched_job_seqno(job),
-			   dma_fence_is_signaled(job->fence) ? 1 : 0,
-			   dma_fence_is_signaled(&job->drm.s_fence->finished) ?
-			   1 : 0);
-	spin_unlock(&sched->job_list_lock);
+			   snapshot->pending_list[i].seqno,
+			   snapshot->pending_list[i].fence,
+			   snapshot->pending_list[i].finished);
+}
+
+/**
+ * xe_guc_engine_snapshot_free - Free all allocated objects for a given
+ * snapshot.
+ * @snapshot: GuC Submit Engine snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_guc_engine_snapshot_free(struct xe_guc_submit_engine_snapshot *snapshot)
+{
+	kfree(snapshot->lrc);
+	kfree(snapshot->pending_list);
+	kfree(snapshot);
+}
+
+static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
+{
+	struct xe_guc_submit_engine_snapshot *snapshot;
+
+	snapshot = xe_guc_engine_snapshot_capture(e);
+	xe_guc_engine_snapshot_print(snapshot, p);
+	xe_guc_engine_snapshot_free(snapshot);
 }
 
+/**
+ * xe_guc_submit_print - GuC Submit Print.
+ * @guc: GuC.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function capture and prints snapshots of **all** GuC Engines.
+ */
 void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p)
 {
 	struct xe_engine *e;
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.h b/drivers/gpu/drm/xe/xe_guc_submit.h
index 8002734d6f24..4153c2d22013 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.h
+++ b/drivers/gpu/drm/xe/xe_guc_submit.h
@@ -13,7 +13,6 @@ struct xe_engine;
 struct xe_guc;
 
 int xe_guc_submit_init(struct xe_guc *guc);
-void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p);
 
 int xe_guc_submit_reset_prepare(struct xe_guc *guc);
 void xe_guc_submit_reset_wait(struct xe_guc *guc);
@@ -27,4 +26,13 @@ int xe_guc_engine_memory_cat_error_handler(struct xe_guc *guc, u32 *msg,
 					   u32 len);
 int xe_guc_engine_reset_failure_handler(struct xe_guc *guc, u32 *msg, u32 len);
 
+struct xe_guc_submit_engine_snapshot *
+xe_guc_engine_snapshot_capture(struct xe_engine *e);
+void
+xe_guc_engine_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
+			     struct drm_printer *p);
+void
+xe_guc_engine_snapshot_free(struct xe_guc_submit_engine_snapshot *snapshot);
+void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p);
+
 #endif
diff --git a/drivers/gpu/drm/xe/xe_guc_submit_types.h b/drivers/gpu/drm/xe/xe_guc_submit_types.h
index d369ea0bad60..0b726609dc14 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit_types.h
+++ b/drivers/gpu/drm/xe/xe_guc_submit_types.h
@@ -61,4 +61,95 @@ struct guc_submit_parallel_scratch {
 	u32 wq[WQ_SIZE / sizeof(u32)];
 };
 
+struct lrc_snapshot {
+	u32 context_desc;
+	u32 head;
+	struct {
+		u32 internal;
+		u32 memory;
+	} tail;
+	u32 start_seqno;
+	u32 seqno;
+};
+
+struct pending_list_snapshot {
+	u32 seqno;
+	bool fence;
+	bool finished;
+};
+
+/**
+ * struct xe_guc_submit_engine_snapshot - Snapshot for devcoredump
+ */
+struct xe_guc_submit_engine_snapshot {
+	/** @name: name of this engine */
+	char name[MAX_FENCE_NAME_LEN];
+	/** @class: class of this engine */
+	enum xe_engine_class class;
+	/**
+	 * @logical_mask: logical mask of where job submitted to engine can run
+	 */
+	u32 logical_mask;
+	/** @width: width (number BB submitted per exec) of this engine */
+	u16 width;
+	/** @refcount: ref count of this engine */
+	u32 refcount;
+	/**
+	 * @sched_timeout: the time after which a job is removed from the
+	 * scheduler.
+	 */
+	long sched_timeout;
+
+	/** @sched_props: scheduling properties */
+	struct {
+		/** @timeslice_us: timeslice period in micro-seconds */
+		u32 timeslice_us;
+		/** @preempt_timeout_us: preemption timeout in micro-seconds */
+		u32 preempt_timeout_us;
+	} sched_props;
+
+	/** @lrc: LRC Snapshot */
+	struct lrc_snapshot *lrc;
+
+	/** @schedule_state: Schedule State at the moment of Crash */
+	u32 schedule_state;
+	/** @engine_flags: Flags of the faulty engine */
+	unsigned long engine_flags;
+
+	/** @guc: GuC Engine Snapshot */
+	struct {
+		/** @wqi_head: work queue item head */
+		u32 wqi_head;
+		/** @wqi_tail: work queue item tail */
+		u32 wqi_tail;
+		/** @id: GuC id for this xe_engine */
+		u16 id;
+	} guc;
+
+	/**
+	 * @parallel_execution: Indication if the failure was during parallel
+	 * execution
+	 */
+	bool parallel_execution;
+	/** @parallel: snapshot of the useful parallel scratch */
+	struct {
+		/** @wq_desc: Workqueue description */
+		struct {
+			/** @head: Workqueue Head */
+			u32 head;
+			/** @tail: Workqueue Tail */
+			u32 tail;
+			/** @status: Workqueue Status */
+			u32 status;
+		} wq_desc;
+		/** @wq: Workqueue Items */
+		u32 wq[WQ_SIZE / sizeof(u32)];
+	} parallel;
+
+	/** @pending_list_size: Size of the pending list snapshot array */
+	int pending_list_size;
+	/** @pending_list: snapshot of the pending list info */
+	struct pending_list_snapshot *pending_list;
+};
+
 #endif
-- 
2.39.2

[PATCH 08/14] drm/xe: Convert GuC Engine print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_submit.c       | 212 +++++++++++++++++++----
 drivers/gpu/drm/xe/xe_guc_submit.h       |  10 +-
 drivers/gpu/drm/xe/xe_guc_submit_types.h |  91 ++++++++++
 3 files changed, 274 insertions(+), 39 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index a5fe7755ce4c..9c06411f857f 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -1596,75 +1596,211 @@ int xe_guc_engine_reset_failure_handler(struct xe_guc *guc, u32 *msg, u32 len)
 	return 0;
 }
 
-static void guc_engine_wq_print(struct xe_engine *e, struct drm_printer *p)
+static void
+guc_engine_wq_snapshot_capture(struct xe_engine *e,
+			       struct xe_guc_submit_engine_snapshot *snapshot)
 {
 	struct xe_guc *guc = engine_to_guc(e);
 	struct xe_device *xe = guc_to_xe(guc);
 	struct iosys_map map = xe_lrc_parallel_map(e->lrc);
 	int i;
 
+	snapshot->guc.wqi_head = e->guc->wqi_head;
+	snapshot->guc.wqi_tail = e->guc->wqi_tail;
+	snapshot->parallel.wq_desc.head = parallel_read(xe, map, wq_desc.head);
+	snapshot->parallel.wq_desc.tail = parallel_read(xe, map, wq_desc.tail);
+	snapshot->parallel.wq_desc.status = parallel_read(xe, map,
+							  wq_desc.wq_status);
+
+	if (snapshot->parallel.wq_desc.head !=
+	    snapshot->parallel.wq_desc.tail) {
+		for (i = snapshot->parallel.wq_desc.head;
+		     i != snapshot->parallel.wq_desc.tail;
+		     i = (i + sizeof(u32)) % WQ_SIZE)
+			snapshot->parallel.wq[i / sizeof(u32)] =
+				parallel_read(xe, map, wq[i / sizeof(u32)]);
+	}
+}
+
+static void
+guc_engine_wq_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
+			     struct drm_printer *p)
+{
+	int i;
+
 	drm_printf(p, "\tWQ head: %u (internal), %d (memory)\n",
-		   e->guc->wqi_head, parallel_read(xe, map, wq_desc.head));
+		   snapshot->guc.wqi_head, snapshot->parallel.wq_desc.head);
 	drm_printf(p, "\tWQ tail: %u (internal), %d (memory)\n",
-		   e->guc->wqi_tail, parallel_read(xe, map, wq_desc.tail));
-	drm_printf(p, "\tWQ status: %u\n",
-		   parallel_read(xe, map, wq_desc.wq_status));
-	if (parallel_read(xe, map, wq_desc.head) !=
-	    parallel_read(xe, map, wq_desc.tail)) {
-		for (i = parallel_read(xe, map, wq_desc.head);
-		     i != parallel_read(xe, map, wq_desc.tail);
+		   snapshot->guc.wqi_tail, snapshot->parallel.wq_desc.tail);
+	drm_printf(p, "\tWQ status: %u\n", snapshot->parallel.wq_desc.status);
+
+	if (snapshot->parallel.wq_desc.head !=
+	    snapshot->parallel.wq_desc.tail) {
+		for (i = snapshot->parallel.wq_desc.head;
+		     i != snapshot->parallel.wq_desc.tail;
 		     i = (i + sizeof(u32)) % WQ_SIZE)
 			drm_printf(p, "\tWQ[%zu]: 0x%08x\n", i / sizeof(u32),
-				   parallel_read(xe, map, wq[i / sizeof(u32)]));
+				   snapshot->parallel.wq[i / sizeof(u32)]);
 	}
 }
 
-static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
+/**
+ * xe_guc_engine_snapshot_capture - Take a quick snapshot of the GuC Engine.
+ * @e: Xe Engine.
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a GuC Submit Engine snapshot object that must be freed by the
+ * 	    caller, using `xe_guc_engine_snapshot_free`.
+ */
+struct xe_guc_submit_engine_snapshot *
+xe_guc_engine_snapshot_capture(struct xe_engine *e)
 {
 	struct drm_gpu_scheduler *sched = &e->guc->sched;
 	struct xe_sched_job *job;
+	struct xe_guc_submit_engine_snapshot *snapshot;
 	int i;
 
-	drm_printf(p, "\nGuC ID: %d\n", e->guc->id);
-	drm_printf(p, "\tName: %s\n", e->name);
-	drm_printf(p, "\tClass: %d\n", e->class);
-	drm_printf(p, "\tLogical mask: 0x%x\n", e->logical_mask);
-	drm_printf(p, "\tWidth: %d\n", e->width);
-	drm_printf(p, "\tRef: %d\n", kref_read(&e->refcount));
-	drm_printf(p, "\tTimeout: %ld (ms)\n", sched->timeout);
-	drm_printf(p, "\tTimeslice: %u (us)\n", e->sched_props.timeslice_us);
-	drm_printf(p, "\tPreempt timeout: %u (us)\n",
-		   e->sched_props.preempt_timeout_us);
+	snapshot = kzalloc(sizeof(struct xe_guc_submit_engine_snapshot),
+			   GFP_ATOMIC);
+
+	snapshot->guc.id = e->guc->id;
+	memcpy(&snapshot->name, &e->name, sizeof(snapshot->name));
+	snapshot->class = e->class;
+	snapshot->logical_mask = e->logical_mask;
+	snapshot->width = e->width;
+	snapshot->refcount = kref_read(&e->refcount);
+	snapshot->sched_timeout = sched->timeout;
+	snapshot->sched_props.timeslice_us = e->sched_props.timeslice_us;
+	snapshot->sched_props.preempt_timeout_us =
+		e->sched_props.preempt_timeout_us;
+
+	snapshot->lrc = kmalloc_array(e->width, sizeof(struct lrc_snapshot),
+				GFP_ATOMIC);
+
 	for (i = 0; i < e->width; ++i ) {
 		struct xe_lrc *lrc = e->lrc + i;
 
+		snapshot->lrc[i].context_desc =
+			lower_32_bits(xe_lrc_ggtt_addr(lrc));
+		snapshot->lrc[i].head = xe_lrc_ring_head(lrc);
+		snapshot->lrc[i].tail.internal = lrc->ring.tail;
+		snapshot->lrc[i].tail.memory =
+			xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL);
+		snapshot->lrc[i].start_seqno = xe_lrc_start_seqno(lrc);
+		snapshot->lrc[i].seqno = xe_lrc_seqno(lrc);
+
+	}
+
+	snapshot->schedule_state = atomic_read(&e->guc->state);
+	snapshot->engine_flags = e->flags;
+
+	snapshot->parallel_execution = xe_engine_is_parallel(e);
+	if (snapshot->parallel_execution)
+		guc_engine_wq_snapshot_capture(e, snapshot);
+
+	spin_lock(&sched->job_list_lock);
+	snapshot->pending_list_size = list_count_nodes(&sched->pending_list);
+	snapshot->pending_list = kmalloc_array(snapshot->pending_list_size,
+					 sizeof(struct pending_list_snapshot),
+					 GFP_ATOMIC);
+	i = 0;
+	list_for_each_entry(job, &sched->pending_list, drm.list) {
+		snapshot->pending_list[i].seqno = xe_sched_job_seqno(job);
+		snapshot->pending_list[i].fence =
+			dma_fence_is_signaled(job->fence) ? 1 : 0;
+		snapshot->pending_list[i].finished =
+			dma_fence_is_signaled(&job->drm.s_fence->finished)
+			? 1 : 0;
+		i++;
+	}
+	spin_unlock(&sched->job_list_lock);
+
+	return snapshot;
+}
+
+/**
+ * xe_guc_engine_snapshot_print - Print out a given GuC Engine snapshot.
+ * @snapshot: GuC Submit Engine snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given GuC Submit Engine snapshot object.
+ */
+void
+xe_guc_engine_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
+			     struct drm_printer *p)
+{
+	int i;
+
+	drm_printf(p, "\nGuC ID: %d\n", snapshot->guc.id);
+	drm_printf(p, "\tName: %s\n", snapshot->name);
+	drm_printf(p, "\tClass: %d\n", snapshot->class);
+	drm_printf(p, "\tLogical mask: 0x%x\n", snapshot->logical_mask);
+	drm_printf(p, "\tWidth: %d\n", snapshot->width);
+	drm_printf(p, "\tRef: %d\n", snapshot->refcount);
+	drm_printf(p, "\tTimeout: %ld (ms)\n", snapshot->sched_timeout);
+	drm_printf(p, "\tTimeslice: %u (us)\n",
+		   snapshot->sched_props.timeslice_us);
+	drm_printf(p, "\tPreempt timeout: %u (us)\n",
+		   snapshot->sched_props.preempt_timeout_us);
+
+	for (i = 0; i < snapshot->width; ++i ) {
 		drm_printf(p, "\tHW Context Desc: 0x%08x\n",
-			   lower_32_bits(xe_lrc_ggtt_addr(lrc)));
+			   snapshot->lrc[i].context_desc);
 		drm_printf(p, "\tLRC Head: (memory) %u\n",
-			   xe_lrc_ring_head(lrc));
+			   snapshot->lrc[i].head);
 		drm_printf(p, "\tLRC Tail: (internal) %u, (memory) %u\n",
-			   lrc->ring.tail,
-			   xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL));
+			   snapshot->lrc[i].tail.internal,
+			   snapshot->lrc[i].tail.memory);
 		drm_printf(p, "\tStart seqno: (memory) %d\n",
-			   xe_lrc_start_seqno(lrc));
-		drm_printf(p, "\tSeqno: (memory) %d\n", xe_lrc_seqno(lrc));
+			   snapshot->lrc[i].start_seqno);
+		drm_printf(p, "\tSeqno: (memory) %d\n", snapshot->lrc[i].seqno);
 	}
-	drm_printf(p, "\tSchedule State: 0x%x\n", atomic_read(&e->guc->state));
-	drm_printf(p, "\tFlags: 0x%lx\n", e->flags);
-	if (xe_engine_is_parallel(e))
-		guc_engine_wq_print(e, p);
+	drm_printf(p, "\tSchedule State: 0x%x\n", snapshot->schedule_state);
+	drm_printf(p, "\tFlags: 0x%lx\n", snapshot->engine_flags);
 
-	spin_lock(&sched->job_list_lock);
+	if (snapshot->parallel_execution)
+		guc_engine_wq_snapshot_print(snapshot, p);
 
-	list_for_each_entry(job, &sched->pending_list, drm.list)
+	for(i = 0; i < snapshot->pending_list_size; i++)
 		drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n",
-			   xe_sched_job_seqno(job),
-			   dma_fence_is_signaled(job->fence) ? 1 : 0,
-			   dma_fence_is_signaled(&job->drm.s_fence->finished) ?
-			   1 : 0);
-	spin_unlock(&sched->job_list_lock);
+			   snapshot->pending_list[i].seqno,
+			   snapshot->pending_list[i].fence,
+			   snapshot->pending_list[i].finished);
+}
+
+/**
+ * xe_guc_engine_snapshot_free - Free all allocated objects for a given
+ * snapshot.
+ * @snapshot: GuC Submit Engine snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_guc_engine_snapshot_free(struct xe_guc_submit_engine_snapshot *snapshot)
+{
+	kfree(snapshot->lrc);
+	kfree(snapshot->pending_list);
+	kfree(snapshot);
+}
+
+static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
+{
+	struct xe_guc_submit_engine_snapshot *snapshot;
+
+	snapshot = xe_guc_engine_snapshot_capture(e);
+	xe_guc_engine_snapshot_print(snapshot, p);
+	xe_guc_engine_snapshot_free(snapshot);
 }
 
+/**
+ * xe_guc_submit_print - GuC Submit Print.
+ * @guc: GuC.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function capture and prints snapshots of **all** GuC Engines.
+ */
 void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p)
 {
 	struct xe_engine *e;
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.h b/drivers/gpu/drm/xe/xe_guc_submit.h
index 8002734d6f24..4153c2d22013 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.h
+++ b/drivers/gpu/drm/xe/xe_guc_submit.h
@@ -13,7 +13,6 @@ struct xe_engine;
 struct xe_guc;
 
 int xe_guc_submit_init(struct xe_guc *guc);
-void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p);
 
 int xe_guc_submit_reset_prepare(struct xe_guc *guc);
 void xe_guc_submit_reset_wait(struct xe_guc *guc);
@@ -27,4 +26,13 @@ int xe_guc_engine_memory_cat_error_handler(struct xe_guc *guc, u32 *msg,
 					   u32 len);
 int xe_guc_engine_reset_failure_handler(struct xe_guc *guc, u32 *msg, u32 len);
 
+struct xe_guc_submit_engine_snapshot *
+xe_guc_engine_snapshot_capture(struct xe_engine *e);
+void
+xe_guc_engine_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
+			     struct drm_printer *p);
+void
+xe_guc_engine_snapshot_free(struct xe_guc_submit_engine_snapshot *snapshot);
+void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p);
+
 #endif
diff --git a/drivers/gpu/drm/xe/xe_guc_submit_types.h b/drivers/gpu/drm/xe/xe_guc_submit_types.h
index d369ea0bad60..0b726609dc14 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit_types.h
+++ b/drivers/gpu/drm/xe/xe_guc_submit_types.h
@@ -61,4 +61,95 @@ struct guc_submit_parallel_scratch {
 	u32 wq[WQ_SIZE / sizeof(u32)];
 };
 
+struct lrc_snapshot {
+	u32 context_desc;
+	u32 head;
+	struct {
+		u32 internal;
+		u32 memory;
+	} tail;
+	u32 start_seqno;
+	u32 seqno;
+};
+
+struct pending_list_snapshot {
+	u32 seqno;
+	bool fence;
+	bool finished;
+};
+
+/**
+ * struct xe_guc_submit_engine_snapshot - Snapshot for devcoredump
+ */
+struct xe_guc_submit_engine_snapshot {
+	/** @name: name of this engine */
+	char name[MAX_FENCE_NAME_LEN];
+	/** @class: class of this engine */
+	enum xe_engine_class class;
+	/**
+	 * @logical_mask: logical mask of where job submitted to engine can run
+	 */
+	u32 logical_mask;
+	/** @width: width (number BB submitted per exec) of this engine */
+	u16 width;
+	/** @refcount: ref count of this engine */
+	u32 refcount;
+	/**
+	 * @sched_timeout: the time after which a job is removed from the
+	 * scheduler.
+	 */
+	long sched_timeout;
+
+	/** @sched_props: scheduling properties */
+	struct {
+		/** @timeslice_us: timeslice period in micro-seconds */
+		u32 timeslice_us;
+		/** @preempt_timeout_us: preemption timeout in micro-seconds */
+		u32 preempt_timeout_us;
+	} sched_props;
+
+	/** @lrc: LRC Snapshot */
+	struct lrc_snapshot *lrc;
+
+	/** @schedule_state: Schedule State at the moment of Crash */
+	u32 schedule_state;
+	/** @engine_flags: Flags of the faulty engine */
+	unsigned long engine_flags;
+
+	/** @guc: GuC Engine Snapshot */
+	struct {
+		/** @wqi_head: work queue item head */
+		u32 wqi_head;
+		/** @wqi_tail: work queue item tail */
+		u32 wqi_tail;
+		/** @id: GuC id for this xe_engine */
+		u16 id;
+	} guc;
+
+	/**
+	 * @parallel_execution: Indication if the failure was during parallel
+	 * execution
+	 */
+	bool parallel_execution;
+	/** @parallel: snapshot of the useful parallel scratch */
+	struct {
+		/** @wq_desc: Workqueue description */
+		struct {
+			/** @head: Workqueue Head */
+			u32 head;
+			/** @tail: Workqueue Tail */
+			u32 tail;
+			/** @status: Workqueue Status */
+			u32 status;
+		} wq_desc;
+		/** @wq: Workqueue Items */
+		u32 wq[WQ_SIZE / sizeof(u32)];
+	} parallel;
+
+	/** @pending_list_size: Size of the pending list snapshot array */
+	int pending_list_size;
+	/** @pending_list: snapshot of the pending list info */
+	struct pending_list_snapshot *pending_list;
+};
+
 #endif
-- 
2.39.2

Re: [Intel-xe] [PATCH 08/14] drm/xe: Convert GuC Engine print to snapshot capture and print.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:07PM -0400, Rodrigo Vivi wrote:
> The goal is to allow for a snapshot capture to be taken at the time
> of the crash, while the print out can happen at a later time through
> the exposed devcoredump virtual device.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_guc_submit.c       | 212 +++++++++++++++++++----
>  drivers/gpu/drm/xe/xe_guc_submit.h       |  10 +-
>  drivers/gpu/drm/xe/xe_guc_submit_types.h |  91 ++++++++++
>  3 files changed, 274 insertions(+), 39 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> index a5fe7755ce4c..9c06411f857f 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> @@ -1596,75 +1596,211 @@ int xe_guc_engine_reset_failure_handler(struct xe_guc *guc, u32 *msg, u32 len)
>  	return 0;
>  }
>  
> -static void guc_engine_wq_print(struct xe_engine *e, struct drm_printer *p)
> +static void
> +guc_engine_wq_snapshot_capture(struct xe_engine *e,
> +			       struct xe_guc_submit_engine_snapshot *snapshot)
>  {
>  	struct xe_guc *guc = engine_to_guc(e);
>  	struct xe_device *xe = guc_to_xe(guc);
>  	struct iosys_map map = xe_lrc_parallel_map(e->lrc);
>  	int i;
>  
> +	snapshot->guc.wqi_head = e->guc->wqi_head;
> +	snapshot->guc.wqi_tail = e->guc->wqi_tail;
> +	snapshot->parallel.wq_desc.head = parallel_read(xe, map, wq_desc.head);
> +	snapshot->parallel.wq_desc.tail = parallel_read(xe, map, wq_desc.tail);
> +	snapshot->parallel.wq_desc.status = parallel_read(xe, map,
> +							  wq_desc.wq_status);
> +
> +	if (snapshot->parallel.wq_desc.head !=
> +	    snapshot->parallel.wq_desc.tail) {
> +		for (i = snapshot->parallel.wq_desc.head;
> +		     i != snapshot->parallel.wq_desc.tail;
> +		     i = (i + sizeof(u32)) % WQ_SIZE)
> +			snapshot->parallel.wq[i / sizeof(u32)] =
> +				parallel_read(xe, map, wq[i / sizeof(u32)]);
> +	}
> +}
> +
> +static void
> +guc_engine_wq_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
> +			     struct drm_printer *p)
> +{
> +	int i;
> +
>  	drm_printf(p, "\tWQ head: %u (internal), %d (memory)\n",
> -		   e->guc->wqi_head, parallel_read(xe, map, wq_desc.head));
> +		   snapshot->guc.wqi_head, snapshot->parallel.wq_desc.head);
>  	drm_printf(p, "\tWQ tail: %u (internal), %d (memory)\n",
> -		   e->guc->wqi_tail, parallel_read(xe, map, wq_desc.tail));
> -	drm_printf(p, "\tWQ status: %u\n",
> -		   parallel_read(xe, map, wq_desc.wq_status));
> -	if (parallel_read(xe, map, wq_desc.head) !=
> -	    parallel_read(xe, map, wq_desc.tail)) {
> -		for (i = parallel_read(xe, map, wq_desc.head);
> -		     i != parallel_read(xe, map, wq_desc.tail);
> +		   snapshot->guc.wqi_tail, snapshot->parallel.wq_desc.tail);
> +	drm_printf(p, "\tWQ status: %u\n", snapshot->parallel.wq_desc.status);
> +
> +	if (snapshot->parallel.wq_desc.head !=
> +	    snapshot->parallel.wq_desc.tail) {
> +		for (i = snapshot->parallel.wq_desc.head;
> +		     i != snapshot->parallel.wq_desc.tail;
>  		     i = (i + sizeof(u32)) % WQ_SIZE)
>  			drm_printf(p, "\tWQ[%zu]: 0x%08x\n", i / sizeof(u32),
> -				   parallel_read(xe, map, wq[i / sizeof(u32)]));
> +				   snapshot->parallel.wq[i / sizeof(u32)]);
>  	}
>  }
>  
> -static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
> +/**
> + * xe_guc_engine_snapshot_capture - Take a quick snapshot of the GuC Engine.
> + * @e: Xe Engine.
> + *
> + * This can be printed out in a later stage like during dev_coredump
> + * analysis.
> + *
> + * Returns: a GuC Submit Engine snapshot object that must be freed by the
> + * 	    caller, using `xe_guc_engine_snapshot_free`.
> + */
> +struct xe_guc_submit_engine_snapshot *
> +xe_guc_engine_snapshot_capture(struct xe_engine *e)
>  {
>  	struct drm_gpu_scheduler *sched = &e->guc->sched;
>  	struct xe_sched_job *job;
> +	struct xe_guc_submit_engine_snapshot *snapshot;
>  	int i;
>  
> -	drm_printf(p, "\nGuC ID: %d\n", e->guc->id);
> -	drm_printf(p, "\tName: %s\n", e->name);
> -	drm_printf(p, "\tClass: %d\n", e->class);
> -	drm_printf(p, "\tLogical mask: 0x%x\n", e->logical_mask);
> -	drm_printf(p, "\tWidth: %d\n", e->width);
> -	drm_printf(p, "\tRef: %d\n", kref_read(&e->refcount));
> -	drm_printf(p, "\tTimeout: %ld (ms)\n", sched->timeout);
> -	drm_printf(p, "\tTimeslice: %u (us)\n", e->sched_props.timeslice_us);
> -	drm_printf(p, "\tPreempt timeout: %u (us)\n",
> -		   e->sched_props.preempt_timeout_us);
> +	snapshot = kzalloc(sizeof(struct xe_guc_submit_engine_snapshot),
> +			   GFP_ATOMIC);

For the whole file you need to check if any of the allocs fail. Also
let's say if just the last alloc fails I wouldn't fail the entire
capture, just abort at that point (i.e. some info is better than none).

Matt

> +
> +	snapshot->guc.id = e->guc->id;
> +	memcpy(&snapshot->name, &e->name, sizeof(snapshot->name));
> +	snapshot->class = e->class;
> +	snapshot->logical_mask = e->logical_mask;
> +	snapshot->width = e->width;
> +	snapshot->refcount = kref_read(&e->refcount);
> +	snapshot->sched_timeout = sched->timeout;
> +	snapshot->sched_props.timeslice_us = e->sched_props.timeslice_us;
> +	snapshot->sched_props.preempt_timeout_us =
> +		e->sched_props.preempt_timeout_us;
> +
> +	snapshot->lrc = kmalloc_array(e->width, sizeof(struct lrc_snapshot),
> +				GFP_ATOMIC);
> +
>  	for (i = 0; i < e->width; ++i ) {
>  		struct xe_lrc *lrc = e->lrc + i;
>  
> +		snapshot->lrc[i].context_desc =
> +			lower_32_bits(xe_lrc_ggtt_addr(lrc));
> +		snapshot->lrc[i].head = xe_lrc_ring_head(lrc);
> +		snapshot->lrc[i].tail.internal = lrc->ring.tail;
> +		snapshot->lrc[i].tail.memory =
> +			xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL);
> +		snapshot->lrc[i].start_seqno = xe_lrc_start_seqno(lrc);
> +		snapshot->lrc[i].seqno = xe_lrc_seqno(lrc);
> +
> +	}
> +
> +	snapshot->schedule_state = atomic_read(&e->guc->state);
> +	snapshot->engine_flags = e->flags;
> +
> +	snapshot->parallel_execution = xe_engine_is_parallel(e);
> +	if (snapshot->parallel_execution)
> +		guc_engine_wq_snapshot_capture(e, snapshot);
> +
> +	spin_lock(&sched->job_list_lock);
> +	snapshot->pending_list_size = list_count_nodes(&sched->pending_list);
> +	snapshot->pending_list = kmalloc_array(snapshot->pending_list_size,
> +					 sizeof(struct pending_list_snapshot),
> +					 GFP_ATOMIC);
> +	i = 0;
> +	list_for_each_entry(job, &sched->pending_list, drm.list) {
> +		snapshot->pending_list[i].seqno = xe_sched_job_seqno(job);
> +		snapshot->pending_list[i].fence =
> +			dma_fence_is_signaled(job->fence) ? 1 : 0;
> +		snapshot->pending_list[i].finished =
> +			dma_fence_is_signaled(&job->drm.s_fence->finished)
> +			? 1 : 0;
> +		i++;
> +	}
> +	spin_unlock(&sched->job_list_lock);
> +
> +	return snapshot;
> +}
> +
> +/**
> + * xe_guc_engine_snapshot_print - Print out a given GuC Engine snapshot.
> + * @snapshot: GuC Submit Engine snapshot object.
> + * @p: drm_printer where it will be printed out.
> + *
> + * This function prints out a given GuC Submit Engine snapshot object.
> + */
> +void
> +xe_guc_engine_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
> +			     struct drm_printer *p)
> +{
> +	int i;
> +
> +	drm_printf(p, "\nGuC ID: %d\n", snapshot->guc.id);
> +	drm_printf(p, "\tName: %s\n", snapshot->name);
> +	drm_printf(p, "\tClass: %d\n", snapshot->class);
> +	drm_printf(p, "\tLogical mask: 0x%x\n", snapshot->logical_mask);
> +	drm_printf(p, "\tWidth: %d\n", snapshot->width);
> +	drm_printf(p, "\tRef: %d\n", snapshot->refcount);
> +	drm_printf(p, "\tTimeout: %ld (ms)\n", snapshot->sched_timeout);
> +	drm_printf(p, "\tTimeslice: %u (us)\n",
> +		   snapshot->sched_props.timeslice_us);
> +	drm_printf(p, "\tPreempt timeout: %u (us)\n",
> +		   snapshot->sched_props.preempt_timeout_us);
> +
> +	for (i = 0; i < snapshot->width; ++i ) {
>  		drm_printf(p, "\tHW Context Desc: 0x%08x\n",
> -			   lower_32_bits(xe_lrc_ggtt_addr(lrc)));
> +			   snapshot->lrc[i].context_desc);
>  		drm_printf(p, "\tLRC Head: (memory) %u\n",
> -			   xe_lrc_ring_head(lrc));
> +			   snapshot->lrc[i].head);
>  		drm_printf(p, "\tLRC Tail: (internal) %u, (memory) %u\n",
> -			   lrc->ring.tail,
> -			   xe_lrc_read_ctx_reg(lrc, CTX_RING_TAIL));
> +			   snapshot->lrc[i].tail.internal,
> +			   snapshot->lrc[i].tail.memory);
>  		drm_printf(p, "\tStart seqno: (memory) %d\n",
> -			   xe_lrc_start_seqno(lrc));
> -		drm_printf(p, "\tSeqno: (memory) %d\n", xe_lrc_seqno(lrc));
> +			   snapshot->lrc[i].start_seqno);
> +		drm_printf(p, "\tSeqno: (memory) %d\n", snapshot->lrc[i].seqno);
>  	}
> -	drm_printf(p, "\tSchedule State: 0x%x\n", atomic_read(&e->guc->state));
> -	drm_printf(p, "\tFlags: 0x%lx\n", e->flags);
> -	if (xe_engine_is_parallel(e))
> -		guc_engine_wq_print(e, p);
> +	drm_printf(p, "\tSchedule State: 0x%x\n", snapshot->schedule_state);
> +	drm_printf(p, "\tFlags: 0x%lx\n", snapshot->engine_flags);
>  
> -	spin_lock(&sched->job_list_lock);
> +	if (snapshot->parallel_execution)
> +		guc_engine_wq_snapshot_print(snapshot, p);
>  
> -	list_for_each_entry(job, &sched->pending_list, drm.list)
> +	for(i = 0; i < snapshot->pending_list_size; i++)
>  		drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n",
> -			   xe_sched_job_seqno(job),
> -			   dma_fence_is_signaled(job->fence) ? 1 : 0,
> -			   dma_fence_is_signaled(&job->drm.s_fence->finished) ?
> -			   1 : 0);
> -	spin_unlock(&sched->job_list_lock);
> +			   snapshot->pending_list[i].seqno,
> +			   snapshot->pending_list[i].fence,
> +			   snapshot->pending_list[i].finished);
> +}
> +
> +/**
> + * xe_guc_engine_snapshot_free - Free all allocated objects for a given
> + * snapshot.
> + * @snapshot: GuC Submit Engine snapshot object.
> + *
> + * This function free all the memory that needed to be allocated at capture
> + * time.
> + */
> +void xe_guc_engine_snapshot_free(struct xe_guc_submit_engine_snapshot *snapshot)
> +{
> +	kfree(snapshot->lrc);
> +	kfree(snapshot->pending_list);
> +	kfree(snapshot);
> +}
> +
> +static void guc_engine_print(struct xe_engine *e, struct drm_printer *p)
> +{
> +	struct xe_guc_submit_engine_snapshot *snapshot;
> +
> +	snapshot = xe_guc_engine_snapshot_capture(e);
> +	xe_guc_engine_snapshot_print(snapshot, p);
> +	xe_guc_engine_snapshot_free(snapshot);
>  }
>  
> +/**
> + * xe_guc_submit_print - GuC Submit Print.
> + * @guc: GuC.
> + * @p: drm_printer where it will be printed out.
> + *
> + * This function capture and prints snapshots of **all** GuC Engines.
> + */
>  void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p)
>  {
>  	struct xe_engine *e;
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.h b/drivers/gpu/drm/xe/xe_guc_submit.h
> index 8002734d6f24..4153c2d22013 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.h
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.h
> @@ -13,7 +13,6 @@ struct xe_engine;
>  struct xe_guc;
>  
>  int xe_guc_submit_init(struct xe_guc *guc);
> -void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p);
>  
>  int xe_guc_submit_reset_prepare(struct xe_guc *guc);
>  void xe_guc_submit_reset_wait(struct xe_guc *guc);
> @@ -27,4 +26,13 @@ int xe_guc_engine_memory_cat_error_handler(struct xe_guc *guc, u32 *msg,
>  					   u32 len);
>  int xe_guc_engine_reset_failure_handler(struct xe_guc *guc, u32 *msg, u32 len);
>  
> +struct xe_guc_submit_engine_snapshot *
> +xe_guc_engine_snapshot_capture(struct xe_engine *e);
> +void
> +xe_guc_engine_snapshot_print(struct xe_guc_submit_engine_snapshot *snapshot,
> +			     struct drm_printer *p);
> +void
> +xe_guc_engine_snapshot_free(struct xe_guc_submit_engine_snapshot *snapshot);
> +void xe_guc_submit_print(struct xe_guc *guc, struct drm_printer *p);
> +
>  #endif
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit_types.h b/drivers/gpu/drm/xe/xe_guc_submit_types.h
> index d369ea0bad60..0b726609dc14 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit_types.h
> +++ b/drivers/gpu/drm/xe/xe_guc_submit_types.h
> @@ -61,4 +61,95 @@ struct guc_submit_parallel_scratch {
>  	u32 wq[WQ_SIZE / sizeof(u32)];
>  };
>  
> +struct lrc_snapshot {
> +	u32 context_desc;
> +	u32 head;
> +	struct {
> +		u32 internal;
> +		u32 memory;
> +	} tail;
> +	u32 start_seqno;
> +	u32 seqno;
> +};
> +
> +struct pending_list_snapshot {
> +	u32 seqno;
> +	bool fence;
> +	bool finished;
> +};
> +
> +/**
> + * struct xe_guc_submit_engine_snapshot - Snapshot for devcoredump
> + */
> +struct xe_guc_submit_engine_snapshot {
> +	/** @name: name of this engine */
> +	char name[MAX_FENCE_NAME_LEN];
> +	/** @class: class of this engine */
> +	enum xe_engine_class class;
> +	/**
> +	 * @logical_mask: logical mask of where job submitted to engine can run
> +	 */
> +	u32 logical_mask;
> +	/** @width: width (number BB submitted per exec) of this engine */
> +	u16 width;
> +	/** @refcount: ref count of this engine */
> +	u32 refcount;
> +	/**
> +	 * @sched_timeout: the time after which a job is removed from the
> +	 * scheduler.
> +	 */
> +	long sched_timeout;
> +
> +	/** @sched_props: scheduling properties */
> +	struct {
> +		/** @timeslice_us: timeslice period in micro-seconds */
> +		u32 timeslice_us;
> +		/** @preempt_timeout_us: preemption timeout in micro-seconds */
> +		u32 preempt_timeout_us;
> +	} sched_props;
> +
> +	/** @lrc: LRC Snapshot */
> +	struct lrc_snapshot *lrc;
> +
> +	/** @schedule_state: Schedule State at the moment of Crash */
> +	u32 schedule_state;
> +	/** @engine_flags: Flags of the faulty engine */
> +	unsigned long engine_flags;
> +
> +	/** @guc: GuC Engine Snapshot */
> +	struct {
> +		/** @wqi_head: work queue item head */
> +		u32 wqi_head;
> +		/** @wqi_tail: work queue item tail */
> +		u32 wqi_tail;
> +		/** @id: GuC id for this xe_engine */
> +		u16 id;
> +	} guc;
> +
> +	/**
> +	 * @parallel_execution: Indication if the failure was during parallel
> +	 * execution
> +	 */
> +	bool parallel_execution;
> +	/** @parallel: snapshot of the useful parallel scratch */
> +	struct {
> +		/** @wq_desc: Workqueue description */
> +		struct {
> +			/** @head: Workqueue Head */
> +			u32 head;
> +			/** @tail: Workqueue Tail */
> +			u32 tail;
> +			/** @status: Workqueue Status */
> +			u32 status;
> +		} wq_desc;
> +		/** @wq: Workqueue Items */
> +		u32 wq[WQ_SIZE / sizeof(u32)];
> +	} parallel;
> +
> +	/** @pending_list_size: Size of the pending list snapshot array */
> +	int pending_list_size;
> +	/** @pending_list: snapshot of the pending list info */
> +	struct pending_list_snapshot *pending_list;
> +};
> +
>  #endif
> -- 
> 2.39.2
> 

[Intel-xe] [PATCH 09/14] drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.

[Rodrigo Vivi]

Let's start to move our existent logs to devcoredump one by
one. Any format change should come on follow-up work.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 7 ++++++-
 drivers/gpu/drm/xe/xe_devcoredump_types.h | 2 ++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index 795581c58d90..0e7ec654a9f2 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -12,6 +12,7 @@
 #include "xe_engine.h"
 #include "xe_gt.h"
 #include "xe_guc_ct.h"
+#include "xe_guc_submit.h"
 
 /**
  * DOC: Xe device coredump
@@ -89,6 +90,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 
 	drm_printf(&p, "\n**** GuC CT ****\n");
 	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
+	xe_guc_engine_snapshot_print(coredump->snapshot.ge, &p);
 
 	mutex_unlock(&coredump->lock);
 
@@ -106,6 +108,7 @@ static void xe_devcoredump_free(void *data)
 	mutex_lock(&coredump->lock);
 
 	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
+	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
 
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
@@ -117,13 +120,15 @@ static void xe_devcoredump_free(void *data)
 static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 {
 	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
-	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
+	struct xe_engine *e = coredump->faulty_engine;
+	struct xe_guc *guc = engine_to_guc(e);
 
 	lockdep_assert_held(&coredump->lock);
 	ss->snapshot_time = ktime_get_real();
 	ss->boot_time = ktime_get_boottime();
 
 	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
+	coredump->snapshot.ge = xe_guc_engine_snapshot_capture(e);
 }
 
 /**
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index 1e44d5346364..e055b266af70 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -27,6 +27,8 @@ struct xe_devcoredump_snapshot {
 	/* GuC snapshots */
 	/** @ct_snapshot: GuC CT snapshot */
 	struct xe_guc_ct_snapshot *ct;
+	/** @ge: Guc Engine snapshot */
+	struct xe_guc_submit_engine_snapshot *ge;
 };
 
 /**
-- 
2.39.2

[PATCH 09/14] drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.

[Rodrigo Vivi]

Let's start to move our existent logs to devcoredump one by
one. Any format change should come on follow-up work.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 7 ++++++-
 drivers/gpu/drm/xe/xe_devcoredump_types.h | 2 ++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index 795581c58d90..0e7ec654a9f2 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -12,6 +12,7 @@
 #include "xe_engine.h"
 #include "xe_gt.h"
 #include "xe_guc_ct.h"
+#include "xe_guc_submit.h"
 
 /**
  * DOC: Xe device coredump
@@ -89,6 +90,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 
 	drm_printf(&p, "\n**** GuC CT ****\n");
 	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
+	xe_guc_engine_snapshot_print(coredump->snapshot.ge, &p);
 
 	mutex_unlock(&coredump->lock);
 
@@ -106,6 +108,7 @@ static void xe_devcoredump_free(void *data)
 	mutex_lock(&coredump->lock);
 
 	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
+	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
 
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
@@ -117,13 +120,15 @@ static void xe_devcoredump_free(void *data)
 static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 {
 	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
-	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
+	struct xe_engine *e = coredump->faulty_engine;
+	struct xe_guc *guc = engine_to_guc(e);
 
 	lockdep_assert_held(&coredump->lock);
 	ss->snapshot_time = ktime_get_real();
 	ss->boot_time = ktime_get_boottime();
 
 	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
+	coredump->snapshot.ge = xe_guc_engine_snapshot_capture(e);
 }
 
 /**
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index 1e44d5346364..e055b266af70 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -27,6 +27,8 @@ struct xe_devcoredump_snapshot {
 	/* GuC snapshots */
 	/** @ct_snapshot: GuC CT snapshot */
 	struct xe_guc_ct_snapshot *ct;
+	/** @ge: Guc Engine snapshot */
+	struct xe_guc_submit_engine_snapshot *ge;
 };
 
 /**
-- 
2.39.2

Re: [PATCH 09/14] drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:08PM -0400, Rodrigo Vivi wrote:
> Let's start to move our existent logs to devcoredump one by
> one. Any format change should come on follow-up work.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c       | 7 ++++++-
>  drivers/gpu/drm/xe/xe_devcoredump_types.h | 2 ++
>  2 files changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index 795581c58d90..0e7ec654a9f2 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -12,6 +12,7 @@
>  #include "xe_engine.h"
>  #include "xe_gt.h"
>  #include "xe_guc_ct.h"
> +#include "xe_guc_submit.h"
>  
>  /**
>   * DOC: Xe device coredump
> @@ -89,6 +90,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  
>  	drm_printf(&p, "\n**** GuC CT ****\n");
>  	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
> +	xe_guc_engine_snapshot_print(coredump->snapshot.ge, &p);
>  
>  	mutex_unlock(&coredump->lock);
>  
> @@ -106,6 +108,7 @@ static void xe_devcoredump_free(void *data)
>  	mutex_lock(&coredump->lock);
>  
>  	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
> +	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
>  
>  	coredump->faulty_engine = NULL;
>  	drm_info(&coredump_to_xe(coredump)->drm,
> @@ -117,13 +120,15 @@ static void xe_devcoredump_free(void *data)
>  static void devcoredump_snapshot(struct xe_devcoredump *coredump)
>  {
>  	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> -	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
> +	struct xe_engine *e = coredump->faulty_engine;
> +	struct xe_guc *guc = engine_to_guc(e);
>  
>  	lockdep_assert_held(&coredump->lock);
>  	ss->snapshot_time = ktime_get_real();
>  	ss->boot_time = ktime_get_boottime();
>  
>  	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
> +	coredump->snapshot.ge = xe_guc_engine_snapshot_capture(e);
>  }
>  
>  /**
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> index 1e44d5346364..e055b266af70 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
> +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> @@ -27,6 +27,8 @@ struct xe_devcoredump_snapshot {
>  	/* GuC snapshots */
>  	/** @ct_snapshot: GuC CT snapshot */
>  	struct xe_guc_ct_snapshot *ct;
> +	/** @ge: Guc Engine snapshot */
> +	struct xe_guc_submit_engine_snapshot *ge;
>  };
>  
>  /**
> -- 
> 2.39.2
> 

Re: [Intel-xe] [PATCH 09/14] drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:08PM -0400, Rodrigo Vivi wrote:
> Let's start to move our existent logs to devcoredump one by
> one. Any format change should come on follow-up work.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c       | 7 ++++++-
>  drivers/gpu/drm/xe/xe_devcoredump_types.h | 2 ++
>  2 files changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index 795581c58d90..0e7ec654a9f2 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -12,6 +12,7 @@
>  #include "xe_engine.h"
>  #include "xe_gt.h"
>  #include "xe_guc_ct.h"
> +#include "xe_guc_submit.h"
>  
>  /**
>   * DOC: Xe device coredump
> @@ -89,6 +90,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  
>  	drm_printf(&p, "\n**** GuC CT ****\n");
>  	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
> +	xe_guc_engine_snapshot_print(coredump->snapshot.ge, &p);
>  
>  	mutex_unlock(&coredump->lock);
>  
> @@ -106,6 +108,7 @@ static void xe_devcoredump_free(void *data)
>  	mutex_lock(&coredump->lock);
>  
>  	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
> +	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
>  
>  	coredump->faulty_engine = NULL;
>  	drm_info(&coredump_to_xe(coredump)->drm,
> @@ -117,13 +120,15 @@ static void xe_devcoredump_free(void *data)
>  static void devcoredump_snapshot(struct xe_devcoredump *coredump)
>  {
>  	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
> -	struct xe_guc *guc = engine_to_guc(coredump->faulty_engine);
> +	struct xe_engine *e = coredump->faulty_engine;
> +	struct xe_guc *guc = engine_to_guc(e);
>  
>  	lockdep_assert_held(&coredump->lock);
>  	ss->snapshot_time = ktime_get_real();
>  	ss->boot_time = ktime_get_boottime();
>  
>  	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
> +	coredump->snapshot.ge = xe_guc_engine_snapshot_capture(e);
>  }
>  
>  /**
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> index 1e44d5346364..e055b266af70 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
> +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> @@ -27,6 +27,8 @@ struct xe_devcoredump_snapshot {
>  	/* GuC snapshots */
>  	/** @ct_snapshot: GuC CT snapshot */
>  	struct xe_guc_ct_snapshot *ct;
> +	/** @ge: Guc Engine snapshot */
> +	struct xe_guc_submit_engine_snapshot *ge;
>  };
>  
>  /**
> -- 
> 2.39.2
> 

[Intel-xe] [PATCH 10/14] drm/xe: Convert Xe HW Engine print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_gt_debugfs.c      |   2 +-
 drivers/gpu/drm/xe/xe_guc_submit.c      |   2 +-
 drivers/gpu/drm/xe/xe_hw_engine.c       | 210 +++++++++++++++++-------
 drivers/gpu/drm/xe/xe_hw_engine.h       |   8 +-
 drivers/gpu/drm/xe/xe_hw_engine_types.h |  78 +++++++++
 5 files changed, 241 insertions(+), 59 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_gt_debugfs.c b/drivers/gpu/drm/xe/xe_gt_debugfs.c
index c45486c2015a..8bf441e850a0 100644
--- a/drivers/gpu/drm/xe/xe_gt_debugfs.c
+++ b/drivers/gpu/drm/xe/xe_gt_debugfs.c
@@ -42,7 +42,7 @@ static int hw_engines(struct seq_file *m, void *data)
 	}
 
 	for_each_hw_engine(hwe, gt, id)
-		xe_hw_engine_print_state(hwe, &p);
+		xe_hw_engine_print(hwe, &p);
 
 	xe_device_mem_access_put(xe);
 	err = xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL);
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index 9c06411f857f..74659d0a69b3 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -751,7 +751,7 @@ static void simple_error_capture(struct xe_engine *e)
 			if (hwe->class != e->hwe->class ||
 			    !(BIT(hwe->logical_instance) & adj_logical_mask))
 				continue;
-			xe_hw_engine_print_state(hwe, &p);
+			xe_hw_engine_print(hwe, &p);
 		}
 		xe_analyze_vm(&p, e->vm, e->gt->info.id);
 		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
index 23b9f120c258..eda0666bfa2f 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.c
+++ b/drivers/gpu/drm/xe/xe_hw_engine.c
@@ -505,77 +505,175 @@ void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec)
 		xe_hw_fence_irq_run(hwe->fence_irq);
 }
 
-void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p)
+/**
+ * xe_hw_engine_snapshot_capture - Take a quick snapshot of the HW Engine.
+ * @hwe: Xe HW Engine.
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a Xe HW Engine snapshot object that must be freed by the
+ * 	    caller, using `xe_hw_engine_snapshot_free`.
+ */
+struct xe_hw_engine_snapshot *
+xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe)
 {
+	struct xe_hw_engine_snapshot *snapshot;
+	int len;
+
 	if (!xe_hw_engine_is_valid(hwe))
-		return;
+		return NULL;
+
+	snapshot = kzalloc(sizeof(struct xe_hw_engine_snapshot), GFP_ATOMIC);
+
+	len = strlen(hwe->name) + 1;
+	snapshot->name = kzalloc(len, GFP_ATOMIC);
+	strscpy(snapshot->name, hwe->name, len);
+	snapshot->class = hwe->class;
+	snapshot->logical_instance = hwe->logical_instance;
+	snapshot->forcewake.domain = hwe->domain;
+	snapshot->forcewake.ref = xe_force_wake_ref(gt_to_fw(hwe->gt),
+						    hwe->domain);
+	snapshot->mmio_base = hwe->mmio_base;
+
+	snapshot->reg.ring_hwstam = hw_engine_mmio_read32(hwe,
+							  RING_HWSTAM(0).reg);
+	snapshot->reg.ring_hws_pga = hw_engine_mmio_read32(hwe,
+							   RING_HWS_PGA(0).reg);
+	snapshot->reg.ring_execlist_status_lo =
+		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_LO(0).reg);
+	snapshot->reg.ring_execlist_status_hi =
+		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg);
+	snapshot->reg.ring_execlist_sq_contents_lo =
+		hw_engine_mmio_read32(hwe,
+				      RING_EXECLIST_SQ_CONTENTS_LO(0).reg);
+	snapshot->reg.ring_execlist_sq_contents_hi =
+		hw_engine_mmio_read32(hwe,
+				      RING_EXECLIST_SQ_CONTENTS_HI(0).reg);
+	snapshot->reg.ring_execlist_control =
+		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg);
+	snapshot->reg.ring_start = hw_engine_mmio_read32(hwe,
+							 RING_START(0).reg);
+	snapshot->reg.ring_head =
+		hw_engine_mmio_read32(hwe, RING_HEAD(0).reg) & HEAD_ADDR;
+	snapshot->reg.ring_tail =
+		hw_engine_mmio_read32(hwe, RING_TAIL(0).reg) & TAIL_ADDR;
+	snapshot->reg.ring_ctl = hw_engine_mmio_read32(hwe, RING_CTL(0).reg);
+	snapshot->reg.ring_mi_mode =
+		hw_engine_mmio_read32(hwe, RING_MI_MODE(0).reg);
+	snapshot->reg.ring_mode_gen7 =
+		hw_engine_mmio_read32(hwe, RING_MODE_GEN7(0).reg);
+	snapshot->reg.ring_imr = hw_engine_mmio_read32(hwe, RING_IMR(0).reg);
+	snapshot->reg.ring_esr = hw_engine_mmio_read32(hwe, RING_ESR(0).reg);
+	snapshot->reg.ring_emr = hw_engine_mmio_read32(hwe, RING_EMR(0).reg);
+	snapshot->reg.ring_eir = hw_engine_mmio_read32(hwe, RING_EIR(0).reg);
+	snapshot->reg.ring_acthd_udw =
+		hw_engine_mmio_read32(hwe, RING_ACTHD_UDW(0).reg);
+	snapshot->reg.ring_acthd = hw_engine_mmio_read32(hwe,
+							 RING_ACTHD(0).reg);
+	snapshot->reg.ring_bbaddr_udw =
+		hw_engine_mmio_read32(hwe, RING_BBADDR_UDW(0).reg);
+	snapshot->reg.ring_bbaddr = hw_engine_mmio_read32(hwe, RING_BBADDR(0).reg);
+	snapshot->reg.ring_dma_fadd_udw =
+		hw_engine_mmio_read32(hwe, RING_DMA_FADD_UDW(0).reg),
+	snapshot->reg.ring_dma_fadd =
+		hw_engine_mmio_read32(hwe, RING_DMA_FADD(0).reg);
+	snapshot->reg.ipeir = hw_engine_mmio_read32(hwe, IPEIR(0).reg);
+	snapshot->reg.ipehr = hw_engine_mmio_read32(hwe, IPEHR(0).reg);
 
-	drm_printf(p, "%s (physical), logical instance=%d\n", hwe->name,
-		hwe->logical_instance);
-	drm_printf(p, "\tForcewake: domain 0x%x, ref %d\n",
-		hwe->domain,
-		xe_force_wake_ref(gt_to_fw(hwe->gt), hwe->domain));
-	drm_printf(p, "\tMMIO base: 0x%08x\n", hwe->mmio_base);
+	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
+		snapshot->reg.rcu_mode = xe_mmio_read32(hwe->gt,
+							GEN12_RCU_MODE.reg);
 
-	drm_printf(p, "\tHWSTAM: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_HWSTAM(0).reg));
-	drm_printf(p, "\tRING_HWS_PGA: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_HWS_PGA(0).reg));
+	return snapshot;
+}
 
+/**
+ * xe_hw_engine_snapshot_print - Print out a given Xe HW Engine snapshot.
+ * @snapshot: Xe HW Engine snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given Xe HW Engine snapshot object.
+ */
+void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot,
+				 struct drm_printer *p)
+{
+	if (!snapshot)
+		return;
+
+	drm_printf(p, "%s (physical), logical instance=%d\n", snapshot->name,
+		snapshot->logical_instance);
+	drm_printf(p, "\tForcewake: domain 0x%x, ref %d\n",
+		snapshot->forcewake.domain, snapshot->forcewake.ref);
+	drm_printf(p, "\tMMIO base: 0x%08x\n", snapshot->mmio_base);
+	drm_printf(p, "\tHWSTAM: 0x%08x\n", snapshot->reg.ring_hwstam);
+	drm_printf(p, "\tRING_HWS_PGA: 0x%08x\n", snapshot->reg.ring_hws_pga);
 	drm_printf(p, "\tRING_EXECLIST_STATUS_LO: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_LO(0).reg));
+		   snapshot->reg.ring_execlist_status_lo);
 	drm_printf(p, "\tRING_EXECLIST_STATUS_HI: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg));
+		   snapshot->reg.ring_execlist_status_hi);
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_LO: 0x%08x\n",
-		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS_LO(0).reg));
+		   snapshot->reg.ring_execlist_sq_contents_lo);
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_HI: 0x%08x\n",
-		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS_HI(0).reg));
+		   snapshot->reg.ring_execlist_sq_contents_hi);
 	drm_printf(p, "\tRING_EXECLIST_CONTROL: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg));
-
-	drm_printf(p, "\tRING_START: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_START(0).reg));
-	drm_printf(p, "\tRING_HEAD:  0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_HEAD(0).reg) & HEAD_ADDR);
-	drm_printf(p, "\tRING_TAIL:  0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_TAIL(0).reg) & TAIL_ADDR);
-	drm_printf(p, "\tRING_CTL: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_CTL(0).reg));
-	drm_printf(p, "\tRING_MODE: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_MI_MODE(0).reg));
+		   snapshot->reg.ring_execlist_control);
+	drm_printf(p, "\tRING_START: 0x%08x\n", snapshot->reg.ring_start);
+	drm_printf(p, "\tRING_HEAD:  0x%08x\n", snapshot->reg.ring_head);
+	drm_printf(p, "\tRING_TAIL:  0x%08x\n", snapshot->reg.ring_tail);
+	drm_printf(p, "\tRING_CTL: 0x%08x\n", snapshot->reg.ring_ctl);
+	drm_printf(p, "\tRING_MODE: 0x%08x\n", snapshot->reg.ring_mi_mode);
 	drm_printf(p, "\tRING_MODE_GEN7: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_MODE_GEN7(0).reg));
-
-	drm_printf(p, "\tRING_IMR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_IMR(0).reg));
-	drm_printf(p, "\tRING_ESR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_ESR(0).reg));
-	drm_printf(p, "\tRING_EMR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EMR(0).reg));
-	drm_printf(p, "\tRING_EIR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EIR(0).reg));
-
-        drm_printf(p, "\tACTHD:  0x%08x_%08x\n",
-		hw_engine_mmio_read32(hwe, RING_ACTHD_UDW(0).reg),
-		hw_engine_mmio_read32(hwe, RING_ACTHD(0).reg));
-        drm_printf(p, "\tBBADDR: 0x%08x_%08x\n",
-		hw_engine_mmio_read32(hwe, RING_BBADDR_UDW(0).reg),
-		hw_engine_mmio_read32(hwe, RING_BBADDR(0).reg));
+		   snapshot->reg.ring_mode_gen7);
+	drm_printf(p, "\tRING_IMR:   0x%08x\n", snapshot->reg.ring_imr);
+	drm_printf(p, "\tRING_ESR:   0x%08x\n", snapshot->reg.ring_esr);
+	drm_printf(p, "\tRING_EMR:   0x%08x\n", snapshot->reg.ring_emr);
+	drm_printf(p, "\tRING_EIR:   0x%08x\n", snapshot->reg.ring_eir);
+        drm_printf(p, "\tACTHD:  0x%08x_%08x\n", snapshot->reg.ring_acthd_udw,
+		   snapshot->reg.ring_acthd);
+        drm_printf(p, "\tBBADDR: 0x%08x_%08x\n", snapshot->reg.ring_bbaddr_udw,
+		   snapshot->reg.ring_bbaddr);
         drm_printf(p, "\tDMA_FADDR: 0x%08x_%08x\n",
-		hw_engine_mmio_read32(hwe, RING_DMA_FADD_UDW(0).reg),
-		hw_engine_mmio_read32(hwe, RING_DMA_FADD(0).reg));
+		   snapshot->reg.ring_dma_fadd_udw,
+		   snapshot->reg.ring_dma_fadd);
+	drm_printf(p, "\tIPEIR: 0x%08x\n", snapshot->reg.ipeir);
+	drm_printf(p, "\tIPEHR: 0x%08x\n\n", snapshot->reg.ipehr);
 
-	drm_printf(p, "\tIPEIR: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, IPEIR(0).reg));
-	drm_printf(p, "\tIPEHR: 0x%08x\n\n",
-		hw_engine_mmio_read32(hwe, IPEHR(0).reg));
-
-	if (hwe->class == XE_ENGINE_CLASS_COMPUTE)
+	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
 		drm_printf(p, "\tGEN12_RCU_MODE: 0x%08x\n",
-			xe_mmio_read32(hwe->gt, GEN12_RCU_MODE.reg));
+			   snapshot->reg.rcu_mode);
+}
+
+/**
+ * xe_hw_engine_snapshot_free - Free all allocated objects for a given snapshot.
+ * @snapshot: Xe HW Engine snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot)
+{
+	if (!snapshot)
+		return;
+
+	kfree(snapshot->name);
+	kfree(snapshot);
+}
+
+/**
+ * xe_hw_engine_print - Xe HW Engine Print.
+ * @hwe: Hardware Engine.
+ * @p: drm_printer.
+ *
+ * This function quickly capture a snapshot and immediately print it out.
+ */
+void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p)
+{
+	struct xe_hw_engine_snapshot *snapshot;
 
+	snapshot = xe_hw_engine_snapshot_capture(hwe);
+	xe_hw_engine_snapshot_print(snapshot, p);
+	xe_hw_engine_snapshot_free(snapshot);
 }
 
 u32 xe_hw_engine_mask_per_class(struct xe_gt *gt,
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h
index ceab65397256..a0514bf859c6 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.h
+++ b/drivers/gpu/drm/xe/xe_hw_engine.h
@@ -14,10 +14,16 @@ int xe_hw_engines_init_early(struct xe_gt *gt);
 int xe_hw_engines_init(struct xe_gt *gt);
 void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec);
 void xe_hw_engine_enable_ring(struct xe_hw_engine *hwe);
-void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p);
 u32 xe_hw_engine_mask_per_class(struct xe_gt *gt,
 				enum xe_engine_class engine_class);
 
+struct xe_hw_engine_snapshot *
+xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe);
+void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot);
+void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot,
+				 struct drm_printer *p);
+void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p);
+
 bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe);
 static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe)
 {
diff --git a/drivers/gpu/drm/xe/xe_hw_engine_types.h b/drivers/gpu/drm/xe/xe_hw_engine_types.h
index 2c40384957da..c15d6c671fbb 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine_types.h
+++ b/drivers/gpu/drm/xe/xe_hw_engine_types.h
@@ -109,4 +109,82 @@ struct xe_hw_engine {
 	enum xe_hw_engine_id engine_id;
 };
 
+/**
+ * struct xe_hw_engine_snapshot - Hardware engine snapshot
+ *
+ * Contains the snapshot of usefull hardware engine info and registers.
+ */
+struct xe_hw_engine_snapshot {
+	/** @name: name of the hw engine */
+	char *name;
+	/** @class: class of this hw engine */
+	enum xe_engine_class class;
+	/** @logical_instance: logical instance of this hw engine */
+	u16 logical_instance;
+	/** @forcewake: Force Wake information snapshot */
+	struct {
+		/** @domain: force wake domain of this hw engine */
+		enum xe_force_wake_domains domain;
+		/** @ref: Forcewake ref for the above domain */
+		int ref;
+	} forcewake;
+	/** @reg: Useful MMIO register snapshot */
+	/** @mmio_base: MMIO base address of this hw engine*/
+	u32 mmio_base;
+	struct {
+		/** @ring_hwstam: RING_HWSTAM */
+		u32 ring_hwstam;
+		/** @ring_hws_pga: RING_HWS_PGA */
+		u32 ring_hws_pga;
+		/** @ring_execlist_status_lo: RING_EXECLIST_STATUS_LO */
+		u32 ring_execlist_status_lo;
+		/** @ring_execlist_status_hi: RING_EXECLIST_STATUS_HI */
+		u32 ring_execlist_status_hi;
+		/** @ring_execlist_sq_contents_lo: RING_EXECLIST_SQ_CONTENTS */
+		u32 ring_execlist_sq_contents_lo;
+		/** @ring_execlist_sq_contents_hi: RING_EXECLIST_SQ_CONTENTS + 4 */
+		u32 ring_execlist_sq_contents_hi;
+		/** @ring_execlist_control: RING_EXECLIST_CONTROL */
+		u32 ring_execlist_control;
+		/** @ring_start: RING_START */
+		u32 ring_start;
+		/** @ring_head: RING_HEAD */
+		u32 ring_head;
+		/** @ring_tail: RING_TAIL */
+		u32 ring_tail;
+		/** @ring_ctl: RING_CTL */
+		u32 ring_ctl;
+		/** @ring_mi_mode: RING_MI_MODE */
+		u32 ring_mi_mode;
+		/** @ring_mode_gen7: RING_MODE_GEN7 */
+		u32 ring_mode_gen7;
+		/** @ring_imr: RING_IMR */
+		u32 ring_imr;
+		/** @ring_esr: RING_ESR */
+		u32 ring_esr;
+		/** @ring_emr: RING_EMR */
+		u32 ring_emr;
+		/** @ring_eir: RING_EIR */
+		u32 ring_eir;
+		/** @ring_acthd_udw: RING_ACTHD_UDW */
+		u32 ring_acthd_udw;
+		/** @ring_acthd: RING_ACTHD */
+		u32 ring_acthd;
+		/** @ring_bbaddr_udw: RING_BBADDR_UDW */
+		u32 ring_bbaddr_udw;
+		/** @ring_bbaddr: RING_BBADDR */
+		u32 ring_bbaddr;
+		/** @ring_dma_fadd_udw: RING_DMA_FADD_UDW */
+		u32 ring_dma_fadd_udw;
+		/** @ring_dma_fadd: RING_DMA_FADD */
+		u32 ring_dma_fadd;
+		/** @ipeir: IPEIR */
+		u32 ipeir;
+		/** @ipehr: IPEHR */
+		u32 ipehr;
+		/** @rcu_mode: GEN12_RCU_MODE */
+		u32 rcu_mode;
+	} reg;
+};
+
 #endif
-- 
2.39.2

[PATCH 10/14] drm/xe: Convert Xe HW Engine print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_gt_debugfs.c      |   2 +-
 drivers/gpu/drm/xe/xe_guc_submit.c      |   2 +-
 drivers/gpu/drm/xe/xe_hw_engine.c       | 210 +++++++++++++++++-------
 drivers/gpu/drm/xe/xe_hw_engine.h       |   8 +-
 drivers/gpu/drm/xe/xe_hw_engine_types.h |  78 +++++++++
 5 files changed, 241 insertions(+), 59 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_gt_debugfs.c b/drivers/gpu/drm/xe/xe_gt_debugfs.c
index c45486c2015a..8bf441e850a0 100644
--- a/drivers/gpu/drm/xe/xe_gt_debugfs.c
+++ b/drivers/gpu/drm/xe/xe_gt_debugfs.c
@@ -42,7 +42,7 @@ static int hw_engines(struct seq_file *m, void *data)
 	}
 
 	for_each_hw_engine(hwe, gt, id)
-		xe_hw_engine_print_state(hwe, &p);
+		xe_hw_engine_print(hwe, &p);
 
 	xe_device_mem_access_put(xe);
 	err = xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL);
diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index 9c06411f857f..74659d0a69b3 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -751,7 +751,7 @@ static void simple_error_capture(struct xe_engine *e)
 			if (hwe->class != e->hwe->class ||
 			    !(BIT(hwe->logical_instance) & adj_logical_mask))
 				continue;
-			xe_hw_engine_print_state(hwe, &p);
+			xe_hw_engine_print(hwe, &p);
 		}
 		xe_analyze_vm(&p, e->vm, e->gt->info.id);
 		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
index 23b9f120c258..eda0666bfa2f 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.c
+++ b/drivers/gpu/drm/xe/xe_hw_engine.c
@@ -505,77 +505,175 @@ void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec)
 		xe_hw_fence_irq_run(hwe->fence_irq);
 }
 
-void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p)
+/**
+ * xe_hw_engine_snapshot_capture - Take a quick snapshot of the HW Engine.
+ * @hwe: Xe HW Engine.
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a Xe HW Engine snapshot object that must be freed by the
+ * 	    caller, using `xe_hw_engine_snapshot_free`.
+ */
+struct xe_hw_engine_snapshot *
+xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe)
 {
+	struct xe_hw_engine_snapshot *snapshot;
+	int len;
+
 	if (!xe_hw_engine_is_valid(hwe))
-		return;
+		return NULL;
+
+	snapshot = kzalloc(sizeof(struct xe_hw_engine_snapshot), GFP_ATOMIC);
+
+	len = strlen(hwe->name) + 1;
+	snapshot->name = kzalloc(len, GFP_ATOMIC);
+	strscpy(snapshot->name, hwe->name, len);
+	snapshot->class = hwe->class;
+	snapshot->logical_instance = hwe->logical_instance;
+	snapshot->forcewake.domain = hwe->domain;
+	snapshot->forcewake.ref = xe_force_wake_ref(gt_to_fw(hwe->gt),
+						    hwe->domain);
+	snapshot->mmio_base = hwe->mmio_base;
+
+	snapshot->reg.ring_hwstam = hw_engine_mmio_read32(hwe,
+							  RING_HWSTAM(0).reg);
+	snapshot->reg.ring_hws_pga = hw_engine_mmio_read32(hwe,
+							   RING_HWS_PGA(0).reg);
+	snapshot->reg.ring_execlist_status_lo =
+		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_LO(0).reg);
+	snapshot->reg.ring_execlist_status_hi =
+		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg);
+	snapshot->reg.ring_execlist_sq_contents_lo =
+		hw_engine_mmio_read32(hwe,
+				      RING_EXECLIST_SQ_CONTENTS_LO(0).reg);
+	snapshot->reg.ring_execlist_sq_contents_hi =
+		hw_engine_mmio_read32(hwe,
+				      RING_EXECLIST_SQ_CONTENTS_HI(0).reg);
+	snapshot->reg.ring_execlist_control =
+		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg);
+	snapshot->reg.ring_start = hw_engine_mmio_read32(hwe,
+							 RING_START(0).reg);
+	snapshot->reg.ring_head =
+		hw_engine_mmio_read32(hwe, RING_HEAD(0).reg) & HEAD_ADDR;
+	snapshot->reg.ring_tail =
+		hw_engine_mmio_read32(hwe, RING_TAIL(0).reg) & TAIL_ADDR;
+	snapshot->reg.ring_ctl = hw_engine_mmio_read32(hwe, RING_CTL(0).reg);
+	snapshot->reg.ring_mi_mode =
+		hw_engine_mmio_read32(hwe, RING_MI_MODE(0).reg);
+	snapshot->reg.ring_mode_gen7 =
+		hw_engine_mmio_read32(hwe, RING_MODE_GEN7(0).reg);
+	snapshot->reg.ring_imr = hw_engine_mmio_read32(hwe, RING_IMR(0).reg);
+	snapshot->reg.ring_esr = hw_engine_mmio_read32(hwe, RING_ESR(0).reg);
+	snapshot->reg.ring_emr = hw_engine_mmio_read32(hwe, RING_EMR(0).reg);
+	snapshot->reg.ring_eir = hw_engine_mmio_read32(hwe, RING_EIR(0).reg);
+	snapshot->reg.ring_acthd_udw =
+		hw_engine_mmio_read32(hwe, RING_ACTHD_UDW(0).reg);
+	snapshot->reg.ring_acthd = hw_engine_mmio_read32(hwe,
+							 RING_ACTHD(0).reg);
+	snapshot->reg.ring_bbaddr_udw =
+		hw_engine_mmio_read32(hwe, RING_BBADDR_UDW(0).reg);
+	snapshot->reg.ring_bbaddr = hw_engine_mmio_read32(hwe, RING_BBADDR(0).reg);
+	snapshot->reg.ring_dma_fadd_udw =
+		hw_engine_mmio_read32(hwe, RING_DMA_FADD_UDW(0).reg),
+	snapshot->reg.ring_dma_fadd =
+		hw_engine_mmio_read32(hwe, RING_DMA_FADD(0).reg);
+	snapshot->reg.ipeir = hw_engine_mmio_read32(hwe, IPEIR(0).reg);
+	snapshot->reg.ipehr = hw_engine_mmio_read32(hwe, IPEHR(0).reg);
 
-	drm_printf(p, "%s (physical), logical instance=%d\n", hwe->name,
-		hwe->logical_instance);
-	drm_printf(p, "\tForcewake: domain 0x%x, ref %d\n",
-		hwe->domain,
-		xe_force_wake_ref(gt_to_fw(hwe->gt), hwe->domain));
-	drm_printf(p, "\tMMIO base: 0x%08x\n", hwe->mmio_base);
+	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
+		snapshot->reg.rcu_mode = xe_mmio_read32(hwe->gt,
+							GEN12_RCU_MODE.reg);
 
-	drm_printf(p, "\tHWSTAM: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_HWSTAM(0).reg));
-	drm_printf(p, "\tRING_HWS_PGA: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_HWS_PGA(0).reg));
+	return snapshot;
+}
 
+/**
+ * xe_hw_engine_snapshot_print - Print out a given Xe HW Engine snapshot.
+ * @snapshot: Xe HW Engine snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given Xe HW Engine snapshot object.
+ */
+void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot,
+				 struct drm_printer *p)
+{
+	if (!snapshot)
+		return;
+
+	drm_printf(p, "%s (physical), logical instance=%d\n", snapshot->name,
+		snapshot->logical_instance);
+	drm_printf(p, "\tForcewake: domain 0x%x, ref %d\n",
+		snapshot->forcewake.domain, snapshot->forcewake.ref);
+	drm_printf(p, "\tMMIO base: 0x%08x\n", snapshot->mmio_base);
+	drm_printf(p, "\tHWSTAM: 0x%08x\n", snapshot->reg.ring_hwstam);
+	drm_printf(p, "\tRING_HWS_PGA: 0x%08x\n", snapshot->reg.ring_hws_pga);
 	drm_printf(p, "\tRING_EXECLIST_STATUS_LO: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_LO(0).reg));
+		   snapshot->reg.ring_execlist_status_lo);
 	drm_printf(p, "\tRING_EXECLIST_STATUS_HI: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg));
+		   snapshot->reg.ring_execlist_status_hi);
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_LO: 0x%08x\n",
-		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS_LO(0).reg));
+		   snapshot->reg.ring_execlist_sq_contents_lo);
 	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_HI: 0x%08x\n",
-		hw_engine_mmio_read32(hwe,
-					 RING_EXECLIST_SQ_CONTENTS_HI(0).reg));
+		   snapshot->reg.ring_execlist_sq_contents_hi);
 	drm_printf(p, "\tRING_EXECLIST_CONTROL: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg));
-
-	drm_printf(p, "\tRING_START: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_START(0).reg));
-	drm_printf(p, "\tRING_HEAD:  0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_HEAD(0).reg) & HEAD_ADDR);
-	drm_printf(p, "\tRING_TAIL:  0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_TAIL(0).reg) & TAIL_ADDR);
-	drm_printf(p, "\tRING_CTL: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_CTL(0).reg));
-	drm_printf(p, "\tRING_MODE: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_MI_MODE(0).reg));
+		   snapshot->reg.ring_execlist_control);
+	drm_printf(p, "\tRING_START: 0x%08x\n", snapshot->reg.ring_start);
+	drm_printf(p, "\tRING_HEAD:  0x%08x\n", snapshot->reg.ring_head);
+	drm_printf(p, "\tRING_TAIL:  0x%08x\n", snapshot->reg.ring_tail);
+	drm_printf(p, "\tRING_CTL: 0x%08x\n", snapshot->reg.ring_ctl);
+	drm_printf(p, "\tRING_MODE: 0x%08x\n", snapshot->reg.ring_mi_mode);
 	drm_printf(p, "\tRING_MODE_GEN7: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_MODE_GEN7(0).reg));
-
-	drm_printf(p, "\tRING_IMR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_IMR(0).reg));
-	drm_printf(p, "\tRING_ESR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_ESR(0).reg));
-	drm_printf(p, "\tRING_EMR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EMR(0).reg));
-	drm_printf(p, "\tRING_EIR:   0x%08x\n",
-		hw_engine_mmio_read32(hwe, RING_EIR(0).reg));
-
-        drm_printf(p, "\tACTHD:  0x%08x_%08x\n",
-		hw_engine_mmio_read32(hwe, RING_ACTHD_UDW(0).reg),
-		hw_engine_mmio_read32(hwe, RING_ACTHD(0).reg));
-        drm_printf(p, "\tBBADDR: 0x%08x_%08x\n",
-		hw_engine_mmio_read32(hwe, RING_BBADDR_UDW(0).reg),
-		hw_engine_mmio_read32(hwe, RING_BBADDR(0).reg));
+		   snapshot->reg.ring_mode_gen7);
+	drm_printf(p, "\tRING_IMR:   0x%08x\n", snapshot->reg.ring_imr);
+	drm_printf(p, "\tRING_ESR:   0x%08x\n", snapshot->reg.ring_esr);
+	drm_printf(p, "\tRING_EMR:   0x%08x\n", snapshot->reg.ring_emr);
+	drm_printf(p, "\tRING_EIR:   0x%08x\n", snapshot->reg.ring_eir);
+        drm_printf(p, "\tACTHD:  0x%08x_%08x\n", snapshot->reg.ring_acthd_udw,
+		   snapshot->reg.ring_acthd);
+        drm_printf(p, "\tBBADDR: 0x%08x_%08x\n", snapshot->reg.ring_bbaddr_udw,
+		   snapshot->reg.ring_bbaddr);
         drm_printf(p, "\tDMA_FADDR: 0x%08x_%08x\n",
-		hw_engine_mmio_read32(hwe, RING_DMA_FADD_UDW(0).reg),
-		hw_engine_mmio_read32(hwe, RING_DMA_FADD(0).reg));
+		   snapshot->reg.ring_dma_fadd_udw,
+		   snapshot->reg.ring_dma_fadd);
+	drm_printf(p, "\tIPEIR: 0x%08x\n", snapshot->reg.ipeir);
+	drm_printf(p, "\tIPEHR: 0x%08x\n\n", snapshot->reg.ipehr);
 
-	drm_printf(p, "\tIPEIR: 0x%08x\n",
-		hw_engine_mmio_read32(hwe, IPEIR(0).reg));
-	drm_printf(p, "\tIPEHR: 0x%08x\n\n",
-		hw_engine_mmio_read32(hwe, IPEHR(0).reg));
-
-	if (hwe->class == XE_ENGINE_CLASS_COMPUTE)
+	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
 		drm_printf(p, "\tGEN12_RCU_MODE: 0x%08x\n",
-			xe_mmio_read32(hwe->gt, GEN12_RCU_MODE.reg));
+			   snapshot->reg.rcu_mode);
+}
+
+/**
+ * xe_hw_engine_snapshot_free - Free all allocated objects for a given snapshot.
+ * @snapshot: Xe HW Engine snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot)
+{
+	if (!snapshot)
+		return;
+
+	kfree(snapshot->name);
+	kfree(snapshot);
+}
+
+/**
+ * xe_hw_engine_print - Xe HW Engine Print.
+ * @hwe: Hardware Engine.
+ * @p: drm_printer.
+ *
+ * This function quickly capture a snapshot and immediately print it out.
+ */
+void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p)
+{
+	struct xe_hw_engine_snapshot *snapshot;
 
+	snapshot = xe_hw_engine_snapshot_capture(hwe);
+	xe_hw_engine_snapshot_print(snapshot, p);
+	xe_hw_engine_snapshot_free(snapshot);
 }
 
 u32 xe_hw_engine_mask_per_class(struct xe_gt *gt,
diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h
index ceab65397256..a0514bf859c6 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine.h
+++ b/drivers/gpu/drm/xe/xe_hw_engine.h
@@ -14,10 +14,16 @@ int xe_hw_engines_init_early(struct xe_gt *gt);
 int xe_hw_engines_init(struct xe_gt *gt);
 void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec);
 void xe_hw_engine_enable_ring(struct xe_hw_engine *hwe);
-void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p);
 u32 xe_hw_engine_mask_per_class(struct xe_gt *gt,
 				enum xe_engine_class engine_class);
 
+struct xe_hw_engine_snapshot *
+xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe);
+void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot);
+void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot,
+				 struct drm_printer *p);
+void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p);
+
 bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe);
 static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe)
 {
diff --git a/drivers/gpu/drm/xe/xe_hw_engine_types.h b/drivers/gpu/drm/xe/xe_hw_engine_types.h
index 2c40384957da..c15d6c671fbb 100644
--- a/drivers/gpu/drm/xe/xe_hw_engine_types.h
+++ b/drivers/gpu/drm/xe/xe_hw_engine_types.h
@@ -109,4 +109,82 @@ struct xe_hw_engine {
 	enum xe_hw_engine_id engine_id;
 };
 
+/**
+ * struct xe_hw_engine_snapshot - Hardware engine snapshot
+ *
+ * Contains the snapshot of usefull hardware engine info and registers.
+ */
+struct xe_hw_engine_snapshot {
+	/** @name: name of the hw engine */
+	char *name;
+	/** @class: class of this hw engine */
+	enum xe_engine_class class;
+	/** @logical_instance: logical instance of this hw engine */
+	u16 logical_instance;
+	/** @forcewake: Force Wake information snapshot */
+	struct {
+		/** @domain: force wake domain of this hw engine */
+		enum xe_force_wake_domains domain;
+		/** @ref: Forcewake ref for the above domain */
+		int ref;
+	} forcewake;
+	/** @reg: Useful MMIO register snapshot */
+	/** @mmio_base: MMIO base address of this hw engine*/
+	u32 mmio_base;
+	struct {
+		/** @ring_hwstam: RING_HWSTAM */
+		u32 ring_hwstam;
+		/** @ring_hws_pga: RING_HWS_PGA */
+		u32 ring_hws_pga;
+		/** @ring_execlist_status_lo: RING_EXECLIST_STATUS_LO */
+		u32 ring_execlist_status_lo;
+		/** @ring_execlist_status_hi: RING_EXECLIST_STATUS_HI */
+		u32 ring_execlist_status_hi;
+		/** @ring_execlist_sq_contents_lo: RING_EXECLIST_SQ_CONTENTS */
+		u32 ring_execlist_sq_contents_lo;
+		/** @ring_execlist_sq_contents_hi: RING_EXECLIST_SQ_CONTENTS + 4 */
+		u32 ring_execlist_sq_contents_hi;
+		/** @ring_execlist_control: RING_EXECLIST_CONTROL */
+		u32 ring_execlist_control;
+		/** @ring_start: RING_START */
+		u32 ring_start;
+		/** @ring_head: RING_HEAD */
+		u32 ring_head;
+		/** @ring_tail: RING_TAIL */
+		u32 ring_tail;
+		/** @ring_ctl: RING_CTL */
+		u32 ring_ctl;
+		/** @ring_mi_mode: RING_MI_MODE */
+		u32 ring_mi_mode;
+		/** @ring_mode_gen7: RING_MODE_GEN7 */
+		u32 ring_mode_gen7;
+		/** @ring_imr: RING_IMR */
+		u32 ring_imr;
+		/** @ring_esr: RING_ESR */
+		u32 ring_esr;
+		/** @ring_emr: RING_EMR */
+		u32 ring_emr;
+		/** @ring_eir: RING_EIR */
+		u32 ring_eir;
+		/** @ring_acthd_udw: RING_ACTHD_UDW */
+		u32 ring_acthd_udw;
+		/** @ring_acthd: RING_ACTHD */
+		u32 ring_acthd;
+		/** @ring_bbaddr_udw: RING_BBADDR_UDW */
+		u32 ring_bbaddr_udw;
+		/** @ring_bbaddr: RING_BBADDR */
+		u32 ring_bbaddr;
+		/** @ring_dma_fadd_udw: RING_DMA_FADD_UDW */
+		u32 ring_dma_fadd_udw;
+		/** @ring_dma_fadd: RING_DMA_FADD */
+		u32 ring_dma_fadd;
+		/** @ipeir: IPEIR */
+		u32 ipeir;
+		/** @ipehr: IPEHR */
+		u32 ipehr;
+		/** @rcu_mode: GEN12_RCU_MODE */
+		u32 rcu_mode;
+	} reg;
+};
+
 #endif
-- 
2.39.2

Re: [Intel-xe] [PATCH 10/14] drm/xe: Convert Xe HW Engine print to snapshot capture and print.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:09PM -0400, Rodrigo Vivi wrote:
> The goal is to allow for a snapshot capture to be taken at the time
> of the crash, while the print out can happen at a later time through
> the exposed devcoredump virtual device.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_gt_debugfs.c      |   2 +-
>  drivers/gpu/drm/xe/xe_guc_submit.c      |   2 +-
>  drivers/gpu/drm/xe/xe_hw_engine.c       | 210 +++++++++++++++++-------
>  drivers/gpu/drm/xe/xe_hw_engine.h       |   8 +-
>  drivers/gpu/drm/xe/xe_hw_engine_types.h |  78 +++++++++
>  5 files changed, 241 insertions(+), 59 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_gt_debugfs.c b/drivers/gpu/drm/xe/xe_gt_debugfs.c
> index c45486c2015a..8bf441e850a0 100644
> --- a/drivers/gpu/drm/xe/xe_gt_debugfs.c
> +++ b/drivers/gpu/drm/xe/xe_gt_debugfs.c
> @@ -42,7 +42,7 @@ static int hw_engines(struct seq_file *m, void *data)
>  	}
>  
>  	for_each_hw_engine(hwe, gt, id)
> -		xe_hw_engine_print_state(hwe, &p);
> +		xe_hw_engine_print(hwe, &p);
>  
>  	xe_device_mem_access_put(xe);
>  	err = xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL);
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> index 9c06411f857f..74659d0a69b3 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> @@ -751,7 +751,7 @@ static void simple_error_capture(struct xe_engine *e)
>  			if (hwe->class != e->hwe->class ||
>  			    !(BIT(hwe->logical_instance) & adj_logical_mask))
>  				continue;
> -			xe_hw_engine_print_state(hwe, &p);
> +			xe_hw_engine_print(hwe, &p);
>  		}
>  		xe_analyze_vm(&p, e->vm, e->gt->info.id);
>  		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
> diff --git a/drivers/gpu/drm/xe/xe_hw_engine.c b/drivers/gpu/drm/xe/xe_hw_engine.c
> index 23b9f120c258..eda0666bfa2f 100644
> --- a/drivers/gpu/drm/xe/xe_hw_engine.c
> +++ b/drivers/gpu/drm/xe/xe_hw_engine.c
> @@ -505,77 +505,175 @@ void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec)
>  		xe_hw_fence_irq_run(hwe->fence_irq);
>  }
>  
> -void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p)
> +/**
> + * xe_hw_engine_snapshot_capture - Take a quick snapshot of the HW Engine.
> + * @hwe: Xe HW Engine.
> + *
> + * This can be printed out in a later stage like during dev_coredump
> + * analysis.
> + *
> + * Returns: a Xe HW Engine snapshot object that must be freed by the
> + * 	    caller, using `xe_hw_engine_snapshot_free`.
> + */
> +struct xe_hw_engine_snapshot *
> +xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe)
>  {
> +	struct xe_hw_engine_snapshot *snapshot;
> +	int len;
> +
>  	if (!xe_hw_engine_is_valid(hwe))
> -		return;
> +		return NULL;
> +
> +	snapshot = kzalloc(sizeof(struct xe_hw_engine_snapshot), GFP_ATOMIC);

Same alloc problem as previous files.

> +
> +	len = strlen(hwe->name) + 1;
> +	snapshot->name = kzalloc(len, GFP_ATOMIC);
> +	strscpy(snapshot->name, hwe->name, len);
> +	snapshot->class = hwe->class;
> +	snapshot->logical_instance = hwe->logical_instance;
> +	snapshot->forcewake.domain = hwe->domain;
> +	snapshot->forcewake.ref = xe_force_wake_ref(gt_to_fw(hwe->gt),
> +						    hwe->domain);
> +	snapshot->mmio_base = hwe->mmio_base;
> +
> +	snapshot->reg.ring_hwstam = hw_engine_mmio_read32(hwe,
> +							  RING_HWSTAM(0).reg);
> +	snapshot->reg.ring_hws_pga = hw_engine_mmio_read32(hwe,
> +							   RING_HWS_PGA(0).reg);
> +	snapshot->reg.ring_execlist_status_lo =
> +		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_LO(0).reg);
> +	snapshot->reg.ring_execlist_status_hi =
> +		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg);
> +	snapshot->reg.ring_execlist_sq_contents_lo =
> +		hw_engine_mmio_read32(hwe,
> +				      RING_EXECLIST_SQ_CONTENTS_LO(0).reg);
> +	snapshot->reg.ring_execlist_sq_contents_hi =
> +		hw_engine_mmio_read32(hwe,
> +				      RING_EXECLIST_SQ_CONTENTS_HI(0).reg);
> +	snapshot->reg.ring_execlist_control =
> +		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg);
> +	snapshot->reg.ring_start = hw_engine_mmio_read32(hwe,
> +							 RING_START(0).reg);
> +	snapshot->reg.ring_head =
> +		hw_engine_mmio_read32(hwe, RING_HEAD(0).reg) & HEAD_ADDR;
> +	snapshot->reg.ring_tail =
> +		hw_engine_mmio_read32(hwe, RING_TAIL(0).reg) & TAIL_ADDR;
> +	snapshot->reg.ring_ctl = hw_engine_mmio_read32(hwe, RING_CTL(0).reg);
> +	snapshot->reg.ring_mi_mode =
> +		hw_engine_mmio_read32(hwe, RING_MI_MODE(0).reg);
> +	snapshot->reg.ring_mode_gen7 =
> +		hw_engine_mmio_read32(hwe, RING_MODE_GEN7(0).reg);
> +	snapshot->reg.ring_imr = hw_engine_mmio_read32(hwe, RING_IMR(0).reg);
> +	snapshot->reg.ring_esr = hw_engine_mmio_read32(hwe, RING_ESR(0).reg);
> +	snapshot->reg.ring_emr = hw_engine_mmio_read32(hwe, RING_EMR(0).reg);
> +	snapshot->reg.ring_eir = hw_engine_mmio_read32(hwe, RING_EIR(0).reg);
> +	snapshot->reg.ring_acthd_udw =
> +		hw_engine_mmio_read32(hwe, RING_ACTHD_UDW(0).reg);
> +	snapshot->reg.ring_acthd = hw_engine_mmio_read32(hwe,
> +							 RING_ACTHD(0).reg);
> +	snapshot->reg.ring_bbaddr_udw =
> +		hw_engine_mmio_read32(hwe, RING_BBADDR_UDW(0).reg);
> +	snapshot->reg.ring_bbaddr = hw_engine_mmio_read32(hwe, RING_BBADDR(0).reg);
> +	snapshot->reg.ring_dma_fadd_udw =
> +		hw_engine_mmio_read32(hwe, RING_DMA_FADD_UDW(0).reg),
> +	snapshot->reg.ring_dma_fadd =
> +		hw_engine_mmio_read32(hwe, RING_DMA_FADD(0).reg);
> +	snapshot->reg.ipeir = hw_engine_mmio_read32(hwe, IPEIR(0).reg);
> +	snapshot->reg.ipehr = hw_engine_mmio_read32(hwe, IPEHR(0).reg);
>  
> -	drm_printf(p, "%s (physical), logical instance=%d\n", hwe->name,
> -		hwe->logical_instance);
> -	drm_printf(p, "\tForcewake: domain 0x%x, ref %d\n",
> -		hwe->domain,
> -		xe_force_wake_ref(gt_to_fw(hwe->gt), hwe->domain));
> -	drm_printf(p, "\tMMIO base: 0x%08x\n", hwe->mmio_base);
> +	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
> +		snapshot->reg.rcu_mode = xe_mmio_read32(hwe->gt,
> +							GEN12_RCU_MODE.reg);
>  
> -	drm_printf(p, "\tHWSTAM: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_HWSTAM(0).reg));
> -	drm_printf(p, "\tRING_HWS_PGA: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_HWS_PGA(0).reg));
> +	return snapshot;
> +}
>  
> +/**
> + * xe_hw_engine_snapshot_print - Print out a given Xe HW Engine snapshot.
> + * @snapshot: Xe HW Engine snapshot object.
> + * @p: drm_printer where it will be printed out.
> + *
> + * This function prints out a given Xe HW Engine snapshot object.
> + */
> +void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot,
> +				 struct drm_printer *p)
> +{
> +	if (!snapshot)
> +		return;
> +
> +	drm_printf(p, "%s (physical), logical instance=%d\n", snapshot->name,
> +		snapshot->logical_instance);
> +	drm_printf(p, "\tForcewake: domain 0x%x, ref %d\n",
> +		snapshot->forcewake.domain, snapshot->forcewake.ref);
> +	drm_printf(p, "\tMMIO base: 0x%08x\n", snapshot->mmio_base);
> +	drm_printf(p, "\tHWSTAM: 0x%08x\n", snapshot->reg.ring_hwstam);
> +	drm_printf(p, "\tRING_HWS_PGA: 0x%08x\n", snapshot->reg.ring_hws_pga);
>  	drm_printf(p, "\tRING_EXECLIST_STATUS_LO: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_LO(0).reg));
> +		   snapshot->reg.ring_execlist_status_lo);
>  	drm_printf(p, "\tRING_EXECLIST_STATUS_HI: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_EXECLIST_STATUS_HI(0).reg));
> +		   snapshot->reg.ring_execlist_status_hi);
>  	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_LO: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe,
> -					 RING_EXECLIST_SQ_CONTENTS_LO(0).reg));
> +		   snapshot->reg.ring_execlist_sq_contents_lo);
>  	drm_printf(p, "\tRING_EXECLIST_SQ_CONTENTS_HI: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe,
> -					 RING_EXECLIST_SQ_CONTENTS_HI(0).reg));
> +		   snapshot->reg.ring_execlist_sq_contents_hi);
>  	drm_printf(p, "\tRING_EXECLIST_CONTROL: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_EXECLIST_CONTROL(0).reg));
> -
> -	drm_printf(p, "\tRING_START: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_START(0).reg));
> -	drm_printf(p, "\tRING_HEAD:  0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_HEAD(0).reg) & HEAD_ADDR);
> -	drm_printf(p, "\tRING_TAIL:  0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_TAIL(0).reg) & TAIL_ADDR);
> -	drm_printf(p, "\tRING_CTL: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_CTL(0).reg));
> -	drm_printf(p, "\tRING_MODE: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_MI_MODE(0).reg));
> +		   snapshot->reg.ring_execlist_control);
> +	drm_printf(p, "\tRING_START: 0x%08x\n", snapshot->reg.ring_start);
> +	drm_printf(p, "\tRING_HEAD:  0x%08x\n", snapshot->reg.ring_head);
> +	drm_printf(p, "\tRING_TAIL:  0x%08x\n", snapshot->reg.ring_tail);
> +	drm_printf(p, "\tRING_CTL: 0x%08x\n", snapshot->reg.ring_ctl);
> +	drm_printf(p, "\tRING_MODE: 0x%08x\n", snapshot->reg.ring_mi_mode);
>  	drm_printf(p, "\tRING_MODE_GEN7: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_MODE_GEN7(0).reg));
> -
> -	drm_printf(p, "\tRING_IMR:   0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_IMR(0).reg));
> -	drm_printf(p, "\tRING_ESR:   0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_ESR(0).reg));
> -	drm_printf(p, "\tRING_EMR:   0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_EMR(0).reg));
> -	drm_printf(p, "\tRING_EIR:   0x%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_EIR(0).reg));
> -
> -        drm_printf(p, "\tACTHD:  0x%08x_%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_ACTHD_UDW(0).reg),
> -		hw_engine_mmio_read32(hwe, RING_ACTHD(0).reg));
> -        drm_printf(p, "\tBBADDR: 0x%08x_%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_BBADDR_UDW(0).reg),
> -		hw_engine_mmio_read32(hwe, RING_BBADDR(0).reg));
> +		   snapshot->reg.ring_mode_gen7);
> +	drm_printf(p, "\tRING_IMR:   0x%08x\n", snapshot->reg.ring_imr);
> +	drm_printf(p, "\tRING_ESR:   0x%08x\n", snapshot->reg.ring_esr);
> +	drm_printf(p, "\tRING_EMR:   0x%08x\n", snapshot->reg.ring_emr);
> +	drm_printf(p, "\tRING_EIR:   0x%08x\n", snapshot->reg.ring_eir);
> +        drm_printf(p, "\tACTHD:  0x%08x_%08x\n", snapshot->reg.ring_acthd_udw,

The alignment looks off here, looks like spaces rather than tabs.

Also I hate 0x%08x_%08x, can we just do 0x%016llx?

> +		   snapshot->reg.ring_acthd);
> +        drm_printf(p, "\tBBADDR: 0x%08x_%08x\n", snapshot->reg.ring_bbaddr_udw,
> +		   snapshot->reg.ring_bbaddr);
>          drm_printf(p, "\tDMA_FADDR: 0x%08x_%08x\n",
> -		hw_engine_mmio_read32(hwe, RING_DMA_FADD_UDW(0).reg),
> -		hw_engine_mmio_read32(hwe, RING_DMA_FADD(0).reg));
> +		   snapshot->reg.ring_dma_fadd_udw,
> +		   snapshot->reg.ring_dma_fadd);
> +	drm_printf(p, "\tIPEIR: 0x%08x\n", snapshot->reg.ipeir);
> +	drm_printf(p, "\tIPEHR: 0x%08x\n\n", snapshot->reg.ipehr);
>  
> -	drm_printf(p, "\tIPEIR: 0x%08x\n",
> -		hw_engine_mmio_read32(hwe, IPEIR(0).reg));
> -	drm_printf(p, "\tIPEHR: 0x%08x\n\n",
> -		hw_engine_mmio_read32(hwe, IPEHR(0).reg));
> -
> -	if (hwe->class == XE_ENGINE_CLASS_COMPUTE)
> +	if (snapshot->class == XE_ENGINE_CLASS_COMPUTE)
>  		drm_printf(p, "\tGEN12_RCU_MODE: 0x%08x\n",
> -			xe_mmio_read32(hwe->gt, GEN12_RCU_MODE.reg));
> +			   snapshot->reg.rcu_mode);
> +}
> +
> +/**
> + * xe_hw_engine_snapshot_free - Free all allocated objects for a given snapshot.
> + * @snapshot: Xe HW Engine snapshot object.
> + *
> + * This function free all the memory that needed to be allocated at capture
> + * time.
> + */
> +void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot)
> +{
> +	if (!snapshot)
> +		return;
> +
> +	kfree(snapshot->name);
> +	kfree(snapshot);
> +}
> +
> +/**
> + * xe_hw_engine_print - Xe HW Engine Print.
> + * @hwe: Hardware Engine.
> + * @p: drm_printer.
> + *
> + * This function quickly capture a snapshot and immediately print it out.
> + */
> +void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p)
> +{
> +	struct xe_hw_engine_snapshot *snapshot;
>  
> +	snapshot = xe_hw_engine_snapshot_capture(hwe);
> +	xe_hw_engine_snapshot_print(snapshot, p);
> +	xe_hw_engine_snapshot_free(snapshot);
>  }
>  
>  u32 xe_hw_engine_mask_per_class(struct xe_gt *gt,
> diff --git a/drivers/gpu/drm/xe/xe_hw_engine.h b/drivers/gpu/drm/xe/xe_hw_engine.h
> index ceab65397256..a0514bf859c6 100644
> --- a/drivers/gpu/drm/xe/xe_hw_engine.h
> +++ b/drivers/gpu/drm/xe/xe_hw_engine.h
> @@ -14,10 +14,16 @@ int xe_hw_engines_init_early(struct xe_gt *gt);
>  int xe_hw_engines_init(struct xe_gt *gt);
>  void xe_hw_engine_handle_irq(struct xe_hw_engine *hwe, u16 intr_vec);
>  void xe_hw_engine_enable_ring(struct xe_hw_engine *hwe);
> -void xe_hw_engine_print_state(struct xe_hw_engine *hwe, struct drm_printer *p);
>  u32 xe_hw_engine_mask_per_class(struct xe_gt *gt,
>  				enum xe_engine_class engine_class);
>  
> +struct xe_hw_engine_snapshot *
> +xe_hw_engine_snapshot_capture(struct xe_hw_engine *hwe);
> +void xe_hw_engine_snapshot_free(struct xe_hw_engine_snapshot *snapshot);
> +void xe_hw_engine_snapshot_print(struct xe_hw_engine_snapshot *snapshot,
> +				 struct drm_printer *p);
> +void xe_hw_engine_print(struct xe_hw_engine *hwe, struct drm_printer *p);
> +
>  bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe);
>  static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe)
>  {
> diff --git a/drivers/gpu/drm/xe/xe_hw_engine_types.h b/drivers/gpu/drm/xe/xe_hw_engine_types.h
> index 2c40384957da..c15d6c671fbb 100644
> --- a/drivers/gpu/drm/xe/xe_hw_engine_types.h
> +++ b/drivers/gpu/drm/xe/xe_hw_engine_types.h
> @@ -109,4 +109,82 @@ struct xe_hw_engine {
>  	enum xe_hw_engine_id engine_id;
>  };
>  
> +/**
> + * struct xe_hw_engine_snapshot - Hardware engine snapshot
> + *
> + * Contains the snapshot of usefull hardware engine info and registers.
> + */
> +struct xe_hw_engine_snapshot {
> +	/** @name: name of the hw engine */
> +	char *name;
> +	/** @class: class of this hw engine */
> +	enum xe_engine_class class;
> +	/** @logical_instance: logical instance of this hw engine */
> +	u16 logical_instance;
> +	/** @forcewake: Force Wake information snapshot */
> +	struct {
> +		/** @domain: force wake domain of this hw engine */
> +		enum xe_force_wake_domains domain;
> +		/** @ref: Forcewake ref for the above domain */
> +		int ref;
> +	} forcewake;
> +	/** @reg: Useful MMIO register snapshot */

Should be down 1 more line.

Matt

> +	/** @mmio_base: MMIO base address of this hw engine*/
> +	u32 mmio_base;
> +	struct {
> +		/** @ring_hwstam: RING_HWSTAM */
> +		u32 ring_hwstam;
> +		/** @ring_hws_pga: RING_HWS_PGA */
> +		u32 ring_hws_pga;
> +		/** @ring_execlist_status_lo: RING_EXECLIST_STATUS_LO */
> +		u32 ring_execlist_status_lo;
> +		/** @ring_execlist_status_hi: RING_EXECLIST_STATUS_HI */
> +		u32 ring_execlist_status_hi;
> +		/** @ring_execlist_sq_contents_lo: RING_EXECLIST_SQ_CONTENTS */
> +		u32 ring_execlist_sq_contents_lo;
> +		/** @ring_execlist_sq_contents_hi: RING_EXECLIST_SQ_CONTENTS + 4 */
> +		u32 ring_execlist_sq_contents_hi;
> +		/** @ring_execlist_control: RING_EXECLIST_CONTROL */
> +		u32 ring_execlist_control;
> +		/** @ring_start: RING_START */
> +		u32 ring_start;
> +		/** @ring_head: RING_HEAD */
> +		u32 ring_head;
> +		/** @ring_tail: RING_TAIL */
> +		u32 ring_tail;
> +		/** @ring_ctl: RING_CTL */
> +		u32 ring_ctl;
> +		/** @ring_mi_mode: RING_MI_MODE */
> +		u32 ring_mi_mode;
> +		/** @ring_mode_gen7: RING_MODE_GEN7 */
> +		u32 ring_mode_gen7;
> +		/** @ring_imr: RING_IMR */
> +		u32 ring_imr;
> +		/** @ring_esr: RING_ESR */
> +		u32 ring_esr;
> +		/** @ring_emr: RING_EMR */
> +		u32 ring_emr;
> +		/** @ring_eir: RING_EIR */
> +		u32 ring_eir;
> +		/** @ring_acthd_udw: RING_ACTHD_UDW */
> +		u32 ring_acthd_udw;
> +		/** @ring_acthd: RING_ACTHD */
> +		u32 ring_acthd;
> +		/** @ring_bbaddr_udw: RING_BBADDR_UDW */
> +		u32 ring_bbaddr_udw;
> +		/** @ring_bbaddr: RING_BBADDR */
> +		u32 ring_bbaddr;
> +		/** @ring_dma_fadd_udw: RING_DMA_FADD_UDW */
> +		u32 ring_dma_fadd_udw;
> +		/** @ring_dma_fadd: RING_DMA_FADD */
> +		u32 ring_dma_fadd;
> +		/** @ipeir: IPEIR */
> +		u32 ipeir;
> +		/** @ipehr: IPEHR */
> +		u32 ipehr;
> +		/** @rcu_mode: GEN12_RCU_MODE */
> +		u32 rcu_mode;
> +	} reg;
> +};
> +
>  #endif
> -- 
> 2.39.2
> 

[PATCH 11/14] drm/xe: Add HW Engine snapshot to xe_devcoredump.

[Rodrigo Vivi]

Let's continue to add our existent simple logs to devcoredump one
by one. Any format change should come on follow-up work.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 45 +++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  4 ++
 2 files changed, 49 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index 0e7ec654a9f2..1ffd12646a99 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -9,10 +9,13 @@
 #include <linux/devcoredump.h>
 #include <generated/utsrelease.h>
 
+#include "xe_device.h"
 #include "xe_engine.h"
+#include "xe_force_wake.h"
 #include "xe_gt.h"
 #include "xe_guc_ct.h"
 #include "xe_guc_submit.h"
+#include "xe_hw_engine.h"
 
 /**
  * DOC: Xe device coredump
@@ -62,6 +65,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	struct drm_printer p;
 	struct drm_print_iterator iter;
 	struct timespec64 ts;
+	struct xe_engine *e;
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
 
 	/* Our device is gone already... */
 	if (!data || !coredump_to_xe(coredump))
@@ -75,6 +81,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	mutex_lock(&coredump->lock);
 
 	ss = &coredump->snapshot;
+	e = coredump->faulty_engine;
 	p = drm_coredump_printer(&iter);
 
 	drm_printf(&p, "**** Xe Device Coredump ****\n");
@@ -92,6 +99,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
 	xe_guc_engine_snapshot_print(coredump->snapshot.ge, &p);
 
+	drm_printf(&p, "\n**** HW Engines ****\n");
+	for_each_hw_engine(hwe, e->gt, id)
+		xe_hw_engine_snapshot_print(coredump->snapshot.hwe[id], &p);
+
 	mutex_unlock(&coredump->lock);
 
 	return count - iter.remain;
@@ -100,6 +111,8 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 static void xe_devcoredump_free(void *data)
 {
 	struct xe_devcoredump *coredump = data;
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
 
 	/* Our device is gone. Nothing to do... */
 	if (!data || !coredump_to_xe(coredump))
@@ -109,6 +122,8 @@ static void xe_devcoredump_free(void *data)
 
 	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
 	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
+	for_each_hw_engine(hwe, coredump->faulty_engine->gt, id)
+		xe_hw_engine_snapshot_free(coredump->snapshot.hwe[id]);
 
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
@@ -122,13 +137,43 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
 	struct xe_engine *e = coredump->faulty_engine;
 	struct xe_guc *guc = engine_to_guc(e);
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
+	u32 adj_logical_mask = e->logical_mask;
+	u32 width_mask = (0x1 << e->width) - 1;
+	int i;
+	bool cookie;
 
 	lockdep_assert_held(&coredump->lock);
 	ss->snapshot_time = ktime_get_real();
 	ss->boot_time = ktime_get_boottime();
 
+	cookie = dma_fence_begin_signalling();
+	for (i = 0; e->width > 1 && i < XE_HW_ENGINE_MAX_INSTANCE;) {
+		if (adj_logical_mask & BIT(i)) {
+			adj_logical_mask |= width_mask << i;
+			i += e->width;
+		} else {
+			++i;
+		}
+	}
+
+	xe_force_wake_get(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
+
 	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
 	coredump->snapshot.ge = xe_guc_engine_snapshot_capture(e);
+
+	for_each_hw_engine(hwe, e->gt, id) {
+		if (hwe->class != e->hwe->class ||
+		    !(BIT(hwe->logical_instance) & adj_logical_mask)) {
+			coredump->snapshot.hwe[id] = NULL;
+			continue;
+		}
+		coredump->snapshot.hwe[id] = xe_hw_engine_snapshot_capture(hwe);
+	}
+
+	xe_force_wake_put(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
+	dma_fence_end_signalling(cookie);
 }
 
 /**
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index e055b266af70..8b17ecf1b6e6 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -9,6 +9,8 @@
 #include <linux/ktime.h>
 #include <linux/mutex.h>
 
+#include "xe_hw_engine_types.h"
+
 struct xe_device;
 
 /**
@@ -29,6 +31,8 @@ struct xe_devcoredump_snapshot {
 	struct xe_guc_ct_snapshot *ct;
 	/** @ge: Guc Engine snapshot */
 	struct xe_guc_submit_engine_snapshot *ge;
+	/** @hwe: HW Engine snapshot array */
+	struct xe_hw_engine_snapshot *hwe[XE_NUM_HW_ENGINES];
 };
 
 /**
-- 
2.39.2

[Intel-xe] [PATCH 11/14] drm/xe: Add HW Engine snapshot to xe_devcoredump.

[Rodrigo Vivi]

Let's continue to add our existent simple logs to devcoredump one
by one. Any format change should come on follow-up work.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 45 +++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h |  4 ++
 2 files changed, 49 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index 0e7ec654a9f2..1ffd12646a99 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -9,10 +9,13 @@
 #include <linux/devcoredump.h>
 #include <generated/utsrelease.h>
 
+#include "xe_device.h"
 #include "xe_engine.h"
+#include "xe_force_wake.h"
 #include "xe_gt.h"
 #include "xe_guc_ct.h"
 #include "xe_guc_submit.h"
+#include "xe_hw_engine.h"
 
 /**
  * DOC: Xe device coredump
@@ -62,6 +65,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	struct drm_printer p;
 	struct drm_print_iterator iter;
 	struct timespec64 ts;
+	struct xe_engine *e;
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
 
 	/* Our device is gone already... */
 	if (!data || !coredump_to_xe(coredump))
@@ -75,6 +81,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	mutex_lock(&coredump->lock);
 
 	ss = &coredump->snapshot;
+	e = coredump->faulty_engine;
 	p = drm_coredump_printer(&iter);
 
 	drm_printf(&p, "**** Xe Device Coredump ****\n");
@@ -92,6 +99,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
 	xe_guc_engine_snapshot_print(coredump->snapshot.ge, &p);
 
+	drm_printf(&p, "\n**** HW Engines ****\n");
+	for_each_hw_engine(hwe, e->gt, id)
+		xe_hw_engine_snapshot_print(coredump->snapshot.hwe[id], &p);
+
 	mutex_unlock(&coredump->lock);
 
 	return count - iter.remain;
@@ -100,6 +111,8 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 static void xe_devcoredump_free(void *data)
 {
 	struct xe_devcoredump *coredump = data;
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
 
 	/* Our device is gone. Nothing to do... */
 	if (!data || !coredump_to_xe(coredump))
@@ -109,6 +122,8 @@ static void xe_devcoredump_free(void *data)
 
 	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
 	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
+	for_each_hw_engine(hwe, coredump->faulty_engine->gt, id)
+		xe_hw_engine_snapshot_free(coredump->snapshot.hwe[id]);
 
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
@@ -122,13 +137,43 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
 	struct xe_engine *e = coredump->faulty_engine;
 	struct xe_guc *guc = engine_to_guc(e);
+	struct xe_hw_engine *hwe;
+	enum xe_hw_engine_id id;
+	u32 adj_logical_mask = e->logical_mask;
+	u32 width_mask = (0x1 << e->width) - 1;
+	int i;
+	bool cookie;
 
 	lockdep_assert_held(&coredump->lock);
 	ss->snapshot_time = ktime_get_real();
 	ss->boot_time = ktime_get_boottime();
 
+	cookie = dma_fence_begin_signalling();
+	for (i = 0; e->width > 1 && i < XE_HW_ENGINE_MAX_INSTANCE;) {
+		if (adj_logical_mask & BIT(i)) {
+			adj_logical_mask |= width_mask << i;
+			i += e->width;
+		} else {
+			++i;
+		}
+	}
+
+	xe_force_wake_get(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
+
 	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
 	coredump->snapshot.ge = xe_guc_engine_snapshot_capture(e);
+
+	for_each_hw_engine(hwe, e->gt, id) {
+		if (hwe->class != e->hwe->class ||
+		    !(BIT(hwe->logical_instance) & adj_logical_mask)) {
+			coredump->snapshot.hwe[id] = NULL;
+			continue;
+		}
+		coredump->snapshot.hwe[id] = xe_hw_engine_snapshot_capture(hwe);
+	}
+
+	xe_force_wake_put(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
+	dma_fence_end_signalling(cookie);
 }
 
 /**
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index e055b266af70..8b17ecf1b6e6 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -9,6 +9,8 @@
 #include <linux/ktime.h>
 #include <linux/mutex.h>
 
+#include "xe_hw_engine_types.h"
+
 struct xe_device;
 
 /**
@@ -29,6 +31,8 @@ struct xe_devcoredump_snapshot {
 	struct xe_guc_ct_snapshot *ct;
 	/** @ge: Guc Engine snapshot */
 	struct xe_guc_submit_engine_snapshot *ge;
+	/** @hwe: HW Engine snapshot array */
+	struct xe_hw_engine_snapshot *hwe[XE_NUM_HW_ENGINES];
 };
 
 /**
-- 
2.39.2

Re: [Intel-xe] [PATCH 11/14] drm/xe: Add HW Engine snapshot to xe_devcoredump.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:10PM -0400, Rodrigo Vivi wrote:
> Let's continue to add our existent simple logs to devcoredump one
> by one. Any format change should come on follow-up work.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c       | 45 +++++++++++++++++++++++
>  drivers/gpu/drm/xe/xe_devcoredump_types.h |  4 ++
>  2 files changed, 49 insertions(+)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index 0e7ec654a9f2..1ffd12646a99 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -9,10 +9,13 @@
>  #include <linux/devcoredump.h>
>  #include <generated/utsrelease.h>
>  
> +#include "xe_device.h"
>  #include "xe_engine.h"
> +#include "xe_force_wake.h"
>  #include "xe_gt.h"
>  #include "xe_guc_ct.h"
>  #include "xe_guc_submit.h"
> +#include "xe_hw_engine.h"
>  
>  /**
>   * DOC: Xe device coredump
> @@ -62,6 +65,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	struct drm_printer p;
>  	struct drm_print_iterator iter;
>  	struct timespec64 ts;
> +	struct xe_engine *e;
> +	struct xe_hw_engine *hwe;
> +	enum xe_hw_engine_id id;
>  
>  	/* Our device is gone already... */
>  	if (!data || !coredump_to_xe(coredump))
> @@ -75,6 +81,7 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	mutex_lock(&coredump->lock);
>  
>  	ss = &coredump->snapshot;
> +	e = coredump->faulty_engine;
>  	p = drm_coredump_printer(&iter);
>  
>  	drm_printf(&p, "**** Xe Device Coredump ****\n");
> @@ -92,6 +99,10 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	xe_guc_ct_snapshot_print(coredump->snapshot.ct, &p);
>  	xe_guc_engine_snapshot_print(coredump->snapshot.ge, &p);
>  
> +	drm_printf(&p, "\n**** HW Engines ****\n");
> +	for_each_hw_engine(hwe, e->gt, id)
> +		xe_hw_engine_snapshot_print(coredump->snapshot.hwe[id], &p);
> +
>  	mutex_unlock(&coredump->lock);
>  
>  	return count - iter.remain;
> @@ -100,6 +111,8 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  static void xe_devcoredump_free(void *data)
>  {
>  	struct xe_devcoredump *coredump = data;
> +	struct xe_hw_engine *hwe;
> +	enum xe_hw_engine_id id;
>  
>  	/* Our device is gone. Nothing to do... */
>  	if (!data || !coredump_to_xe(coredump))
> @@ -109,6 +122,8 @@ static void xe_devcoredump_free(void *data)
>  
>  	xe_guc_ct_snapshot_free(coredump->snapshot.ct);
>  	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
> +	for_each_hw_engine(hwe, coredump->faulty_engine->gt, id)
> +		xe_hw_engine_snapshot_free(coredump->snapshot.hwe[id]);
>  
>  	coredump->faulty_engine = NULL;
>  	drm_info(&coredump_to_xe(coredump)->drm,
> @@ -122,13 +137,43 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump)
>  	struct xe_devcoredump_snapshot *ss = &coredump->snapshot;
>  	struct xe_engine *e = coredump->faulty_engine;
>  	struct xe_guc *guc = engine_to_guc(e);
> +	struct xe_hw_engine *hwe;
> +	enum xe_hw_engine_id id;
> +	u32 adj_logical_mask = e->logical_mask;
> +	u32 width_mask = (0x1 << e->width) - 1;
> +	int i;
> +	bool cookie;
>  
>  	lockdep_assert_held(&coredump->lock);
>  	ss->snapshot_time = ktime_get_real();
>  	ss->boot_time = ktime_get_boottime();
>  
> +	cookie = dma_fence_begin_signalling();

Why the annotation here? Otherwise LGTM.

Matt

> +	for (i = 0; e->width > 1 && i < XE_HW_ENGINE_MAX_INSTANCE;) {
> +		if (adj_logical_mask & BIT(i)) {
> +			adj_logical_mask |= width_mask << i;
> +			i += e->width;
> +		} else {
> +			++i;
> +		}
> +	}
> +
> +	xe_force_wake_get(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
> +
>  	coredump->snapshot.ct = xe_guc_ct_snapshot_capture(&guc->ct);
>  	coredump->snapshot.ge = xe_guc_engine_snapshot_capture(e);
> +
> +	for_each_hw_engine(hwe, e->gt, id) {
> +		if (hwe->class != e->hwe->class ||
> +		    !(BIT(hwe->logical_instance) & adj_logical_mask)) {
> +			coredump->snapshot.hwe[id] = NULL;
> +			continue;
> +		}
> +		coredump->snapshot.hwe[id] = xe_hw_engine_snapshot_capture(hwe);
> +	}
> +
> +	xe_force_wake_put(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
> +	dma_fence_end_signalling(cookie);
>  }
>  
>  /**
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> index e055b266af70..8b17ecf1b6e6 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
> +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> @@ -9,6 +9,8 @@
>  #include <linux/ktime.h>
>  #include <linux/mutex.h>
>  
> +#include "xe_hw_engine_types.h"
> +
>  struct xe_device;
>  
>  /**
> @@ -29,6 +31,8 @@ struct xe_devcoredump_snapshot {
>  	struct xe_guc_ct_snapshot *ct;
>  	/** @ge: Guc Engine snapshot */
>  	struct xe_guc_submit_engine_snapshot *ge;
> +	/** @hwe: HW Engine snapshot array */
> +	struct xe_hw_engine_snapshot *hwe[XE_NUM_HW_ENGINES];
>  };
>  
>  /**
> -- 
> 2.39.2
> 

[PATCH 12/14] drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.

[Rodrigo Vivi]

There are multiple kind of config prints and with the upcoming
devcoredump there will be another layer. Let's limit the config
to the top level functions and leave the clean-up work for the
compilers so we don't create a spider-web of configs.

No functional change. Just a preparation for the devcoredump.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_vm.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index bdf82d34eb66..4cffdb84680a 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -3369,7 +3369,6 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
 	return 0;
 }
 
-#if IS_ENABLED(CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE)
 int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
 {
 	struct rb_node *node;
@@ -3405,9 +3404,3 @@ int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
 
 	return 0;
 }
-#else
-int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
-{
-	return 0;
-}
-#endif
-- 
2.39.2

[Intel-xe] [PATCH 12/14] drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.

[Rodrigo Vivi]

There are multiple kind of config prints and with the upcoming
devcoredump there will be another layer. Let's limit the config
to the top level functions and leave the clean-up work for the
compilers so we don't create a spider-web of configs.

No functional change. Just a preparation for the devcoredump.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_vm.c | 7 -------
 1 file changed, 7 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index bdf82d34eb66..4cffdb84680a 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -3369,7 +3369,6 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
 	return 0;
 }
 
-#if IS_ENABLED(CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE)
 int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
 {
 	struct rb_node *node;
@@ -3405,9 +3404,3 @@ int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
 
 	return 0;
 }
-#else
-int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
-{
-	return 0;
-}
-#endif
-- 
2.39.2

Re: [PATCH 12/14] drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:11PM -0400, Rodrigo Vivi wrote:
> There are multiple kind of config prints and with the upcoming
> devcoredump there will be another layer. Let's limit the config
> to the top level functions and leave the clean-up work for the
> compilers so we don't create a spider-web of configs.
> 
> No functional change. Just a preparation for the devcoredump.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Again will conflict with GPUVA, not that big of a deal, I'll leave it up
to you.

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_vm.c | 7 -------
>  1 file changed, 7 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index bdf82d34eb66..4cffdb84680a 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -3369,7 +3369,6 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
>  	return 0;
>  }
>  
> -#if IS_ENABLED(CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE)
>  int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
>  {
>  	struct rb_node *node;
> @@ -3405,9 +3404,3 @@ int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
>  
>  	return 0;
>  }
> -#else
> -int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
> -{
> -	return 0;
> -}
> -#endif
> -- 
> 2.39.2
> 

Re: [Intel-xe] [PATCH 12/14] drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:11PM -0400, Rodrigo Vivi wrote:
> There are multiple kind of config prints and with the upcoming
> devcoredump there will be another layer. Let's limit the config
> to the top level functions and leave the clean-up work for the
> compilers so we don't create a spider-web of configs.
> 
> No functional change. Just a preparation for the devcoredump.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Again will conflict with GPUVA, not that big of a deal, I'll leave it up
to you.

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_vm.c | 7 -------
>  1 file changed, 7 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index bdf82d34eb66..4cffdb84680a 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -3369,7 +3369,6 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
>  	return 0;
>  }
>  
> -#if IS_ENABLED(CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE)
>  int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
>  {
>  	struct rb_node *node;
> @@ -3405,9 +3404,3 @@ int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
>  
>  	return 0;
>  }
> -#else
> -int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
> -{
> -	return 0;
> -}
> -#endif
> -- 
> 2.39.2
> 

[PATCH 13/14] drm/xe: Convert VM print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_submit.c |   2 +-
 drivers/gpu/drm/xe/xe_vm.c         | 137 +++++++++++++++++++++++++----
 drivers/gpu/drm/xe/xe_vm.h         |   6 +-
 drivers/gpu/drm/xe/xe_vm_types.h   |  18 ++++
 4 files changed, 143 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index 74659d0a69b3..ac98bc1843e8 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -753,7 +753,7 @@ static void simple_error_capture(struct xe_engine *e)
 				continue;
 			xe_hw_engine_print(hwe, &p);
 		}
-		xe_analyze_vm(&p, e->vm, e->gt->info.id);
+		xe_vm_print(&p, e->vm, e->gt->info.id);
 		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
 		dma_fence_end_signalling(cookie);
 	}
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 4cffdb84680a..075640dbdff0 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -3369,38 +3369,139 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
 	return 0;
 }
 
-int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
+/**
+ * xe_vm_snapshot_capture - Take a quick snapshot of the HW Engine.
+ * @vm: Xe VM
+ * @gt_id: GT id number
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a Xe VM snapshot object that must be freed by the
+ * 	    caller, using `xe_vm_snapshot_free`.
+ */
+struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id)
 {
+	struct xe_vm_snapshot *snapshot;
 	struct rb_node *node;
-	bool is_vram;
-	uint64_t addr;
+	int i = 0;
+
+	snapshot = kzalloc(sizeof(struct xe_vm_snapshot), GFP_ATOMIC);
+
+	if (!down_read_trylock(&vm->lock))
+		return snapshot;
+
+	snapshot->acquired = true;
+
+	for (node = rb_first(&vm->vmas); node; node = rb_next(node))
+		snapshot->num_nodes++;
+
+	snapshot->vm_nodes = kmalloc_array(snapshot->num_nodes,
+					   sizeof(struct vm_node_snapshot),
+					   GFP_ATOMIC);
 
-	if (!down_read_trylock(&vm->lock)) {
-		drm_printf(p, " Failed to acquire VM lock to dump capture");
-		return 0;
-	}
 	if (vm->pt_root[gt_id]) {
-		addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0, GEN8_PAGE_SIZE, &is_vram);
-		drm_printf(p, " VM root: A:0x%llx %s\n", addr, is_vram ? "VRAM" : "SYS");
+		snapshot->vm_root = kzalloc(sizeof(struct vm_node_snapshot),
+				      GFP_ATOMIC);
+		snapshot->vm_root->addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0,
+					       GEN8_PAGE_SIZE,
+					       &snapshot->vm_root->is_vram);
 	}
 
 	for (node = rb_first(&vm->vmas); node; node = rb_next(node)) {
 		struct xe_vma *vma = to_xe_vma(node);
-		bool is_userptr = xe_vma_is_userptr(vma);
+		snapshot->vm_nodes[i].is_userptr = xe_vma_is_userptr(vma);
 
-		if (is_userptr) {
+		if (snapshot->vm_nodes[i].is_userptr) {
 			struct xe_res_cursor cur;
 
-			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE, &cur);
-			addr = xe_res_dma(&cur);
+			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE,
+					&cur);
+			snapshot->vm_nodes[i].addr = xe_res_dma(&cur);
 		} else {
-			addr = xe_bo_addr(vma->bo, 0, GEN8_PAGE_SIZE, &is_vram);
+			snapshot->vm_nodes[i].addr = xe_bo_addr(vma->bo, 0,
+							  GEN8_PAGE_SIZE,
+							  &snapshot->vm_nodes[i].is_vram);
 		}
-		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
-			   vma->start, vma->end, vma->end - vma->start + 1ull,
-			   addr, is_userptr ? "USR" : is_vram ? "VRAM" : "SYS");
+		snapshot->vm_nodes[i].vma.start = vma->start;
+		snapshot->vm_nodes[i].vma.end = vma->end;
+		i++;
 	}
 	up_read(&vm->lock);
 
-	return 0;
+	return snapshot;
+}
+
+/**
+ * xe_vm_snapshot_print - Print out a given Xe HW Engine snapshot.
+ * @snapshot: Xe VM snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given Xe HW Engine snapshot object.
+ */
+void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
+			  struct drm_printer *p)
+{
+	int i;
+
+	if (!snapshot)
+		return;
+
+	if (!snapshot->acquired) {
+		drm_printf(p, " Failed to acquire VM lock to dump capture");
+		return;
+	}
+
+	if (snapshot->vm_root) {
+		drm_printf(p, " VM root: A:0x%llx %s\n",
+			   snapshot->vm_root->addr,
+			   snapshot->vm_root->is_vram ? "VRAM" : "SYS");
+	}
+
+	for (i = 0; snapshot->vm_nodes && i < snapshot->num_nodes; i++)
+		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
+			   snapshot->vm_nodes[i].vma.start,
+			   snapshot->vm_nodes[i].vma.end,
+			   snapshot->vm_nodes[i].vma.end -
+			   snapshot->vm_nodes[i].vma.start + 1ull,
+			   snapshot->vm_nodes[i].addr,
+			   snapshot->vm_nodes[i].is_userptr ?
+			   "USR" : snapshot->vm_nodes[i].is_vram ?
+			   "VRAM" : "SYS");
+}
+
+/**
+ * xe_vm_snapshot_free - Free all allocated objects for a given snapshot.
+ * @snapshot: Xe VM snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot)
+{
+	if (!snapshot)
+		return;
+
+	if (snapshot->vm_root)
+		kfree(snapshot->vm_root);
+	if (snapshot->vm_nodes)
+		kfree(snapshot->vm_nodes);
+	kfree(snapshot);
+}
+
+/**
+ * xe_vm_print - Xe VM Print.
+ * @p: drm_printer
+ * @vm: Xe VM
+ * @gt_id: GT id number
+ *
+ * This function quickly capture a snapshot and immediately print it out.
+ */
+void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id)
+{
+	struct xe_vm_snapshot *snapshot;
+
+	snapshot = xe_vm_snapshot_capture(vm, gt_id);
+	xe_vm_snapshot_print(snapshot, p);
+	xe_vm_snapshot_free(snapshot);
 }
diff --git a/drivers/gpu/drm/xe/xe_vm.h b/drivers/gpu/drm/xe/xe_vm.h
index 748dc16ebed9..924884b36469 100644
--- a/drivers/gpu/drm/xe/xe_vm.h
+++ b/drivers/gpu/drm/xe/xe_vm.h
@@ -145,7 +145,11 @@ void xe_vm_unlock_dma_resv(struct xe_vm *vm,
 void xe_vm_fence_all_extobjs(struct xe_vm *vm, struct dma_fence *fence,
 			     enum dma_resv_usage usage);
 
-int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);
+struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id);
+void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
+			  struct drm_printer *p);
+void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot);
+void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id);
 
 #if IS_ENABLED(CONFIG_DRM_XE_DEBUG_VM)
 #define vm_dbg drm_dbg
diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
index fada7896867f..18e79b6a2182 100644
--- a/drivers/gpu/drm/xe/xe_vm_types.h
+++ b/drivers/gpu/drm/xe/xe_vm_types.h
@@ -149,6 +149,24 @@ struct xe_vma {
 	} extobj;
 };
 
+
+struct vm_node_snapshot {
+	bool is_userptr;
+	bool is_vram;
+	struct {
+		u64 start;
+		u64 end;
+	} vma;
+	u64 addr;
+};
+
+struct xe_vm_snapshot {
+	bool acquired;
+	struct vm_node_snapshot *vm_root;
+	struct vm_node_snapshot *vm_nodes;
+	int num_nodes;
+};
+
 struct xe_device;
 
 #define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->resv)
-- 
2.39.2

[Intel-xe] [PATCH 13/14] drm/xe: Convert VM print to snapshot capture and print.

[Rodrigo Vivi]

The goal is to allow for a snapshot capture to be taken at the time
of the crash, while the print out can happen at a later time through
the exposed devcoredump virtual device.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_guc_submit.c |   2 +-
 drivers/gpu/drm/xe/xe_vm.c         | 137 +++++++++++++++++++++++++----
 drivers/gpu/drm/xe/xe_vm.h         |   6 +-
 drivers/gpu/drm/xe/xe_vm_types.h   |  18 ++++
 4 files changed, 143 insertions(+), 20 deletions(-)

diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
index 74659d0a69b3..ac98bc1843e8 100644
--- a/drivers/gpu/drm/xe/xe_guc_submit.c
+++ b/drivers/gpu/drm/xe/xe_guc_submit.c
@@ -753,7 +753,7 @@ static void simple_error_capture(struct xe_engine *e)
 				continue;
 			xe_hw_engine_print(hwe, &p);
 		}
-		xe_analyze_vm(&p, e->vm, e->gt->info.id);
+		xe_vm_print(&p, e->vm, e->gt->info.id);
 		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
 		dma_fence_end_signalling(cookie);
 	}
diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
index 4cffdb84680a..075640dbdff0 100644
--- a/drivers/gpu/drm/xe/xe_vm.c
+++ b/drivers/gpu/drm/xe/xe_vm.c
@@ -3369,38 +3369,139 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
 	return 0;
 }
 
-int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
+/**
+ * xe_vm_snapshot_capture - Take a quick snapshot of the HW Engine.
+ * @vm: Xe VM
+ * @gt_id: GT id number
+ *
+ * This can be printed out in a later stage like during dev_coredump
+ * analysis.
+ *
+ * Returns: a Xe VM snapshot object that must be freed by the
+ * 	    caller, using `xe_vm_snapshot_free`.
+ */
+struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id)
 {
+	struct xe_vm_snapshot *snapshot;
 	struct rb_node *node;
-	bool is_vram;
-	uint64_t addr;
+	int i = 0;
+
+	snapshot = kzalloc(sizeof(struct xe_vm_snapshot), GFP_ATOMIC);
+
+	if (!down_read_trylock(&vm->lock))
+		return snapshot;
+
+	snapshot->acquired = true;
+
+	for (node = rb_first(&vm->vmas); node; node = rb_next(node))
+		snapshot->num_nodes++;
+
+	snapshot->vm_nodes = kmalloc_array(snapshot->num_nodes,
+					   sizeof(struct vm_node_snapshot),
+					   GFP_ATOMIC);
 
-	if (!down_read_trylock(&vm->lock)) {
-		drm_printf(p, " Failed to acquire VM lock to dump capture");
-		return 0;
-	}
 	if (vm->pt_root[gt_id]) {
-		addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0, GEN8_PAGE_SIZE, &is_vram);
-		drm_printf(p, " VM root: A:0x%llx %s\n", addr, is_vram ? "VRAM" : "SYS");
+		snapshot->vm_root = kzalloc(sizeof(struct vm_node_snapshot),
+				      GFP_ATOMIC);
+		snapshot->vm_root->addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0,
+					       GEN8_PAGE_SIZE,
+					       &snapshot->vm_root->is_vram);
 	}
 
 	for (node = rb_first(&vm->vmas); node; node = rb_next(node)) {
 		struct xe_vma *vma = to_xe_vma(node);
-		bool is_userptr = xe_vma_is_userptr(vma);
+		snapshot->vm_nodes[i].is_userptr = xe_vma_is_userptr(vma);
 
-		if (is_userptr) {
+		if (snapshot->vm_nodes[i].is_userptr) {
 			struct xe_res_cursor cur;
 
-			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE, &cur);
-			addr = xe_res_dma(&cur);
+			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE,
+					&cur);
+			snapshot->vm_nodes[i].addr = xe_res_dma(&cur);
 		} else {
-			addr = xe_bo_addr(vma->bo, 0, GEN8_PAGE_SIZE, &is_vram);
+			snapshot->vm_nodes[i].addr = xe_bo_addr(vma->bo, 0,
+							  GEN8_PAGE_SIZE,
+							  &snapshot->vm_nodes[i].is_vram);
 		}
-		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
-			   vma->start, vma->end, vma->end - vma->start + 1ull,
-			   addr, is_userptr ? "USR" : is_vram ? "VRAM" : "SYS");
+		snapshot->vm_nodes[i].vma.start = vma->start;
+		snapshot->vm_nodes[i].vma.end = vma->end;
+		i++;
 	}
 	up_read(&vm->lock);
 
-	return 0;
+	return snapshot;
+}
+
+/**
+ * xe_vm_snapshot_print - Print out a given Xe HW Engine snapshot.
+ * @snapshot: Xe VM snapshot object.
+ * @p: drm_printer where it will be printed out.
+ *
+ * This function prints out a given Xe HW Engine snapshot object.
+ */
+void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
+			  struct drm_printer *p)
+{
+	int i;
+
+	if (!snapshot)
+		return;
+
+	if (!snapshot->acquired) {
+		drm_printf(p, " Failed to acquire VM lock to dump capture");
+		return;
+	}
+
+	if (snapshot->vm_root) {
+		drm_printf(p, " VM root: A:0x%llx %s\n",
+			   snapshot->vm_root->addr,
+			   snapshot->vm_root->is_vram ? "VRAM" : "SYS");
+	}
+
+	for (i = 0; snapshot->vm_nodes && i < snapshot->num_nodes; i++)
+		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
+			   snapshot->vm_nodes[i].vma.start,
+			   snapshot->vm_nodes[i].vma.end,
+			   snapshot->vm_nodes[i].vma.end -
+			   snapshot->vm_nodes[i].vma.start + 1ull,
+			   snapshot->vm_nodes[i].addr,
+			   snapshot->vm_nodes[i].is_userptr ?
+			   "USR" : snapshot->vm_nodes[i].is_vram ?
+			   "VRAM" : "SYS");
+}
+
+/**
+ * xe_vm_snapshot_free - Free all allocated objects for a given snapshot.
+ * @snapshot: Xe VM snapshot object.
+ *
+ * This function free all the memory that needed to be allocated at capture
+ * time.
+ */
+void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot)
+{
+	if (!snapshot)
+		return;
+
+	if (snapshot->vm_root)
+		kfree(snapshot->vm_root);
+	if (snapshot->vm_nodes)
+		kfree(snapshot->vm_nodes);
+	kfree(snapshot);
+}
+
+/**
+ * xe_vm_print - Xe VM Print.
+ * @p: drm_printer
+ * @vm: Xe VM
+ * @gt_id: GT id number
+ *
+ * This function quickly capture a snapshot and immediately print it out.
+ */
+void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id)
+{
+	struct xe_vm_snapshot *snapshot;
+
+	snapshot = xe_vm_snapshot_capture(vm, gt_id);
+	xe_vm_snapshot_print(snapshot, p);
+	xe_vm_snapshot_free(snapshot);
 }
diff --git a/drivers/gpu/drm/xe/xe_vm.h b/drivers/gpu/drm/xe/xe_vm.h
index 748dc16ebed9..924884b36469 100644
--- a/drivers/gpu/drm/xe/xe_vm.h
+++ b/drivers/gpu/drm/xe/xe_vm.h
@@ -145,7 +145,11 @@ void xe_vm_unlock_dma_resv(struct xe_vm *vm,
 void xe_vm_fence_all_extobjs(struct xe_vm *vm, struct dma_fence *fence,
 			     enum dma_resv_usage usage);
 
-int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);
+struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id);
+void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
+			  struct drm_printer *p);
+void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot);
+void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id);
 
 #if IS_ENABLED(CONFIG_DRM_XE_DEBUG_VM)
 #define vm_dbg drm_dbg
diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
index fada7896867f..18e79b6a2182 100644
--- a/drivers/gpu/drm/xe/xe_vm_types.h
+++ b/drivers/gpu/drm/xe/xe_vm_types.h
@@ -149,6 +149,24 @@ struct xe_vma {
 	} extobj;
 };
 
+
+struct vm_node_snapshot {
+	bool is_userptr;
+	bool is_vram;
+	struct {
+		u64 start;
+		u64 end;
+	} vma;
+	u64 addr;
+};
+
+struct xe_vm_snapshot {
+	bool acquired;
+	struct vm_node_snapshot *vm_root;
+	struct vm_node_snapshot *vm_nodes;
+	int num_nodes;
+};
+
 struct xe_device;
 
 #define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->resv)
-- 
2.39.2

Re: [Intel-xe] [PATCH 13/14] drm/xe: Convert VM print to snapshot capture and print.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:12PM -0400, Rodrigo Vivi wrote:
> The goal is to allow for a snapshot capture to be taken at the time
> of the crash, while the print out can happen at a later time through
> the exposed devcoredump virtual device.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

This is an example of a patch I'd like staged behind the GPUVA changes
as GPUVA will change all of this...

Matt

> ---
>  drivers/gpu/drm/xe/xe_guc_submit.c |   2 +-
>  drivers/gpu/drm/xe/xe_vm.c         | 137 +++++++++++++++++++++++++----
>  drivers/gpu/drm/xe/xe_vm.h         |   6 +-
>  drivers/gpu/drm/xe/xe_vm_types.h   |  18 ++++
>  4 files changed, 143 insertions(+), 20 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> index 74659d0a69b3..ac98bc1843e8 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> @@ -753,7 +753,7 @@ static void simple_error_capture(struct xe_engine *e)
>  				continue;
>  			xe_hw_engine_print(hwe, &p);
>  		}
> -		xe_analyze_vm(&p, e->vm, e->gt->info.id);
> +		xe_vm_print(&p, e->vm, e->gt->info.id);
>  		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
>  		dma_fence_end_signalling(cookie);
>  	}
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 4cffdb84680a..075640dbdff0 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -3369,38 +3369,139 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
>  	return 0;
>  }
>  
> -int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
> +/**
> + * xe_vm_snapshot_capture - Take a quick snapshot of the HW Engine.
> + * @vm: Xe VM
> + * @gt_id: GT id number
> + *
> + * This can be printed out in a later stage like during dev_coredump
> + * analysis.
> + *
> + * Returns: a Xe VM snapshot object that must be freed by the
> + * 	    caller, using `xe_vm_snapshot_free`.
> + */
> +struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id)
>  {
> +	struct xe_vm_snapshot *snapshot;
>  	struct rb_node *node;
> -	bool is_vram;
> -	uint64_t addr;
> +	int i = 0;
> +
> +	snapshot = kzalloc(sizeof(struct xe_vm_snapshot), GFP_ATOMIC);
> +
> +	if (!down_read_trylock(&vm->lock))
> +		return snapshot;
> +
> +	snapshot->acquired = true;
> +
> +	for (node = rb_first(&vm->vmas); node; node = rb_next(node))
> +		snapshot->num_nodes++;
> +
> +	snapshot->vm_nodes = kmalloc_array(snapshot->num_nodes,
> +					   sizeof(struct vm_node_snapshot),
> +					   GFP_ATOMIC);
>  
> -	if (!down_read_trylock(&vm->lock)) {
> -		drm_printf(p, " Failed to acquire VM lock to dump capture");
> -		return 0;
> -	}
>  	if (vm->pt_root[gt_id]) {
> -		addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0, GEN8_PAGE_SIZE, &is_vram);
> -		drm_printf(p, " VM root: A:0x%llx %s\n", addr, is_vram ? "VRAM" : "SYS");
> +		snapshot->vm_root = kzalloc(sizeof(struct vm_node_snapshot),
> +				      GFP_ATOMIC);
> +		snapshot->vm_root->addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0,
> +					       GEN8_PAGE_SIZE,
> +					       &snapshot->vm_root->is_vram);
>  	}
>  
>  	for (node = rb_first(&vm->vmas); node; node = rb_next(node)) {
>  		struct xe_vma *vma = to_xe_vma(node);
> -		bool is_userptr = xe_vma_is_userptr(vma);
> +		snapshot->vm_nodes[i].is_userptr = xe_vma_is_userptr(vma);
>  
> -		if (is_userptr) {
> +		if (snapshot->vm_nodes[i].is_userptr) {
>  			struct xe_res_cursor cur;
>  
> -			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE, &cur);
> -			addr = xe_res_dma(&cur);
> +			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE,
> +					&cur);
> +			snapshot->vm_nodes[i].addr = xe_res_dma(&cur);
>  		} else {
> -			addr = xe_bo_addr(vma->bo, 0, GEN8_PAGE_SIZE, &is_vram);
> +			snapshot->vm_nodes[i].addr = xe_bo_addr(vma->bo, 0,
> +							  GEN8_PAGE_SIZE,
> +							  &snapshot->vm_nodes[i].is_vram);
>  		}
> -		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
> -			   vma->start, vma->end, vma->end - vma->start + 1ull,
> -			   addr, is_userptr ? "USR" : is_vram ? "VRAM" : "SYS");
> +		snapshot->vm_nodes[i].vma.start = vma->start;
> +		snapshot->vm_nodes[i].vma.end = vma->end;
> +		i++;
>  	}
>  	up_read(&vm->lock);
>  
> -	return 0;
> +	return snapshot;
> +}
> +
> +/**
> + * xe_vm_snapshot_print - Print out a given Xe HW Engine snapshot.
> + * @snapshot: Xe VM snapshot object.
> + * @p: drm_printer where it will be printed out.
> + *
> + * This function prints out a given Xe HW Engine snapshot object.
> + */
> +void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
> +			  struct drm_printer *p)
> +{
> +	int i;
> +
> +	if (!snapshot)
> +		return;
> +
> +	if (!snapshot->acquired) {
> +		drm_printf(p, " Failed to acquire VM lock to dump capture");
> +		return;
> +	}
> +
> +	if (snapshot->vm_root) {
> +		drm_printf(p, " VM root: A:0x%llx %s\n",
> +			   snapshot->vm_root->addr,
> +			   snapshot->vm_root->is_vram ? "VRAM" : "SYS");
> +	}
> +
> +	for (i = 0; snapshot->vm_nodes && i < snapshot->num_nodes; i++)
> +		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
> +			   snapshot->vm_nodes[i].vma.start,
> +			   snapshot->vm_nodes[i].vma.end,
> +			   snapshot->vm_nodes[i].vma.end -
> +			   snapshot->vm_nodes[i].vma.start + 1ull,
> +			   snapshot->vm_nodes[i].addr,
> +			   snapshot->vm_nodes[i].is_userptr ?
> +			   "USR" : snapshot->vm_nodes[i].is_vram ?
> +			   "VRAM" : "SYS");
> +}
> +
> +/**
> + * xe_vm_snapshot_free - Free all allocated objects for a given snapshot.
> + * @snapshot: Xe VM snapshot object.
> + *
> + * This function free all the memory that needed to be allocated at capture
> + * time.
> + */
> +void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot)
> +{
> +	if (!snapshot)
> +		return;
> +
> +	if (snapshot->vm_root)
> +		kfree(snapshot->vm_root);
> +	if (snapshot->vm_nodes)
> +		kfree(snapshot->vm_nodes);
> +	kfree(snapshot);
> +}
> +
> +/**
> + * xe_vm_print - Xe VM Print.
> + * @p: drm_printer
> + * @vm: Xe VM
> + * @gt_id: GT id number
> + *
> + * This function quickly capture a snapshot and immediately print it out.
> + */
> +void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id)
> +{
> +	struct xe_vm_snapshot *snapshot;
> +
> +	snapshot = xe_vm_snapshot_capture(vm, gt_id);
> +	xe_vm_snapshot_print(snapshot, p);
> +	xe_vm_snapshot_free(snapshot);
>  }
> diff --git a/drivers/gpu/drm/xe/xe_vm.h b/drivers/gpu/drm/xe/xe_vm.h
> index 748dc16ebed9..924884b36469 100644
> --- a/drivers/gpu/drm/xe/xe_vm.h
> +++ b/drivers/gpu/drm/xe/xe_vm.h
> @@ -145,7 +145,11 @@ void xe_vm_unlock_dma_resv(struct xe_vm *vm,
>  void xe_vm_fence_all_extobjs(struct xe_vm *vm, struct dma_fence *fence,
>  			     enum dma_resv_usage usage);
>  
> -int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);
> +struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id);
> +void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
> +			  struct drm_printer *p);
> +void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot);
> +void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id);
>  
>  #if IS_ENABLED(CONFIG_DRM_XE_DEBUG_VM)
>  #define vm_dbg drm_dbg
> diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
> index fada7896867f..18e79b6a2182 100644
> --- a/drivers/gpu/drm/xe/xe_vm_types.h
> +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> @@ -149,6 +149,24 @@ struct xe_vma {
>  	} extobj;
>  };
>  
> +
> +struct vm_node_snapshot {
> +	bool is_userptr;
> +	bool is_vram;
> +	struct {
> +		u64 start;
> +		u64 end;
> +	} vma;
> +	u64 addr;
> +};
> +
> +struct xe_vm_snapshot {
> +	bool acquired;
> +	struct vm_node_snapshot *vm_root;
> +	struct vm_node_snapshot *vm_nodes;
> +	int num_nodes;
> +};
> +
>  struct xe_device;
>  
>  #define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->resv)
> -- 
> 2.39.2
> 

Re: [Intel-xe] [PATCH 13/14] drm/xe: Convert VM print to snapshot capture and print.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:12PM -0400, Rodrigo Vivi wrote:
> The goal is to allow for a snapshot capture to be taken at the time
> of the crash, while the print out can happen at a later time through
> the exposed devcoredump virtual device.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Also thinking out loud here, at some point we are going to need a hook
to dump the entire contexts of the VMAs...

I can think of a few options.

1. Flag on the VM creation, dump the entire VM.
2. Flag on VM binds, dump VMAs with the flag set.
3. Have both options.

Thoughts?

Matt

> ---
>  drivers/gpu/drm/xe/xe_guc_submit.c |   2 +-
>  drivers/gpu/drm/xe/xe_vm.c         | 137 +++++++++++++++++++++++++----
>  drivers/gpu/drm/xe/xe_vm.h         |   6 +-
>  drivers/gpu/drm/xe/xe_vm_types.h   |  18 ++++
>  4 files changed, 143 insertions(+), 20 deletions(-)
> 
> diff --git a/drivers/gpu/drm/xe/xe_guc_submit.c b/drivers/gpu/drm/xe/xe_guc_submit.c
> index 74659d0a69b3..ac98bc1843e8 100644
> --- a/drivers/gpu/drm/xe/xe_guc_submit.c
> +++ b/drivers/gpu/drm/xe/xe_guc_submit.c
> @@ -753,7 +753,7 @@ static void simple_error_capture(struct xe_engine *e)
>  				continue;
>  			xe_hw_engine_print(hwe, &p);
>  		}
> -		xe_analyze_vm(&p, e->vm, e->gt->info.id);
> +		xe_vm_print(&p, e->vm, e->gt->info.id);
>  		xe_force_wake_put(gt_to_fw(guc_to_gt(guc)), XE_FORCEWAKE_ALL);
>  		dma_fence_end_signalling(cookie);
>  	}
> diff --git a/drivers/gpu/drm/xe/xe_vm.c b/drivers/gpu/drm/xe/xe_vm.c
> index 4cffdb84680a..075640dbdff0 100644
> --- a/drivers/gpu/drm/xe/xe_vm.c
> +++ b/drivers/gpu/drm/xe/xe_vm.c
> @@ -3369,38 +3369,139 @@ int xe_vm_invalidate_vma(struct xe_vma *vma)
>  	return 0;
>  }
>  
> -int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id)
> +/**
> + * xe_vm_snapshot_capture - Take a quick snapshot of the HW Engine.
> + * @vm: Xe VM
> + * @gt_id: GT id number
> + *
> + * This can be printed out in a later stage like during dev_coredump
> + * analysis.
> + *
> + * Returns: a Xe VM snapshot object that must be freed by the
> + * 	    caller, using `xe_vm_snapshot_free`.
> + */
> +struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id)
>  {
> +	struct xe_vm_snapshot *snapshot;
>  	struct rb_node *node;
> -	bool is_vram;
> -	uint64_t addr;
> +	int i = 0;
> +
> +	snapshot = kzalloc(sizeof(struct xe_vm_snapshot), GFP_ATOMIC);
> +
> +	if (!down_read_trylock(&vm->lock))
> +		return snapshot;
> +
> +	snapshot->acquired = true;
> +
> +	for (node = rb_first(&vm->vmas); node; node = rb_next(node))
> +		snapshot->num_nodes++;
> +
> +	snapshot->vm_nodes = kmalloc_array(snapshot->num_nodes,
> +					   sizeof(struct vm_node_snapshot),
> +					   GFP_ATOMIC);
>  
> -	if (!down_read_trylock(&vm->lock)) {
> -		drm_printf(p, " Failed to acquire VM lock to dump capture");
> -		return 0;
> -	}
>  	if (vm->pt_root[gt_id]) {
> -		addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0, GEN8_PAGE_SIZE, &is_vram);
> -		drm_printf(p, " VM root: A:0x%llx %s\n", addr, is_vram ? "VRAM" : "SYS");
> +		snapshot->vm_root = kzalloc(sizeof(struct vm_node_snapshot),
> +				      GFP_ATOMIC);
> +		snapshot->vm_root->addr = xe_bo_addr(vm->pt_root[gt_id]->bo, 0,
> +					       GEN8_PAGE_SIZE,
> +					       &snapshot->vm_root->is_vram);
>  	}
>  
>  	for (node = rb_first(&vm->vmas); node; node = rb_next(node)) {
>  		struct xe_vma *vma = to_xe_vma(node);
> -		bool is_userptr = xe_vma_is_userptr(vma);
> +		snapshot->vm_nodes[i].is_userptr = xe_vma_is_userptr(vma);
>  
> -		if (is_userptr) {
> +		if (snapshot->vm_nodes[i].is_userptr) {
>  			struct xe_res_cursor cur;
>  
> -			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE, &cur);
> -			addr = xe_res_dma(&cur);
> +			xe_res_first_sg(vma->userptr.sg, 0, GEN8_PAGE_SIZE,
> +					&cur);
> +			snapshot->vm_nodes[i].addr = xe_res_dma(&cur);
>  		} else {
> -			addr = xe_bo_addr(vma->bo, 0, GEN8_PAGE_SIZE, &is_vram);
> +			snapshot->vm_nodes[i].addr = xe_bo_addr(vma->bo, 0,
> +							  GEN8_PAGE_SIZE,
> +							  &snapshot->vm_nodes[i].is_vram);
>  		}
> -		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
> -			   vma->start, vma->end, vma->end - vma->start + 1ull,
> -			   addr, is_userptr ? "USR" : is_vram ? "VRAM" : "SYS");
> +		snapshot->vm_nodes[i].vma.start = vma->start;
> +		snapshot->vm_nodes[i].vma.end = vma->end;
> +		i++;
>  	}
>  	up_read(&vm->lock);
>  
> -	return 0;
> +	return snapshot;
> +}
> +
> +/**
> + * xe_vm_snapshot_print - Print out a given Xe HW Engine snapshot.
> + * @snapshot: Xe VM snapshot object.
> + * @p: drm_printer where it will be printed out.
> + *
> + * This function prints out a given Xe HW Engine snapshot object.
> + */
> +void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
> +			  struct drm_printer *p)
> +{
> +	int i;
> +
> +	if (!snapshot)
> +		return;
> +
> +	if (!snapshot->acquired) {
> +		drm_printf(p, " Failed to acquire VM lock to dump capture");
> +		return;
> +	}
> +
> +	if (snapshot->vm_root) {
> +		drm_printf(p, " VM root: A:0x%llx %s\n",
> +			   snapshot->vm_root->addr,
> +			   snapshot->vm_root->is_vram ? "VRAM" : "SYS");
> +	}
> +
> +	for (i = 0; snapshot->vm_nodes && i < snapshot->num_nodes; i++)
> +		drm_printf(p, " [%016llx-%016llx] S:0x%016llx A:%016llx %s\n",
> +			   snapshot->vm_nodes[i].vma.start,
> +			   snapshot->vm_nodes[i].vma.end,
> +			   snapshot->vm_nodes[i].vma.end -
> +			   snapshot->vm_nodes[i].vma.start + 1ull,
> +			   snapshot->vm_nodes[i].addr,
> +			   snapshot->vm_nodes[i].is_userptr ?
> +			   "USR" : snapshot->vm_nodes[i].is_vram ?
> +			   "VRAM" : "SYS");
> +}
> +
> +/**
> + * xe_vm_snapshot_free - Free all allocated objects for a given snapshot.
> + * @snapshot: Xe VM snapshot object.
> + *
> + * This function free all the memory that needed to be allocated at capture
> + * time.
> + */
> +void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot)
> +{
> +	if (!snapshot)
> +		return;
> +
> +	if (snapshot->vm_root)
> +		kfree(snapshot->vm_root);
> +	if (snapshot->vm_nodes)
> +		kfree(snapshot->vm_nodes);
> +	kfree(snapshot);
> +}
> +
> +/**
> + * xe_vm_print - Xe VM Print.
> + * @p: drm_printer
> + * @vm: Xe VM
> + * @gt_id: GT id number
> + *
> + * This function quickly capture a snapshot and immediately print it out.
> + */
> +void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id)
> +{
> +	struct xe_vm_snapshot *snapshot;
> +
> +	snapshot = xe_vm_snapshot_capture(vm, gt_id);
> +	xe_vm_snapshot_print(snapshot, p);
> +	xe_vm_snapshot_free(snapshot);
>  }
> diff --git a/drivers/gpu/drm/xe/xe_vm.h b/drivers/gpu/drm/xe/xe_vm.h
> index 748dc16ebed9..924884b36469 100644
> --- a/drivers/gpu/drm/xe/xe_vm.h
> +++ b/drivers/gpu/drm/xe/xe_vm.h
> @@ -145,7 +145,11 @@ void xe_vm_unlock_dma_resv(struct xe_vm *vm,
>  void xe_vm_fence_all_extobjs(struct xe_vm *vm, struct dma_fence *fence,
>  			     enum dma_resv_usage usage);
>  
> -int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);
> +struct xe_vm_snapshot *xe_vm_snapshot_capture(struct xe_vm *vm, int gt_id);
> +void xe_vm_snapshot_print(struct xe_vm_snapshot *snapshot,
> +			  struct drm_printer *p);
> +void xe_vm_snapshot_free(struct xe_vm_snapshot *snapshot);
> +void xe_vm_print(struct drm_printer *p, struct xe_vm *vm, int gt_id);
>  
>  #if IS_ENABLED(CONFIG_DRM_XE_DEBUG_VM)
>  #define vm_dbg drm_dbg
> diff --git a/drivers/gpu/drm/xe/xe_vm_types.h b/drivers/gpu/drm/xe/xe_vm_types.h
> index fada7896867f..18e79b6a2182 100644
> --- a/drivers/gpu/drm/xe/xe_vm_types.h
> +++ b/drivers/gpu/drm/xe/xe_vm_types.h
> @@ -149,6 +149,24 @@ struct xe_vma {
>  	} extobj;
>  };
>  
> +
> +struct vm_node_snapshot {
> +	bool is_userptr;
> +	bool is_vram;
> +	struct {
> +		u64 start;
> +		u64 end;
> +	} vma;
> +	u64 addr;
> +};
> +
> +struct xe_vm_snapshot {
> +	bool acquired;
> +	struct vm_node_snapshot *vm_root;
> +	struct vm_node_snapshot *vm_nodes;
> +	int num_nodes;
> +};
> +
>  struct xe_device;
>  
>  #define xe_vm_assert_held(vm) dma_resv_assert_held(&(vm)->resv)
> -- 
> 2.39.2
> 

[PATCH 14/14] drm/xe: Add VM snapshot to xe_devcoredump.

[Rodrigo Vivi]

With this patch, we now have some parity between xe_devcoredump
and the simple_error_capture. The only difference is that
xe_devcoredump will only stash the 'first' hang, which is the one
that we care most and should analyze first, while
simple_error_capture will dump them all the kernel log.

But this is just a start point to start building a useful and
organized crash dump, using standard infrastructure. Later this
will be changed to have output that can be parsed by tools and
used for error replay.

Also, it is important to highlight that the goal is not to replace
the simple_error_capture which is still useful for some cases.
But simple_error_capture should be protected under DEBUG and
EXPERT flags, while the devcoredump has its own production config
and will be useful for bug reporting and for error replay.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 6 ++++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h | 3 +++
 2 files changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index 1ffd12646a99..9dbafd586fbd 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -16,6 +16,7 @@
 #include "xe_guc_ct.h"
 #include "xe_guc_submit.h"
 #include "xe_hw_engine.h"
+#include "xe_vm.h"
 
 /**
  * DOC: Xe device coredump
@@ -103,6 +104,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	for_each_hw_engine(hwe, e->gt, id)
 		xe_hw_engine_snapshot_print(coredump->snapshot.hwe[id], &p);
 
+	drm_printf(&p, "\n**** VM ****\n");
+	xe_vm_snapshot_print(coredump->snapshot.vm, &p);
+
 	mutex_unlock(&coredump->lock);
 
 	return count - iter.remain;
@@ -124,6 +128,7 @@ static void xe_devcoredump_free(void *data)
 	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
 	for_each_hw_engine(hwe, coredump->faulty_engine->gt, id)
 		xe_hw_engine_snapshot_free(coredump->snapshot.hwe[id]);
+	xe_vm_snapshot_free(coredump->snapshot.vm);
 
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
@@ -172,6 +177,7 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 		coredump->snapshot.hwe[id] = xe_hw_engine_snapshot_capture(hwe);
 	}
 
+	coredump->snapshot.vm = xe_vm_snapshot_capture(e->vm, e->gt->info.id);
 	xe_force_wake_put(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
 	dma_fence_end_signalling(cookie);
 }
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index 8b17ecf1b6e6..f508eca292f7 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -31,8 +31,11 @@ struct xe_devcoredump_snapshot {
 	struct xe_guc_ct_snapshot *ct;
 	/** @ge: Guc Engine snapshot */
 	struct xe_guc_submit_engine_snapshot *ge;
+
 	/** @hwe: HW Engine snapshot array */
 	struct xe_hw_engine_snapshot *hwe[XE_NUM_HW_ENGINES];
+	/** @vm: VM snapshot */
+	struct xe_vm_snapshot *vm;
 };
 
 /**
-- 
2.39.2

[Intel-xe] [PATCH 14/14] drm/xe: Add VM snapshot to xe_devcoredump.

[Rodrigo Vivi]

With this patch, we now have some parity between xe_devcoredump
and the simple_error_capture. The only difference is that
xe_devcoredump will only stash the 'first' hang, which is the one
that we care most and should analyze first, while
simple_error_capture will dump them all the kernel log.

But this is just a start point to start building a useful and
organized crash dump, using standard infrastructure. Later this
will be changed to have output that can be parsed by tools and
used for error replay.

Also, it is important to highlight that the goal is not to replace
the simple_error_capture which is still useful for some cases.
But simple_error_capture should be protected under DEBUG and
EXPERT flags, while the devcoredump has its own production config
and will be useful for bug reporting and for error replay.

Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
---
 drivers/gpu/drm/xe/xe_devcoredump.c       | 6 ++++++
 drivers/gpu/drm/xe/xe_devcoredump_types.h | 3 +++
 2 files changed, 9 insertions(+)

diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
index 1ffd12646a99..9dbafd586fbd 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump.c
+++ b/drivers/gpu/drm/xe/xe_devcoredump.c
@@ -16,6 +16,7 @@
 #include "xe_guc_ct.h"
 #include "xe_guc_submit.h"
 #include "xe_hw_engine.h"
+#include "xe_vm.h"
 
 /**
  * DOC: Xe device coredump
@@ -103,6 +104,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
 	for_each_hw_engine(hwe, e->gt, id)
 		xe_hw_engine_snapshot_print(coredump->snapshot.hwe[id], &p);
 
+	drm_printf(&p, "\n**** VM ****\n");
+	xe_vm_snapshot_print(coredump->snapshot.vm, &p);
+
 	mutex_unlock(&coredump->lock);
 
 	return count - iter.remain;
@@ -124,6 +128,7 @@ static void xe_devcoredump_free(void *data)
 	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
 	for_each_hw_engine(hwe, coredump->faulty_engine->gt, id)
 		xe_hw_engine_snapshot_free(coredump->snapshot.hwe[id]);
+	xe_vm_snapshot_free(coredump->snapshot.vm);
 
 	coredump->faulty_engine = NULL;
 	drm_info(&coredump_to_xe(coredump)->drm,
@@ -172,6 +177,7 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump)
 		coredump->snapshot.hwe[id] = xe_hw_engine_snapshot_capture(hwe);
 	}
 
+	coredump->snapshot.vm = xe_vm_snapshot_capture(e->vm, e->gt->info.id);
 	xe_force_wake_put(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
 	dma_fence_end_signalling(cookie);
 }
diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
index 8b17ecf1b6e6..f508eca292f7 100644
--- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
+++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
@@ -31,8 +31,11 @@ struct xe_devcoredump_snapshot {
 	struct xe_guc_ct_snapshot *ct;
 	/** @ge: Guc Engine snapshot */
 	struct xe_guc_submit_engine_snapshot *ge;
+
 	/** @hwe: HW Engine snapshot array */
 	struct xe_hw_engine_snapshot *hwe[XE_NUM_HW_ENGINES];
+	/** @vm: VM snapshot */
+	struct xe_vm_snapshot *vm;
 };
 
 /**
-- 
2.39.2

Re: [Intel-xe] [PATCH 14/14] drm/xe: Add VM snapshot to xe_devcoredump.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:57:13PM -0400, Rodrigo Vivi wrote:
> With this patch, we now have some parity between xe_devcoredump
> and the simple_error_capture. The only difference is that
> xe_devcoredump will only stash the 'first' hang, which is the one
> that we care most and should analyze first, while
> simple_error_capture will dump them all the kernel log.
> 
> But this is just a start point to start building a useful and
> organized crash dump, using standard infrastructure. Later this
> will be changed to have output that can be parsed by tools and
> used for error replay.
> 
> Also, it is important to highlight that the goal is not to replace
> the simple_error_capture which is still useful for some cases.
> But simple_error_capture should be protected under DEBUG and
> EXPERT flags, while the devcoredump has its own production config
> and will be useful for bug reporting and for error replay.
> 
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>

Again maybe hold this off after GPUVA but LGTM. Also 1 nit below.

Reviewed-by: Matthew Brost <matthew.brost@intel.com>

> ---
>  drivers/gpu/drm/xe/xe_devcoredump.c       | 6 ++++++
>  drivers/gpu/drm/xe/xe_devcoredump_types.h | 3 +++
>  2 files changed, 9 insertions(+)
> 
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump.c b/drivers/gpu/drm/xe/xe_devcoredump.c
> index 1ffd12646a99..9dbafd586fbd 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump.c
> +++ b/drivers/gpu/drm/xe/xe_devcoredump.c
> @@ -16,6 +16,7 @@
>  #include "xe_guc_ct.h"
>  #include "xe_guc_submit.h"
>  #include "xe_hw_engine.h"
> +#include "xe_vm.h"
>  
>  /**
>   * DOC: Xe device coredump
> @@ -103,6 +104,9 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset,
>  	for_each_hw_engine(hwe, e->gt, id)
>  		xe_hw_engine_snapshot_print(coredump->snapshot.hwe[id], &p);
>  
> +	drm_printf(&p, "\n**** VM ****\n");
> +	xe_vm_snapshot_print(coredump->snapshot.vm, &p);
> +
>  	mutex_unlock(&coredump->lock);
>  
>  	return count - iter.remain;
> @@ -124,6 +128,7 @@ static void xe_devcoredump_free(void *data)
>  	xe_guc_engine_snapshot_free(coredump->snapshot.ge);
>  	for_each_hw_engine(hwe, coredump->faulty_engine->gt, id)
>  		xe_hw_engine_snapshot_free(coredump->snapshot.hwe[id]);
> +	xe_vm_snapshot_free(coredump->snapshot.vm);
>  
>  	coredump->faulty_engine = NULL;
>  	drm_info(&coredump_to_xe(coredump)->drm,
> @@ -172,6 +177,7 @@ static void devcoredump_snapshot(struct xe_devcoredump *coredump)
>  		coredump->snapshot.hwe[id] = xe_hw_engine_snapshot_capture(hwe);
>  	}
>  
> +	coredump->snapshot.vm = xe_vm_snapshot_capture(e->vm, e->gt->info.id);
>  	xe_force_wake_put(gt_to_fw(e->gt), XE_FORCEWAKE_ALL);
>  	dma_fence_end_signalling(cookie);
>  }
> diff --git a/drivers/gpu/drm/xe/xe_devcoredump_types.h b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> index 8b17ecf1b6e6..f508eca292f7 100644
> --- a/drivers/gpu/drm/xe/xe_devcoredump_types.h
> +++ b/drivers/gpu/drm/xe/xe_devcoredump_types.h
> @@ -31,8 +31,11 @@ struct xe_devcoredump_snapshot {
>  	struct xe_guc_ct_snapshot *ct;
>  	/** @ge: Guc Engine snapshot */
>  	struct xe_guc_submit_engine_snapshot *ge;
> +

Nit extra newline.

>  	/** @hwe: HW Engine snapshot array */
>  	struct xe_hw_engine_snapshot *hwe[XE_NUM_HW_ENGINES];
> +	/** @vm: VM snapshot */
> +	struct xe_vm_snapshot *vm;
>  };
>  
>  /**
> -- 
> 2.39.2
> 

[Intel-xe] ✓ CI.Patch_applied: success for Introduce xe_devcoredump.

[Patchwork]

== Series Details ==

Series: Introduce xe_devcoredump.
URL   : https://patchwork.freedesktop.org/series/117017/
State : success

== Summary ==

=== Applying kernel patches on branch 'drm-xe-next' with base: ===
Base commit: fd98c167b drm/xe: Limit the system memory size to half of the system memory
=== git am output follows ===
Applying: drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI
Applying: drm/xe: Introduce the dev_coredump infrastructure.
Applying: drm/xe: Do not take any action if our device was removed.
Applying: drm/xe: Extract non mapped regions out of GuC CTB into its own struct.
Applying: drm/xe: Convert GuC CT print to snapshot capture and print.
Applying: drm/xe: Add GuC CT snapshot to xe_devcoredump.
Applying: drm/xe: Introduce guc_submit_types.h with relevant structs.
Applying: drm/xe: Convert GuC Engine print to snapshot capture and print.
Applying: drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.
Applying: drm/xe: Convert Xe HW Engine print to snapshot capture and print.
Applying: drm/xe: Add HW Engine snapshot to xe_devcoredump.
Applying: drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.
Applying: drm/xe: Convert VM print to snapshot capture and print.
Applying: drm/xe: Add VM snapshot to xe_devcoredump.

[Intel-xe] ✓ CI.KUnit: success for Introduce xe_devcoredump.

[Patchwork]

== Series Details ==

Series: Introduce xe_devcoredump.
URL   : https://patchwork.freedesktop.org/series/117017/
State : success

== Summary ==

+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
stty: 'standard input': Inappropriate ioctl for device
[21:01:28] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[21:01:32] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
[21:01:54] Starting KUnit Kernel (1/1)...
[21:01:54] ============================================================
[21:01:54] ==================== xe_bo (2 subtests) ====================
[21:01:54] [SKIPPED] xe_ccs_migrate_kunit
[21:01:54] [SKIPPED] xe_bo_evict_kunit
[21:01:54] ===================== [SKIPPED] xe_bo ======================
[21:01:54] ================== xe_dma_buf (1 subtest) ==================
[21:01:54] [SKIPPED] xe_dma_buf_kunit
[21:01:54] =================== [SKIPPED] xe_dma_buf ===================
[21:01:54] ================== xe_migrate (1 subtest) ==================
[21:01:54] [SKIPPED] xe_migrate_sanity_kunit
[21:01:54] =================== [SKIPPED] xe_migrate ===================
[21:01:54] =================== xe_pci (2 subtests) ====================
[21:01:54] [PASSED] xe_gmdid_graphics_ip
[21:01:54] [PASSED] xe_gmdid_media_ip
[21:01:54] ===================== [PASSED] xe_pci ======================
[21:01:54] ==================== xe_rtp (1 subtest) ====================
[21:01:54] ================== xe_rtp_process_tests  ===================
[21:01:54] [PASSED] coalesce-same-reg
[21:01:54] [PASSED] no-match-no-add
[21:01:54] [PASSED] no-match-no-add-multiple-rules
[21:01:54] [PASSED] two-regs-two-entries
[21:01:54] [PASSED] clr-one-set-other
[21:01:54] [PASSED] set-field
[21:01:54] [PASSED] conflict-duplicate
[21:01:54] [PASSED] conflict-not-disjoint
[21:01:54] [PASSED] conflict-reg-type
[21:01:54] ============== [PASSED] xe_rtp_process_tests ===============
[21:01:54] ===================== [PASSED] xe_rtp ======================
[21:01:54] ==================== xe_wa (1 subtest) =====================
[21:01:54] ======================== xe_wa_gt  =========================
[21:01:54] [PASSED] TIGERLAKE (B0)
[21:01:54] [PASSED] DG1 (A0)
[21:01:54] [PASSED] DG1 (B0)
[21:01:54] [PASSED] ALDERLAKE_S (A0)
[21:01:54] [PASSED] ALDERLAKE_S (B0)
[21:01:54] [PASSED] ALDERLAKE_S (C0)
[21:01:54] [PASSED] ALDERLAKE_S (D0)
[21:01:54] [PASSED] DG2_G10 (A0)
[21:01:54] [PASSED] DG2_G10 (A1)
[21:01:54] [PASSED] DG2_G10 (B0)
[21:01:54] [PASSED] DG2_G10 (C0)
[21:01:54] [PASSED] DG2_G11 (A0)
[21:01:54] [PASSED] DG2_G11 (B0)
[21:01:54] [PASSED] DG2_G11 (B1)
[21:01:54] [PASSED] DG2_G12 (A0)
[21:01:54] [PASSED] DG2_G12 (A1)
[21:01:54] [PASSED] PVC (B0)
[21:01:54] [PASSED] PVC (B1)
[21:01:54] [PASSED] PVC (C0)
[21:01:54] ==================== [PASSED] xe_wa_gt =====================
[21:01:54] ====================== [PASSED] xe_wa ======================
[21:01:54] ============================================================
[21:01:54] Testing complete. Ran 34 tests: passed: 30, skipped: 4
[21:01:54] Elapsed time: 26.241s total, 4.191s configuring, 21.930s building, 0.097s running

+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/tests/.kunitconfig
[21:01:54] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
[21:01:56] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
[21:02:15] Starting KUnit Kernel (1/1)...
[21:02:15] ============================================================
[21:02:15] ============ drm_test_pick_cmdline (2 subtests) ============
[21:02:15] [PASSED] drm_test_pick_cmdline_res_1920_1080_60
[21:02:15] =============== drm_test_pick_cmdline_named  ===============
[21:02:15] [PASSED] NTSC
[21:02:15] [PASSED] NTSC-J
[21:02:15] [PASSED] PAL
[21:02:15] [PASSED] PAL-M
[21:02:15] =========== [PASSED] drm_test_pick_cmdline_named ===========
[21:02:15] ============== [PASSED] drm_test_pick_cmdline ==============
[21:02:15] ================== drm_buddy (6 subtests) ==================
[21:02:15] [PASSED] drm_test_buddy_alloc_limit
[21:02:15] [PASSED] drm_test_buddy_alloc_range
[21:02:15] [PASSED] drm_test_buddy_alloc_optimistic
[21:02:15] [PASSED] drm_test_buddy_alloc_pessimistic
[21:02:15] [PASSED] drm_test_buddy_alloc_smoke
[21:02:15] [PASSED] drm_test_buddy_alloc_pathological
[21:02:15] ==================== [PASSED] drm_buddy ====================
[21:02:15] ============= drm_cmdline_parser (40 subtests) =============
[21:02:15] [PASSED] drm_test_cmdline_force_d_only
[21:02:15] [PASSED] drm_test_cmdline_force_D_only_dvi
[21:02:15] [PASSED] drm_test_cmdline_force_D_only_hdmi
[21:02:15] [PASSED] drm_test_cmdline_force_D_only_not_digital
[21:02:15] [PASSED] drm_test_cmdline_force_e_only
[21:02:15] [PASSED] drm_test_cmdline_res
[21:02:15] [PASSED] drm_test_cmdline_res_vesa
[21:02:15] [PASSED] drm_test_cmdline_res_vesa_rblank
[21:02:15] [PASSED] drm_test_cmdline_res_rblank
[21:02:15] [PASSED] drm_test_cmdline_res_bpp
[21:02:15] [PASSED] drm_test_cmdline_res_refresh
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh_margins
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh_force_off
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_analog
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_digital
[21:02:15] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced_margins_force_on
[21:02:15] [PASSED] drm_test_cmdline_res_margins_force_on
[21:02:15] [PASSED] drm_test_cmdline_res_vesa_margins
[21:02:15] [PASSED] drm_test_cmdline_name
[21:02:15] [PASSED] drm_test_cmdline_name_bpp
[21:02:15] [PASSED] drm_test_cmdline_name_option
[21:02:15] [PASSED] drm_test_cmdline_name_bpp_option
[21:02:15] [PASSED] drm_test_cmdline_rotate_0
[21:02:15] [PASSED] drm_test_cmdline_rotate_90
[21:02:15] [PASSED] drm_test_cmdline_rotate_180
[21:02:15] [PASSED] drm_test_cmdline_rotate_270
[21:02:15] [PASSED] drm_test_cmdline_hmirror
[21:02:15] [PASSED] drm_test_cmdline_vmirror
[21:02:15] [PASSED] drm_test_cmdline_margin_options
[21:02:15] [PASSED] drm_test_cmdline_multiple_options
[21:02:15] [PASSED] drm_test_cmdline_bpp_extra_and_option
[21:02:15] [PASSED] drm_test_cmdline_extra_and_option
[21:02:15] [PASSED] drm_test_cmdline_freestanding_options
[21:02:15] [PASSED] drm_test_cmdline_freestanding_force_e_and_options
[21:02:15] [PASSED] drm_test_cmdline_panel_orientation
[21:02:15] ================ drm_test_cmdline_invalid  =================
[21:02:15] [PASSED] margin_only
[21:02:15] [PASSED] interlace_only
[21:02:15] [PASSED] res_missing_x
[21:02:15] [PASSED] res_missing_y
[21:02:15] [PASSED] res_bad_y
[21:02:15] [PASSED] res_missing_y_bpp
[21:02:15] [PASSED] res_bad_bpp
[21:02:15] [PASSED] res_bad_refresh
[21:02:15] [PASSED] res_bpp_refresh_force_on_off
[21:02:15] [PASSED] res_invalid_mode
[21:02:15] [PASSED] res_bpp_wrong_place_mode
[21:02:15] [PASSED] name_bpp_refresh
[21:02:15] [PASSED] name_refresh
[21:02:15] [PASSED] name_refresh_wrong_mode
[21:02:15] [PASSED] name_refresh_invalid_mode
[21:02:15] [PASSED] rotate_multiple
[21:02:15] [PASSED] rotate_invalid_val
[21:02:15] [PASSED] rotate_truncated
[21:02:15] [PASSED] invalid_option
[21:02:15] [PASSED] invalid_tv_option
[21:02:15] [PASSED] truncated_tv_option
[21:02:15] ============ [PASSED] drm_test_cmdline_invalid =============
[21:02:15] =============== drm_test_cmdline_tv_options  ===============
[21:02:15] [PASSED] NTSC
[21:02:15] [PASSED] NTSC_443
[21:02:15] [PASSED] NTSC_J
[21:02:15] [PASSED] PAL
[21:02:15] [PASSED] PAL_M
[21:02:15] [PASSED] PAL_N
[21:02:15] [PASSED] SECAM
[21:02:15] =========== [PASSED] drm_test_cmdline_tv_options ===========
[21:02:15] =============== [PASSED] drm_cmdline_parser ================
[21:02:15] ========== drm_get_tv_mode_from_name (2 subtests) ==========
[21:02:15] ========== drm_test_get_tv_mode_from_name_valid  ===========
[21:02:15] [PASSED] NTSC
[21:02:15] [PASSED] NTSC-443
[21:02:15] [PASSED] NTSC-J
[21:02:15] [PASSED] PAL
[21:02:15] [PASSED] PAL-M
[21:02:15] [PASSED] PAL-N
[21:02:15] [PASSED] SECAM
[21:02:15] ====== [PASSED] drm_test_get_tv_mode_from_name_valid =======
[21:02:15] [PASSED] drm_test_get_tv_mode_from_name_truncated
[21:02:15] ============ [PASSED] drm_get_tv_mode_from_name ============
[21:02:15] ============= drm_damage_helper (21 subtests) ==============
[21:02:15] [PASSED] drm_test_damage_iter_no_damage
[21:02:15] [PASSED] drm_test_damage_iter_no_damage_fractional_src
[21:02:15] [PASSED] drm_test_damage_iter_no_damage_src_moved
[21:02:15] [PASSED] drm_test_damage_iter_no_damage_fractional_src_moved
[21:02:15] [PASSED] drm_test_damage_iter_no_damage_not_visible
[21:02:15] [PASSED] drm_test_damage_iter_no_damage_no_crtc
[21:02:15] [PASSED] drm_test_damage_iter_no_damage_no_fb
[21:02:15] [PASSED] drm_test_damage_iter_simple_damage
[21:02:15] [PASSED] drm_test_damage_iter_single_damage
[21:02:15] [PASSED] drm_test_damage_iter_single_damage_intersect_src
[21:02:15] [PASSED] drm_test_damage_iter_single_damage_outside_src
[21:02:15] [PASSED] drm_test_damage_iter_single_damage_fractional_src
[21:02:15] [PASSED] drm_test_damage_iter_single_damage_intersect_fractional_src
[21:02:15] [PASSED] drm_test_damage_iter_single_damage_outside_fractional_src
[21:02:15] [PASSED] drm_test_damage_iter_single_damage_src_moved
[21:02:15] [PASSED] drm_test_damage_iter_single_damage_fractional_src_moved
[21:02:15] [PASSED] drm_test_damage_iter_damage
[21:02:15] [PASSED] drm_test_damage_iter_damage_one_intersect
[21:02:15] [PASSED] drm_test_damage_iter_damage_one_outside
[21:02:15] [PASSED] drm_test_damage_iter_damage_src_moved
[21:02:15] [PASSED] drm_test_damage_iter_damage_not_visible
[21:02:15] ================ [PASSED] drm_damage_helper ================
[21:02:15] ============== drm_dp_mst_helper (2 subtests) ==============
[21:02:15] ============== drm_test_dp_mst_calc_pbn_mode  ==============
[21:02:15] [PASSED] Clock 154000 BPP 30 DSC disabled
[21:02:15] [PASSED] Clock 234000 BPP 30 DSC disabled
[21:02:15] [PASSED] Clock 297000 BPP 24 DSC disabled
[21:02:15] [PASSED] Clock 332880 BPP 24 DSC enabled
[21:02:15] [PASSED] Clock 324540 BPP 24 DSC enabled
[21:02:15] ========== [PASSED] drm_test_dp_mst_calc_pbn_mode ==========
[21:02:15] ========= drm_test_dp_mst_sideband_msg_req_decode  =========
[21:02:15] [PASSED] DP_ENUM_PATH_RESOURCES with port number
[21:02:15] [PASSED] DP_POWER_UP_PHY with port number
[21:02:15] [PASSED] DP_POWER_DOWN_PHY with port number
[21:02:15] [PASSED] DP_ALLOCATE_PAYLOAD with SDP stream sinks
[21:02:15] [PASSED] DP_ALLOCATE_PAYLOAD with port number
[21:02:15] [PASSED] DP_ALLOCATE_PAYLOAD with VCPI
[21:02:15] [PASSED] DP_ALLOCATE_PAYLOAD with PBN
[21:02:15] [PASSED] DP_QUERY_PAYLOAD with port number
[21:02:15] [PASSED] DP_QUERY_PAYLOAD with VCPI
[21:02:15] [PASSED] DP_REMOTE_DPCD_READ with port number
[21:02:15] [PASSED] DP_REMOTE_DPCD_READ with DPCD address
[21:02:15] [PASSED] DP_REMOTE_DPCD_READ with max number of bytes
[21:02:15] [PASSED] DP_REMOTE_DPCD_WRITE with port number
[21:02:15] [PASSED] DP_REMOTE_DPCD_WRITE with DPCD address
[21:02:15] [PASSED] DP_REMOTE_DPCD_WRITE with data array
[21:02:15] [PASSED] DP_REMOTE_I2C_READ with port number
[21:02:15] [PASSED] DP_REMOTE_I2C_READ with I2C device ID
[21:02:15] [PASSED] DP_REMOTE_I2C_READ with transactions array
[21:02:15] [PASSED] DP_REMOTE_I2C_WRITE with port number
[21:02:15] [PASSED] DP_REMOTE_I2C_WRITE with I2C device ID
[21:02:15] [PASSED] DP_REMOTE_I2C_WRITE with data array
[21:02:15] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream ID
[21:02:15] [PASSED] DP_QUERY_STREAM_ENC_STATUS with client ID
[21:02:15] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream event
[21:02:15] [PASSED] DP_QUERY_STREAM_ENC_STATUS with valid stream event
[21:02:15] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream behavior
[21:02:15] [PASSED] DP_QUERY_STREAM_ENC_STATUS with a valid stream behavior
[21:02:15] ===== [PASSED] drm_test_dp_mst_sideband_msg_req_decode =====
[21:02:15] ================ [PASSED] drm_dp_mst_helper ================
[21:02:15] =========== drm_format_helper_test (11 subtests) ===========
[21:02:15] ============== drm_test_fb_xrgb8888_to_gray8  ==============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ========== [PASSED] drm_test_fb_xrgb8888_to_gray8 ==========
[21:02:15] ============= drm_test_fb_xrgb8888_to_rgb332  ==============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb332 ==========
[21:02:15] ============= drm_test_fb_xrgb8888_to_rgb565  ==============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb565 ==========
[21:02:15] ============ drm_test_fb_xrgb8888_to_xrgb1555  =============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ======== [PASSED] drm_test_fb_xrgb8888_to_xrgb1555 =========
[21:02:15] ============ drm_test_fb_xrgb8888_to_argb1555  =============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ======== [PASSED] drm_test_fb_xrgb8888_to_argb1555 =========
[21:02:15] ============ drm_test_fb_xrgb8888_to_rgba5551  =============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ======== [PASSED] drm_test_fb_xrgb8888_to_rgba5551 =========
[21:02:15] ============= drm_test_fb_xrgb8888_to_rgb888  ==============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb888 ==========
[21:02:15] ============ drm_test_fb_xrgb8888_to_argb8888  =============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ======== [PASSED] drm_test_fb_xrgb8888_to_argb8888 =========
[21:02:15] =========== drm_test_fb_xrgb8888_to_xrgb2101010  ===========
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ======= [PASSED] drm_test_fb_xrgb8888_to_xrgb2101010 =======
[21:02:15] =========== drm_test_fb_xrgb8888_to_argb2101010  ===========
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ======= [PASSED] drm_test_fb_xrgb8888_to_argb2101010 =======
[21:02:15] ============== drm_test_fb_xrgb8888_to_mono  ===============
[21:02:15] [PASSED] single_pixel_source_buffer
[21:02:15] [PASSED] single_pixel_clip_rectangle
[21:02:15] [PASSED] well_known_colors
[21:02:15] [PASSED] destination_pitch
[21:02:15] ========== [PASSED] drm_test_fb_xrgb8888_to_mono ===========
[21:02:15] ============= [PASSED] drm_format_helper_test ==============
[21:02:15] ================= drm_format (18 subtests) =================
[21:02:15] [PASSED] drm_test_format_block_width_invalid
[21:02:15] [PASSED] drm_test_format_block_width_one_plane
[21:02:15] [PASSED] drm_test_format_block_width_two_plane
[21:02:15] [PASSED] drm_test_format_block_width_three_plane
[21:02:15] [PASSED] drm_test_format_block_width_tiled
[21:02:15] [PASSED] drm_test_format_block_height_invalid
[21:02:15] [PASSED] drm_test_format_block_height_one_plane
[21:02:15] [PASSED] drm_test_format_block_height_two_plane
[21:02:15] [PASSED] drm_test_format_block_height_three_plane
[21:02:15] [PASSED] drm_test_format_block_height_tiled
[21:02:15] [PASSED] drm_test_format_min_pitch_invalid
[21:02:15] [PASSED] drm_test_format_min_pitch_one_plane_8bpp
[21:02:15] [PASSED] drm_test_format_min_pitch_one_plane_16bpp
[21:02:15] [PASSED] drm_test_format_min_pitch_one_plane_24bpp
[21:02:15] [PASSED] drm_test_format_min_pitch_one_plane_32bpp
[21:02:15] [PASSED] drm_test_format_min_pitch_two_plane
[21:02:15] [PASSED] drm_test_format_min_pitch_three_plane_8bpp
[21:02:15] [PASSED] drm_test_format_min_pitch_tiled
[21:02:15] =================== [PASSED] drm_format ====================
[21:02:15] =============== drm_framebuffer (1 subtest) ================
[21:02:15] =============== drm_test_framebuffer_create  ===============
[21:02:15] [PASSED] ABGR8888 normal sizes
[21:02:15] [PASSED] ABGR8888 max sizes
[21:02:15] [PASSED] ABGR8888 pitch greater than min required
[21:02:15] [PASSED] ABGR8888 pitch less than min required
[21:02:15] [PASSED] ABGR8888 Invalid width
[21:02:15] [PASSED] ABGR8888 Invalid buffer handle
[21:02:15] [PASSED] No pixel format
[21:02:15] [PASSED] ABGR8888 Width 0
[21:02:15] [PASSED] ABGR8888 Height 0
[21:02:15] [PASSED] ABGR8888 Out of bound height * pitch combination
[21:02:15] [PASSED] ABGR8888 Large buffer offset
[21:02:15] [PASSED] ABGR8888 Set DRM_MODE_FB_MODIFIERS without modifiers
[21:02:15] [PASSED] ABGR8888 Valid buffer modifier
[21:02:15] [PASSED] ABGR8888 Invalid buffer modifier(DRM_FORMAT_MOD_SAMSUNG_64_32_TILE)
[21:02:15] [PASSED] ABGR8888 Extra pitches without DRM_MODE_FB_MODIFIERS
[21:02:15] [PASSED] ABGR8888 Extra pitches with DRM_MODE_FB_MODIFIERS
[21:02:15] [PASSED] NV12 Normal sizes
[21:02:15] [PASSED] NV12 Max sizes
[21:02:15] [PASSED] NV12 Invalid pitch
[21:02:15] [PASSED] NV12 Invalid modifier/missing DRM_MODE_FB_MODIFIERS flag
[21:02:15] [PASSED] NV12 different  modifier per-plane
[21:02:15] [PASSED] NV12 with DRM_FORMAT_MOD_SAMSUNG_64_32_TILE
[21:02:15] [PASSED] NV12 Valid modifiers without DRM_MODE_FB_MODIFIERS
[21:02:15] [PASSED] NV12 Modifier for inexistent plane
[21:02:15] [PASSED] NV12 Handle for inexistent plane
[21:02:15] [PASSED] NV12 Handle for inexistent plane without DRM_MODE_FB_MODIFIERS
[21:02:15] [PASSED] YVU420 Normal sizes
[21:02:15] [PASSED] YVU420 DRM_MODE_FB_MODIFIERS set without modifier
[21:02:15] [PASSED] YVU420 Max sizes
[21:02:15] [PASSED] YVU420 Invalid pitch
[21:02:15] [PASSED] YVU420 Different pitches
[21:02:15] [PASSED] YVU420 Different buffer offsets/pitches
[21:02:15] [PASSED] YVU420 Modifier set just for plane 0, without DRM_MODE_FB_MODIFIERS
[21:02:15] [PASSED] YVU420 Modifier set just for planes 0, 1, without DRM_MODE_FB_MODIFIERS
[21:02:15] [PASSED] YVU420 Modifier set just for plane 0, 1, with DRM_MODE_FB_MODIFIERS
[21:02:15] [PASSED] YVU420 Valid modifier
[21:02:15] [PASSED] YVU420 Different modifiers per plane
[21:02:15] [PASSED] YVU420 Modifier for inexistent plane
[21:02:15] [PASSED] X0L2 Normal sizes
[21:02:15] [PASSED] X0L2 Max sizes
[21:02:15] [PASSED] X0L2 Invalid pitch
[21:02:15] [PASSED] X0L2 Pitch greater than minimum required
stty: 'standard input': Inappropriate ioctl for device
[21:02:15] [PASSED] X0L2 Handle for inexistent plane
[21:02:15] [PASSED] X0L2 Offset for inexistent plane, without DRM_MODE_FB_MODIFIERS set
[21:02:15] [PASSED] X0L2 Modifier without DRM_MODE_FB_MODIFIERS set
[21:02:15] [PASSED] X0L2 Valid modifier
[21:02:15] [PASSED] X0L2 Modifier for inexistent plane
[21:02:15] =========== [PASSED] drm_test_framebuffer_create ===========
[21:02:15] ================= [PASSED] drm_framebuffer =================
[21:02:15] =============== drm-test-managed (1 subtest) ===============
[21:02:15] [PASSED] drm_test_managed_run_action
[21:02:15] ================ [PASSED] drm-test-managed =================
[21:02:15] =================== drm_mm (19 subtests) ===================
[21:02:15] [PASSED] drm_test_mm_init
[21:02:15] [PASSED] drm_test_mm_debug
[21:02:25] [PASSED] drm_test_mm_reserve
[21:02:35] [PASSED] drm_test_mm_insert
[21:02:36] [PASSED] drm_test_mm_replace
[21:02:36] [PASSED] drm_test_mm_insert_range
[21:02:36] [PASSED] drm_test_mm_frag
[21:02:36] [PASSED] drm_test_mm_align
[21:02:36] [PASSED] drm_test_mm_align32
[21:02:36] [PASSED] drm_test_mm_align64
[21:02:37] [PASSED] drm_test_mm_evict
[21:02:37] [PASSED] drm_test_mm_evict_range
[21:02:37] [PASSED] drm_test_mm_topdown
[21:02:37] [PASSED] drm_test_mm_bottomup
[21:02:37] [PASSED] drm_test_mm_lowest
[21:02:37] [PASSED] drm_test_mm_highest
[21:02:37] [PASSED] drm_test_mm_color
[21:02:38] [PASSED] drm_test_mm_color_evict
[21:02:38] [PASSED] drm_test_mm_color_evict_range
[21:02:38] ===================== [PASSED] drm_mm ======================
[21:02:38] ============= drm_modes_analog_tv (4 subtests) =============
[21:02:38] [PASSED] drm_test_modes_analog_tv_ntsc_480i
[21:02:38] [PASSED] drm_test_modes_analog_tv_ntsc_480i_inlined
[21:02:38] [PASSED] drm_test_modes_analog_tv_pal_576i
[21:02:38] [PASSED] drm_test_modes_analog_tv_pal_576i_inlined
[21:02:38] =============== [PASSED] drm_modes_analog_tv ===============
[21:02:38] ============== drm_plane_helper (2 subtests) ===============
[21:02:38] =============== drm_test_check_plane_state  ================
[21:02:38] [PASSED] clipping_simple
[21:02:38] [PASSED] clipping_rotate_reflect
[21:02:38] [PASSED] positioning_simple
[21:02:38] [PASSED] upscaling
[21:02:38] [PASSED] downscaling
[21:02:38] [PASSED] rounding1
[21:02:38] [PASSED] rounding2
[21:02:38] [PASSED] rounding3
[21:02:38] [PASSED] rounding4
[21:02:38] =========== [PASSED] drm_test_check_plane_state ============
[21:02:38] =========== drm_test_check_invalid_plane_state  ============
[21:02:38] [PASSED] positioning_invalid
[21:02:38] [PASSED] upscaling_invalid
[21:02:38] [PASSED] downscaling_invalid
[21:02:38] ======= [PASSED] drm_test_check_invalid_plane_state ========
[21:02:38] ================ [PASSED] drm_plane_helper =================
[21:02:38] ====== drm_connector_helper_tv_get_modes (1 subtest) =======
[21:02:38] ====== drm_test_connector_helper_tv_get_modes_check  =======
[21:02:38] [PASSED] None
[21:02:38] [PASSED] PAL
[21:02:38] [PASSED] NTSC
[21:02:38] [PASSED] Both, NTSC Default
[21:02:38] [PASSED] Both, PAL Default
[21:02:38] [PASSED] Both, NTSC Default, with PAL on command-line
[21:02:38] [PASSED] Both, PAL Default, with NTSC on command-line
[21:02:38] == [PASSED] drm_test_connector_helper_tv_get_modes_check ===
[21:02:38] ======== [PASSED] drm_connector_helper_tv_get_modes ========
[21:02:38] ================== drm_rect (4 subtests) ===================
[21:02:38] [PASSED] drm_test_rect_clip_scaled_div_by_zero
[21:02:38] [PASSED] drm_test_rect_clip_scaled_not_clipped
[21:02:38] [PASSED] drm_test_rect_clip_scaled_clipped
[21:02:38] [PASSED] drm_test_rect_clip_scaled_signed_vs_unsigned
[21:02:38] ==================== [PASSED] drm_rect =====================
[21:02:38] ============================================================
[21:02:38] Testing complete. Ran 294 tests: passed: 294
[21:02:38] Elapsed time: 43.656s total, 1.704s configuring, 18.671s building, 23.272s running

+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel

[Intel-xe] ✓ CI.Build: success for Introduce xe_devcoredump.

[Patchwork]

== Series Details ==

Series: Introduce xe_devcoredump.
URL   : https://patchwork.freedesktop.org/series/117017/
State : success

== Summary ==

+ trap cleanup EXIT
+ cd /kernel
+ git clone https://gitlab.freedesktop.org/drm/xe/ci.git .ci
Cloning into '.ci'...
++ date +%s
+ echo -e '\e[0Ksection_start:1682542968:build_x86_64[collapsed=true]\r\e[0KBuild x86-64'
+ mkdir -p build64
^[[0Ksection_start:1682542968:build_x86_64[collapsed=true]
^[[0KBuild x86-64
+ cat .ci/kernel/kconfig
+ make O=build64 olddefconfig
make[1]: Entering directory '/kernel/build64'
  GEN     Makefile
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/menu.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTCC  scripts/kconfig/preprocess.o
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/util.o
  HOSTLD  scripts/kconfig/conf
#
# configuration written to .config
#
make[1]: Leaving directory '/kernel/build64'
++ nproc
+ make O=build64 -j48
make[1]: Entering directory '/kernel/build64'
  GEN     Makefile
  WRAP    arch/x86/include/generated/uapi/asm/bpf_perf_event.h
  WRAP    arch/x86/include/generated/uapi/asm/errno.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_32.h
  WRAP    arch/x86/include/generated/uapi/asm/fcntl.h
  WRAP    arch/x86/include/generated/uapi/asm/ioctl.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_64.h
  SYSHDR  arch/x86/include/generated/uapi/asm/unistd_x32.h
  WRAP    arch/x86/include/generated/uapi/asm/ioctls.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_64_x32.h
  SYSHDR  arch/x86/include/generated/asm/unistd_32_ia32.h
  WRAP    arch/x86/include/generated/uapi/asm/param.h
  WRAP    arch/x86/include/generated/uapi/asm/ipcbuf.h
  SYSTBL  arch/x86/include/generated/asm/syscalls_64.h
  WRAP    arch/x86/include/generated/uapi/asm/poll.h
  WRAP    arch/x86/include/generated/uapi/asm/resource.h
  WRAP    arch/x86/include/generated/uapi/asm/socket.h
  WRAP    arch/x86/include/generated/uapi/asm/sockios.h
  WRAP    arch/x86/include/generated/uapi/asm/termbits.h
  WRAP    arch/x86/include/generated/uapi/asm/termios.h
  WRAP    arch/x86/include/generated/uapi/asm/types.h
  UPD     include/generated/uapi/linux/version.h
  UPD     include/config/kernel.release
  UPD     include/generated/compile.h
  HOSTCC  arch/x86/tools/relocs_32.o
  HOSTCC  arch/x86/tools/relocs_64.o
  HOSTCC  arch/x86/tools/relocs_common.o
  WRAP    arch/x86/include/generated/asm/early_ioremap.h
  WRAP    arch/x86/include/generated/asm/export.h
  WRAP    arch/x86/include/generated/asm/irq_regs.h
  WRAP    arch/x86/include/generated/asm/mcs_spinlock.h
  WRAP    arch/x86/include/generated/asm/kmap_size.h
  WRAP    arch/x86/include/generated/asm/local64.h
  WRAP    arch/x86/include/generated/asm/mmiowb.h
  WRAP    arch/x86/include/generated/asm/module.lds.h
  WRAP    arch/x86/include/generated/asm/rwonce.h
  WRAP    arch/x86/include/generated/asm/unaligned.h
  HOSTCC  scripts/unifdef
  UPD     include/generated/utsrelease.h
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/sorttable
  HOSTCC  scripts/asn1_compiler
  DESCEND objtool
  HOSTCC  /kernel/build64/tools/objtool/fixdep.o
  HOSTLD  /kernel/build64/tools/objtool/fixdep-in.o
  LINK    /kernel/build64/tools/objtool/fixdep
  INSTALL /kernel/build64/tools/objtool/libsubcmd/include/subcmd/exec-cmd.h
  INSTALL /kernel/build64/tools/objtool/libsubcmd/include/subcmd/help.h
  INSTALL /kernel/build64/tools/objtool/libsubcmd/include/subcmd/pager.h
  INSTALL /kernel/build64/tools/objtool/libsubcmd/include/subcmd/parse-options.h
  INSTALL /kernel/build64/tools/objtool/libsubcmd/include/subcmd/run-command.h
  CC      /kernel/build64/tools/objtool/libsubcmd/exec-cmd.o
  CC      /kernel/build64/tools/objtool/libsubcmd/help.o
  CC      /kernel/build64/tools/objtool/libsubcmd/pager.o
  INSTALL libsubcmd_headers
  CC      /kernel/build64/tools/objtool/libsubcmd/parse-options.o
  CC      /kernel/build64/tools/objtool/libsubcmd/run-command.o
  CC      /kernel/build64/tools/objtool/libsubcmd/sigchain.o
  CC      /kernel/build64/tools/objtool/libsubcmd/subcmd-config.o
  HOSTLD  arch/x86/tools/relocs
  CC      scripts/mod/empty.o
  HOSTCC  scripts/mod/mk_elfconfig
  CC      scripts/mod/devicetable-offsets.s
  HDRINST usr/include/video/edid.h
  HDRINST usr/include/video/sisfb.h
  HDRINST usr/include/video/uvesafb.h
  HDRINST usr/include/drm/i915_drm.h
  HDRINST usr/include/drm/amdgpu_drm.h
  HDRINST usr/include/drm/vgem_drm.h
  HDRINST usr/include/drm/virtgpu_drm.h
  HDRINST usr/include/drm/xe_drm.h
  HDRINST usr/include/drm/omap_drm.h
  HDRINST usr/include/drm/radeon_drm.h
  HDRINST usr/include/drm/tegra_drm.h
  HDRINST usr/include/drm/drm_mode.h
  HDRINST usr/include/drm/ivpu_accel.h
  HDRINST usr/include/drm/exynos_drm.h
  HDRINST usr/include/drm/drm_sarea.h
  HDRINST usr/include/drm/v3d_drm.h
  HDRINST usr/include/drm/qxl_drm.h
  HDRINST usr/include/drm/drm_fourcc.h
  HDRINST usr/include/drm/nouveau_drm.h
  HDRINST usr/include/drm/habanalabs_accel.h
  HDRINST usr/include/drm/vmwgfx_drm.h
  HDRINST usr/include/drm/msm_drm.h
  HDRINST usr/include/drm/etnaviv_drm.h
  HDRINST usr/include/drm/vc4_drm.h
  HDRINST usr/include/drm/lima_drm.h
  HDRINST usr/include/drm/panfrost_drm.h
  HDRINST usr/include/drm/drm.h
  HDRINST usr/include/drm/armada_drm.h
  HDRINST usr/include/mtd/nftl-user.h
  HDRINST usr/include/mtd/inftl-user.h
  HDRINST usr/include/mtd/mtd-user.h
  HDRINST usr/include/mtd/ubi-user.h
  HDRINST usr/include/mtd/mtd-abi.h
  HDRINST usr/include/xen/gntdev.h
  HDRINST usr/include/xen/gntalloc.h
  HDRINST usr/include/xen/evtchn.h
  HDRINST usr/include/xen/privcmd.h
  HDRINST usr/include/asm-generic/auxvec.h
  HDRINST usr/include/asm-generic/bitsperlong.h
  HDRINST usr/include/asm-generic/posix_types.h
  HDRINST usr/include/asm-generic/ioctls.h
  HDRINST usr/include/asm-generic/mman.h
  HDRINST usr/include/asm-generic/shmbuf.h
  HDRINST usr/include/asm-generic/bpf_perf_event.h
  HDRINST usr/include/asm-generic/types.h
  HDRINST usr/include/asm-generic/poll.h
  HDRINST usr/include/asm-generic/msgbuf.h
  HDRINST usr/include/asm-generic/swab.h
  HDRINST usr/include/asm-generic/statfs.h
  HDRINST usr/include/asm-generic/unistd.h
  HDRINST usr/include/asm-generic/hugetlb_encode.h
  HDRINST usr/include/asm-generic/resource.h
  HDRINST usr/include/asm-generic/param.h
  HDRINST usr/include/asm-generic/termbits-common.h
  HDRINST usr/include/asm-generic/sockios.h
  HDRINST usr/include/asm-generic/kvm_para.h
  HDRINST usr/include/asm-generic/errno.h
  HDRINST usr/include/asm-generic/termios.h
  HDRINST usr/include/asm-generic/mman-common.h
  HDRINST usr/include/asm-generic/ioctl.h
  HDRINST usr/include/asm-generic/socket.h
  HDRINST usr/include/asm-generic/signal-defs.h
  HDRINST usr/include/asm-generic/termbits.h
  HDRINST usr/include/asm-generic/int-ll64.h
  UPD     scripts/mod/devicetable-offsets.h
  HDRINST usr/include/asm-generic/signal.h
  HDRINST usr/include/asm-generic/siginfo.h
  HDRINST usr/include/asm-generic/stat.h
  HDRINST usr/include/asm-generic/int-l64.h
  HDRINST usr/include/asm-generic/errno-base.h
  HDRINST usr/include/asm-generic/fcntl.h
  HDRINST usr/include/asm-generic/setup.h
  HDRINST usr/include/asm-generic/ipcbuf.h
  HDRINST usr/include/asm-generic/sembuf.h
  HDRINST usr/include/asm-generic/ucontext.h
  HDRINST usr/include/rdma/mlx5_user_ioctl_cmds.h
  HDRINST usr/include/rdma/irdma-abi.h
  HDRINST usr/include/rdma/hfi/hfi1_user.h
  HDRINST usr/include/rdma/mana-abi.h
  HDRINST usr/include/rdma/hfi/hfi1_ioctl.h
  HDRINST usr/include/rdma/rdma_user_rxe.h
  HDRINST usr/include/rdma/rdma_user_ioctl.h
  HDRINST usr/include/rdma/mlx5_user_ioctl_verbs.h
  HDRINST usr/include/rdma/bnxt_re-abi.h
  HDRINST usr/include/rdma/hns-abi.h
  HDRINST usr/include/rdma/qedr-abi.h
  HDRINST usr/include/rdma/ib_user_ioctl_cmds.h
  HDRINST usr/include/rdma/vmw_pvrdma-abi.h
  HDRINST usr/include/rdma/ib_user_sa.h
  HDRINST usr/include/rdma/ib_user_ioctl_verbs.h
  HDRINST usr/include/rdma/rvt-abi.h
  HDRINST usr/include/rdma/mlx5-abi.h
  HDRINST usr/include/rdma/rdma_netlink.h
  HDRINST usr/include/rdma/erdma-abi.h
  HDRINST usr/include/rdma/rdma_user_ioctl_cmds.h
  HDRINST usr/include/rdma/rdma_user_cm.h
  HDRINST usr/include/rdma/ib_user_verbs.h
  HDRINST usr/include/rdma/efa-abi.h
  HDRINST usr/include/rdma/mlx4-abi.h
  HDRINST usr/include/rdma/siw-abi.h
  HDRINST usr/include/rdma/mthca-abi.h
  HDRINST usr/include/rdma/ib_user_mad.h
  HDRINST usr/include/rdma/ocrdma-abi.h
  HDRINST usr/include/rdma/cxgb4-abi.h
  HDRINST usr/include/misc/xilinx_sdfec.h
  HDRINST usr/include/misc/uacce/hisi_qm.h
  HDRINST usr/include/misc/uacce/uacce.h
  HDRINST usr/include/misc/cxl.h
  HDRINST usr/include/misc/ocxl.h
  HDRINST usr/include/misc/fastrpc.h
  HDRINST usr/include/misc/pvpanic.h
  HDRINST usr/include/linux/i8k.h
  HDRINST usr/include/linux/acct.h
  HDRINST usr/include/linux/atmmpc.h
  HDRINST usr/include/linux/fs.h
  HDRINST usr/include/linux/cifs/cifs_mount.h
  HDRINST usr/include/linux/cifs/cifs_netlink.h
  HDRINST usr/include/linux/if_packet.h
  HDRINST usr/include/linux/route.h
  HDRINST usr/include/linux/patchkey.h
  HDRINST usr/include/linux/tc_ematch/tc_em_cmp.h
  HDRINST usr/include/linux/tc_ematch/tc_em_ipt.h
  HDRINST usr/include/linux/tc_ematch/tc_em_meta.h
  MKELF   scripts/mod/elfconfig.h
  HDRINST usr/include/linux/tc_ematch/tc_em_nbyte.h
  HDRINST usr/include/linux/tc_ematch/tc_em_text.h
  HDRINST usr/include/linux/virtio_pmem.h
  HOSTCC  scripts/mod/modpost.o
  HDRINST usr/include/linux/rkisp1-config.h
  HOSTCC  scripts/mod/file2alias.o
  HDRINST usr/include/linux/vhost.h
  HOSTCC  scripts/mod/sumversion.o
  HDRINST usr/include/linux/cec-funcs.h
  HDRINST usr/include/linux/ppdev.h
  HDRINST usr/include/linux/isdn/capicmd.h
  HDRINST usr/include/linux/virtio_fs.h
  HDRINST usr/include/linux/netfilter_ipv6.h
  HDRINST usr/include/linux/lirc.h
  HDRINST usr/include/linux/mroute6.h
  HDRINST usr/include/linux/nl80211-vnd-intel.h
  HDRINST usr/include/linux/ivtvfb.h
  HDRINST usr/include/linux/auxvec.h
  HDRINST usr/include/linux/dm-log-userspace.h
  HDRINST usr/include/linux/dccp.h
  HDRINST usr/include/linux/virtio_scmi.h
  HDRINST usr/include/linux/atmarp.h
  HDRINST usr/include/linux/arcfb.h
  HDRINST usr/include/linux/nbd-netlink.h
  HDRINST usr/include/linux/sched/types.h
  HDRINST usr/include/linux/tcp.h
  HDRINST usr/include/linux/neighbour.h
  HDRINST usr/include/linux/dlm_device.h
  HDRINST usr/include/linux/wmi.h
  HDRINST usr/include/linux/btrfs_tree.h
  HDRINST usr/include/linux/virtio_crypto.h
  HDRINST usr/include/linux/vbox_err.h
  HDRINST usr/include/linux/edd.h
  HDRINST usr/include/linux/loop.h
  HDRINST usr/include/linux/nvme_ioctl.h
  HDRINST usr/include/linux/mmtimer.h
  HDRINST usr/include/linux/if_pppol2tp.h
  HDRINST usr/include/linux/mtio.h
  HDRINST usr/include/linux/if_arcnet.h
  HDRINST usr/include/linux/romfs_fs.h
  HDRINST usr/include/linux/posix_types.h
  HDRINST usr/include/linux/rtc.h
  HDRINST usr/include/linux/landlock.h
  HDRINST usr/include/linux/gpio.h
  HDRINST usr/include/linux/selinux_netlink.h
  HDRINST usr/include/linux/pps.h
  HDRINST usr/include/linux/ndctl.h
  HDRINST usr/include/linux/virtio_gpu.h
  HDRINST usr/include/linux/android/binderfs.h
  HDRINST usr/include/linux/android/binder.h
  HDRINST usr/include/linux/virtio_vsock.h
  HDRINST usr/include/linux/sound.h
  HDRINST usr/include/linux/vtpm_proxy.h
  HDRINST usr/include/linux/nfs_fs.h
  HDRINST usr/include/linux/elf-fdpic.h
  HDRINST usr/include/linux/adfs_fs.h
  HDRINST usr/include/linux/target_core_user.h
  HDRINST usr/include/linux/netlink_diag.h
  HDRINST usr/include/linux/const.h
  HDRINST usr/include/linux/firewire-cdev.h
  HDRINST usr/include/linux/vdpa.h
  HDRINST usr/include/linux/if_infiniband.h
  HDRINST usr/include/linux/serial.h
  HDRINST usr/include/linux/iio/types.h
  HDRINST usr/include/linux/iio/buffer.h
  HDRINST usr/include/linux/iio/events.h
  HDRINST usr/include/linux/baycom.h
  HDRINST usr/include/linux/major.h
  HDRINST usr/include/linux/atmppp.h
  HDRINST usr/include/linux/ipv6_route.h
  HDRINST usr/include/linux/spi/spidev.h
  HDRINST usr/include/linux/spi/spi.h
  HDRINST usr/include/linux/virtio_ring.h
  HDRINST usr/include/linux/hdlc/ioctl.h
  HDRINST usr/include/linux/remoteproc_cdev.h
  HDRINST usr/include/linux/hyperv.h
  HDRINST usr/include/linux/rpl_iptunnel.h
  HDRINST usr/include/linux/sync_file.h
  HDRINST usr/include/linux/igmp.h
  HDRINST usr/include/linux/v4l2-dv-timings.h
  HDRINST usr/include/linux/virtio_i2c.h
  HDRINST usr/include/linux/xfrm.h
  HDRINST usr/include/linux/capability.h
  HDRINST usr/include/linux/gtp.h
  HDRINST usr/include/linux/xdp_diag.h
  HDRINST usr/include/linux/pkt_cls.h
  HDRINST usr/include/linux/suspend_ioctls.h
  HDRINST usr/include/linux/vt.h
  HDRINST usr/include/linux/loadpin.h
  HDRINST usr/include/linux/dlm_plock.h
  HDRINST usr/include/linux/fb.h
  HDRINST usr/include/linux/max2175.h
  HDRINST usr/include/linux/sunrpc/debug.h
  HDRINST usr/include/linux/gsmmux.h
  HDRINST usr/include/linux/watchdog.h
  HDRINST usr/include/linux/vhost_types.h
  HDRINST usr/include/linux/vduse.h
  HDRINST usr/include/linux/ila.h
  HDRINST usr/include/linux/tdx-guest.h
  HDRINST usr/include/linux/close_range.h
  HDRINST usr/include/linux/ivtv.h
  HDRINST usr/include/linux/cryptouser.h
  HDRINST usr/include/linux/netfilter/xt_string.h
  HDRINST usr/include/linux/netfilter/nfnetlink_compat.h
  HDRINST usr/include/linux/netfilter/nf_nat.h
  HDRINST usr/include/linux/netfilter/xt_recent.h
  HDRINST usr/include/linux/netfilter/xt_addrtype.h
  HDRINST usr/include/linux/netfilter/nf_conntrack_tcp.h
  HDRINST usr/include/linux/netfilter/xt_MARK.h
  HDRINST usr/include/linux/netfilter/xt_SYNPROXY.h
  HDRINST usr/include/linux/netfilter/xt_multiport.h
  HDRINST usr/include/linux/netfilter/nfnetlink.h
  HDRINST usr/include/linux/netfilter/xt_cgroup.h
  HDRINST usr/include/linux/netfilter/nf_synproxy.h
  HDRINST usr/include/linux/netfilter/xt_TCPOPTSTRIP.h
  HDRINST usr/include/linux/netfilter/nfnetlink_log.h
  HDRINST usr/include/linux/netfilter/xt_TPROXY.h
  HDRINST usr/include/linux/netfilter/xt_u32.h
  HDRINST usr/include/linux/netfilter/nfnetlink_osf.h
  HDRINST usr/include/linux/netfilter/xt_ecn.h
  HDRINST usr/include/linux/netfilter/xt_esp.h
  HDRINST usr/include/linux/netfilter/nfnetlink_hook.h
  HDRINST usr/include/linux/netfilter/xt_mac.h
  HDRINST usr/include/linux/netfilter/xt_comment.h
  HDRINST usr/include/linux/netfilter/xt_NFQUEUE.h
  HDRINST usr/include/linux/netfilter/xt_osf.h
  HDRINST usr/include/linux/netfilter/xt_hashlimit.h
  HDRINST usr/include/linux/netfilter/nf_conntrack_sctp.h
  HDRINST usr/include/linux/netfilter/xt_socket.h
  HDRINST usr/include/linux/netfilter/xt_connmark.h
  HDRINST usr/include/linux/netfilter/xt_sctp.h
  HDRINST usr/include/linux/netfilter/xt_tcpudp.h
  HDRINST usr/include/linux/netfilter/xt_DSCP.h
  HDRINST usr/include/linux/netfilter/xt_IDLETIMER.h
  HDRINST usr/include/linux/netfilter/xt_time.h
  HDRINST usr/include/linux/netfilter/xt_policy.h
  HDRINST usr/include/linux/netfilter/xt_rpfilter.h
  HDRINST usr/include/linux/netfilter/xt_nfacct.h
  HDRINST usr/include/linux/netfilter/xt_SECMARK.h
  HDRINST usr/include/linux/netfilter/xt_length.h
  HDRINST usr/include/linux/netfilter/nfnetlink_cthelper.h
  HDRINST usr/include/linux/netfilter/xt_quota.h
  HDRINST usr/include/linux/netfilter/xt_ipcomp.h
  HDRINST usr/include/linux/netfilter/xt_CLASSIFY.h
  HDRINST usr/include/linux/netfilter/xt_iprange.h
  HDRINST usr/include/linux/netfilter/xt_bpf.h
  HDRINST usr/include/linux/netfilter/xt_LOG.h
  HDRINST usr/include/linux/netfilter/xt_rateest.h
  HDRINST usr/include/linux/netfilter/xt_CONNSECMARK.h
  HDRINST usr/include/linux/netfilter/xt_HMARK.h
  HDRINST usr/include/linux/netfilter/xt_CONNMARK.h
  HDRINST usr/include/linux/netfilter/xt_pkttype.h
  HDRINST usr/include/linux/netfilter/xt_ipvs.h
  HDRINST usr/include/linux/netfilter/xt_devgroup.h
  HDRINST usr/include/linux/netfilter/xt_AUDIT.h
  HDRINST usr/include/linux/netfilter/xt_realm.h
  HDRINST usr/include/linux/netfilter/nf_conntrack_common.h
  HDRINST usr/include/linux/netfilter/xt_set.h
  HDRINST usr/include/linux/netfilter/xt_LED.h
  HDRINST usr/include/linux/netfilter/xt_connlabel.h
  HDRINST usr/include/linux/netfilter/xt_owner.h
  HDRINST usr/include/linux/netfilter/xt_dccp.h
  HDRINST usr/include/linux/netfilter/xt_limit.h
  HDRINST usr/include/linux/netfilter/xt_conntrack.h
  HDRINST usr/include/linux/netfilter/xt_TEE.h
  HDRINST usr/include/linux/netfilter/xt_RATEEST.h
  HDRINST usr/include/linux/netfilter/xt_connlimit.h
  HDRINST usr/include/linux/netfilter/ipset/ip_set.h
  HDRINST usr/include/linux/netfilter/ipset/ip_set_list.h
  HDRINST usr/include/linux/netfilter/ipset/ip_set_hash.h
  HDRINST usr/include/linux/netfilter/ipset/ip_set_bitmap.h
  HDRINST usr/include/linux/netfilter/x_tables.h
  HDRINST usr/include/linux/netfilter/xt_dscp.h
  HDRINST usr/include/linux/netfilter/nf_conntrack_ftp.h
  HDRINST usr/include/linux/netfilter/xt_cluster.h
  HDRINST usr/include/linux/netfilter/nf_conntrack_tuple_common.h
  HDRINST usr/include/linux/netfilter/nf_log.h
  HDRINST usr/include/linux/netfilter/xt_tcpmss.h
  HDRINST usr/include/linux/netfilter/xt_NFLOG.h
  HDRINST usr/include/linux/netfilter/xt_l2tp.h
  HDRINST usr/include/linux/netfilter/xt_helper.h
  HDRINST usr/include/linux/netfilter/xt_statistic.h
  HDRINST usr/include/linux/netfilter/nfnetlink_queue.h
  HDRINST usr/include/linux/netfilter/nfnetlink_cttimeout.h
  HDRINST usr/include/linux/netfilter/xt_CT.h
  HDRINST usr/include/linux/netfilter/xt_CHECKSUM.h
  HDRINST usr/include/linux/netfilter/xt_connbytes.h
  HDRINST usr/include/linux/netfilter/xt_state.h
  HDRINST usr/include/linux/netfilter/nf_tables.h
  HDRINST usr/include/linux/netfilter/xt_mark.h
  HDRINST usr/include/linux/netfilter/xt_cpu.h
  HDRINST usr/include/linux/netfilter/nf_tables_compat.h
  HDRINST usr/include/linux/netfilter/xt_physdev.h
  HDRINST usr/include/linux/netfilter/nfnetlink_conntrack.h
  HDRINST usr/include/linux/netfilter/nfnetlink_acct.h
  HDRINST usr/include/linux/netfilter/xt_TCPMSS.h
  HDRINST usr/include/linux/tty_flags.h
  HDRINST usr/include/linux/if_phonet.h
  HDRINST usr/include/linux/elf-em.h
  HDRINST usr/include/linux/vm_sockets.h
  HDRINST usr/include/linux/dlmconstants.h
  HDRINST usr/include/linux/bsg.h
  HDRINST usr/include/linux/matroxfb.h
  HDRINST usr/include/linux/sysctl.h
  HDRINST usr/include/linux/unix_diag.h
  HDRINST usr/include/linux/pcitest.h
  HDRINST usr/include/linux/mman.h
  HDRINST usr/include/linux/if_plip.h
  HDRINST usr/include/linux/virtio_balloon.h
  HDRINST usr/include/linux/pidfd.h
  HDRINST usr/include/linux/f2fs.h
  HDRINST usr/include/linux/x25.h
  HDRINST usr/include/linux/if_cablemodem.h
  HDRINST usr/include/linux/utsname.h
  HDRINST usr/include/linux/counter.h
  HDRINST usr/include/linux/atm_tcp.h
  HDRINST usr/include/linux/atalk.h
  HDRINST usr/include/linux/virtio_rng.h
  HDRINST usr/include/linux/vboxguest.h
  HDRINST usr/include/linux/bpf_perf_event.h
  HDRINST usr/include/linux/ipmi_ssif_bmc.h
  HDRINST usr/include/linux/nfs_mount.h
  HDRINST usr/include/linux/sonet.h
  HDRINST usr/include/linux/netfilter.h
  HDRINST usr/include/linux/keyctl.h
  HDRINST usr/include/linux/nl80211.h
  HDRINST usr/include/linux/misc/bcm_vk.h
  HDRINST usr/include/linux/audit.h
  HDRINST usr/include/linux/tipc_config.h
  HDRINST usr/include/linux/tipc_sockets_diag.h
  HDRINST usr/include/linux/futex.h
  HDRINST usr/include/linux/sev-guest.h
  HDRINST usr/include/linux/ublk_cmd.h
  HDRINST usr/include/linux/types.h
  HDRINST usr/include/linux/virtio_input.h
  HDRINST usr/include/linux/if_slip.h
  HDRINST usr/include/linux/personality.h
  HDRINST usr/include/linux/openat2.h
  HDRINST usr/include/linux/poll.h
  HDRINST usr/include/linux/posix_acl.h
  HDRINST usr/include/linux/smc_diag.h
  HDRINST usr/include/linux/snmp.h
  HDRINST usr/include/linux/errqueue.h
  HDRINST usr/include/linux/if_tunnel.h
  HDRINST usr/include/linux/fanotify.h
  HDRINST usr/include/linux/kernel.h
  HDRINST usr/include/linux/rtnetlink.h
  HDRINST usr/include/linux/rpl.h
  HDRINST usr/include/linux/memfd.h
  HDRINST usr/include/linux/serial_core.h
  HDRINST usr/include/linux/dns_resolver.h
  HDRINST usr/include/linux/pr.h
  HDRINST usr/include/linux/atm_eni.h
  HDRINST usr/include/linux/lp.h
  HDRINST usr/include/linux/virtio_mem.h
  HDRINST usr/include/linux/ultrasound.h
  HDRINST usr/include/linux/sctp.h
  HDRINST usr/include/linux/uio.h
  HDRINST usr/include/linux/tcp_metrics.h
  HDRINST usr/include/linux/wwan.h
  HDRINST usr/include/linux/atmbr2684.h
  HDRINST usr/include/linux/in_route.h
  HDRINST usr/include/linux/qemu_fw_cfg.h
  HDRINST usr/include/linux/if_macsec.h
  HDRINST usr/include/linux/usb/charger.h
  HDRINST usr/include/linux/usb/g_uvc.h
  HDRINST usr/include/linux/usb/gadgetfs.h
  HDRINST usr/include/linux/usb/raw_gadget.h
  HDRINST usr/include/linux/usb/cdc-wdm.h
  HDRINST usr/include/linux/usb/g_printer.h
  HDRINST usr/include/linux/usb/midi.h
  HDRINST usr/include/linux/usb/tmc.h
  HDRINST usr/include/linux/usb/video.h
  HDRINST usr/include/linux/usb/functionfs.h
  HDRINST usr/include/linux/usb/audio.h
  HDRINST usr/include/linux/usb/ch11.h
  HDRINST usr/include/linux/usb/ch9.h
  HDRINST usr/include/linux/usb/cdc.h
  HDRINST usr/include/linux/jffs2.h
  HDRINST usr/include/linux/ax25.h
  HDRINST usr/include/linux/auto_fs.h
  HDRINST usr/include/linux/tiocl.h
  HDRINST usr/include/linux/scc.h
  HDRINST usr/include/linux/psci.h
  HDRINST usr/include/linux/swab.h
  HDRINST usr/include/linux/cec.h
  HDRINST usr/include/linux/kfd_ioctl.h
  HDRINST usr/include/linux/smc.h
  HDRINST usr/include/linux/qrtr.h
  HDRINST usr/include/linux/screen_info.h
  HDRINST usr/include/linux/nfsacl.h
  HDRINST usr/include/linux/seg6_hmac.h
  HDRINST usr/include/linux/gameport.h
  HDRINST usr/include/linux/wireless.h
  HDRINST usr/include/linux/fdreg.h
  HDRINST usr/include/linux/cciss_defs.h
  HDRINST usr/include/linux/serial_reg.h
  HDRINST usr/include/linux/perf_event.h
  HDRINST usr/include/linux/in6.h
  HDRINST usr/include/linux/hid.h
  HDRINST usr/include/linux/netlink.h
  HDRINST usr/include/linux/fuse.h
  HDRINST usr/include/linux/magic.h
  HDRINST usr/include/linux/ioam6_iptunnel.h
  HDRINST usr/include/linux/stm.h
  HDRINST usr/include/linux/vsockmon.h
  HDRINST usr/include/linux/seg6.h
  HDRINST usr/include/linux/idxd.h
  HDRINST usr/include/linux/nitro_enclaves.h
  HDRINST usr/include/linux/ptrace.h
  HDRINST usr/include/linux/ioam6_genl.h
  HDRINST usr/include/linux/qnx4_fs.h
  HDRINST usr/include/linux/fsl_mc.h
  HDRINST usr/include/linux/net_tstamp.h
  HDRINST usr/include/linux/msg.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_TTL.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_ttl.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_ah.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_ECN.h
  HDRINST usr/include/linux/netfilter_ipv4/ip_tables.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_ecn.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_CLUSTERIP.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_REJECT.h
  HDRINST usr/include/linux/netfilter_ipv4/ipt_LOG.h
  HDRINST usr/include/linux/sem.h
  HDRINST usr/include/linux/net_namespace.h
  HDRINST usr/include/linux/radeonfb.h
  HDRINST usr/include/linux/tee.h
  HDRINST usr/include/linux/udp.h
  HDRINST usr/include/linux/virtio_bt.h
  HDRINST usr/include/linux/v4l2-subdev.h
  HDRINST usr/include/linux/posix_acl_xattr.h
  HDRINST usr/include/linux/v4l2-mediabus.h
  HDRINST usr/include/linux/atmapi.h
  HDRINST usr/include/linux/raid/md_p.h
  HDRINST usr/include/linux/raid/md_u.h
  HDRINST usr/include/linux/zorro_ids.h
  HDRINST usr/include/linux/nbd.h
  HDRINST usr/include/linux/isst_if.h
  HDRINST usr/include/linux/rxrpc.h
  HDRINST usr/include/linux/unistd.h
  HDRINST usr/include/linux/if_arp.h
  HDRINST usr/include/linux/atm_zatm.h
  HDRINST usr/include/linux/io_uring.h
  HDRINST usr/include/linux/if_fddi.h
  HDRINST usr/include/linux/bpqether.h
  HDRINST usr/include/linux/sysinfo.h
  HDRINST usr/include/linux/auto_dev-ioctl.h
  HDRINST usr/include/linux/nfs4_mount.h
  HDRINST usr/include/linux/keyboard.h
  HDRINST usr/include/linux/virtio_mmio.h
  HDRINST usr/include/linux/input.h
  HDRINST usr/include/linux/qnxtypes.h
  HDRINST usr/include/linux/mdio.h
  HDRINST usr/include/linux/lwtunnel.h
  HDRINST usr/include/linux/gfs2_ondisk.h
  HDRINST usr/include/linux/nfs4.h
  HDRINST usr/include/linux/ptp_clock.h
  HDRINST usr/include/linux/nubus.h
  HDRINST usr/include/linux/if_bonding.h
  HDRINST usr/include/linux/kcov.h
  HDRINST usr/include/linux/fadvise.h
  HDRINST usr/include/linux/taskstats.h
  HDRINST usr/include/linux/veth.h
  HDRINST usr/include/linux/atm.h
  HDRINST usr/include/linux/ipmi.h
  HDRINST usr/include/linux/kdev_t.h
  HDRINST usr/include/linux/mount.h
  HDRINST usr/include/linux/shm.h
  HDRINST usr/include/linux/resource.h
  HDRINST usr/include/linux/prctl.h
  HDRINST usr/include/linux/watch_queue.h
  HDRINST usr/include/linux/sched.h
  HDRINST usr/include/linux/phonet.h
  HDRINST usr/include/linux/random.h
  HDRINST usr/include/linux/tty.h
  HDRINST usr/include/linux/apm_bios.h
  HDRINST usr/include/linux/fd.h
  HDRINST usr/include/linux/um_timetravel.h
  HDRINST usr/include/linux/tls.h
  HDRINST usr/include/linux/rpmsg_types.h
  HDRINST usr/include/linux/pfrut.h
  HDRINST usr/include/linux/mei.h
  HDRINST usr/include/linux/fsi.h
  HDRINST usr/include/linux/rds.h
  HDRINST usr/include/linux/if_x25.h
  HDRINST usr/include/linux/param.h
  HDRINST usr/include/linux/netdevice.h
  HDRINST usr/include/linux/binfmts.h
  HDRINST usr/include/linux/if_pppox.h
  HDRINST usr/include/linux/sockios.h
  HDRINST usr/include/linux/kcm.h
  HDRINST usr/include/linux/virtio_9p.h
  HDRINST usr/include/linux/genwqe/genwqe_card.h
  HDRINST usr/include/linux/if_tun.h
  HDRINST usr/include/linux/if_ether.h
  HDRINST usr/include/linux/kvm_para.h
  HDRINST usr/include/linux/kernel-page-flags.h
  HDRINST usr/include/linux/cdrom.h
  HDRINST usr/include/linux/un.h
  HDRINST usr/include/linux/module.h
  HDRINST usr/include/linux/mqueue.h
  HDRINST usr/include/linux/a.out.h
  HDRINST usr/include/linux/input-event-codes.h
  HDRINST usr/include/linux/coda.h
  HDRINST usr/include/linux/rio_mport_cdev.h
  HDRINST usr/include/linux/ipsec.h
  HDRINST usr/include/linux/blkpg.h
  HDRINST usr/include/linux/blkzoned.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_arpreply.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_redirect.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_nflog.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_802_3.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_nat.h
  LD      /kernel/build64/tools/objtool/libsubcmd/libsubcmd-in.o
  HDRINST usr/include/linux/netfilter_bridge/ebt_mark_m.h
  HDRINST usr/include/linux/netfilter_bridge/ebtables.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_vlan.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_limit.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_log.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_stp.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_pkttype.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_ip.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_ip6.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_arp.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_mark_t.h
  HDRINST usr/include/linux/netfilter_bridge/ebt_among.h
  HDRINST usr/include/linux/reiserfs_fs.h
  HDRINST usr/include/linux/cciss_ioctl.h
  HDRINST usr/include/linux/fsmap.h
  HDRINST usr/include/linux/smiapp.h
  HDRINST usr/include/linux/switchtec_ioctl.h
  HDRINST usr/include/linux/atmdev.h
  HDRINST usr/include/linux/hpet.h
  HDRINST usr/include/linux/virtio_config.h
  HDRINST usr/include/linux/string.h
  HDRINST usr/include/linux/kfd_sysfs.h
  HDRINST usr/include/linux/inet_diag.h
  HDRINST usr/include/linux/xattr.h
  HDRINST usr/include/linux/netdev.h
  HDRINST usr/include/linux/iommufd.h
  HDRINST usr/include/linux/errno.h
  HDRINST usr/include/linux/icmp.h
  HDRINST usr/include/linux/i2o-dev.h
  AR      /kernel/build64/tools/objtool/libsubcmd/libsubcmd.a
  HDRINST usr/include/linux/pg.h
  HDRINST usr/include/linux/if_bridge.h
  HDRINST usr/include/linux/thermal.h
  HDRINST usr/include/linux/uinput.h
  HDRINST usr/include/linux/dqblk_xfs.h
  HDRINST usr/include/linux/v4l2-common.h
  HDRINST usr/include/linux/nvram.h
  HDRINST usr/include/linux/if_vlan.h
  HDRINST usr/include/linux/uhid.h
  HDRINST usr/include/linux/omap3isp.h
  HDRINST usr/include/linux/rose.h
  HDRINST usr/include/linux/phantom.h
  HDRINST usr/include/linux/ipmi_msgdefs.h
  HDRINST usr/include/linux/bcm933xx_hcs.h
  HDRINST usr/include/linux/bpf.h
  HDRINST usr/include/linux/mempolicy.h
  HDRINST usr/include/linux/efs_fs_sb.h
  HDRINST usr/include/linux/nexthop.h
  HDRINST usr/include/linux/net_dropmon.h
  HDRINST usr/include/linux/surface_aggregator/cdev.h
  HDRINST usr/include/linux/surface_aggregator/dtx.h
  HDRINST usr/include/linux/net.h
  HDRINST usr/include/linux/mii.h
  HDRINST usr/include/linux/cm4000_cs.h
  HDRINST usr/include/linux/virtio_pcidev.h
  HDRINST usr/include/linux/termios.h
  HDRINST usr/include/linux/cgroupstats.h
  HDRINST usr/include/linux/mpls.h
  HDRINST usr/include/linux/iommu.h
  HDRINST usr/include/linux/toshiba.h
  HDRINST usr/include/linux/virtio_scsi.h
  HDRINST usr/include/linux/zorro.h
  HDRINST usr/include/linux/chio.h
  HDRINST usr/include/linux/pkt_sched.h
  HDRINST usr/include/linux/cramfs_fs.h
  HDRINST usr/include/linux/nfs3.h
  HDRINST usr/include/linux/vfio_ccw.h
  HDRINST usr/include/linux/atm_nicstar.h
  HDRINST usr/include/linux/ncsi.h
  HDRINST usr/include/linux/virtio_net.h
  HDRINST usr/include/linux/ioctl.h
  HDRINST usr/include/linux/stddef.h
  HDRINST usr/include/linux/limits.h
  HDRINST usr/include/linux/ipmi_bmc.h
  HDRINST usr/include/linux/netfilter_arp.h
  HDRINST usr/include/linux/if_addr.h
  HDRINST usr/include/linux/rpmsg.h
  HDRINST usr/include/linux/media-bus-format.h
  HDRINST usr/include/linux/kernelcapi.h
  HDRINST usr/include/linux/ppp_defs.h
  HDRINST usr/include/linux/ethtool.h
  HDRINST usr/include/linux/aspeed-video.h
  HDRINST usr/include/linux/hdlc.h
  HDRINST usr/include/linux/fscrypt.h
  HDRINST usr/include/linux/batadv_packet.h
  CC      /kernel/build64/tools/objtool/weak.o
  HDRINST usr/include/linux/uuid.h
  CC      /kernel/build64/tools/objtool/check.o
  HDRINST usr/include/linux/capi.h
  HDRINST usr/include/linux/mptcp.h
  CC      /kernel/build64/tools/objtool/special.o
  HDRINST usr/include/linux/hidraw.h
  CC      /kernel/build64/tools/objtool/builtin-check.o
  HDRINST usr/include/linux/virtio_console.h
  HDRINST usr/include/linux/irqnr.h
  CC      /kernel/build64/tools/objtool/elf.o
  MKDIR   /kernel/build64/tools/objtool/arch/x86/
  HDRINST usr/include/linux/coresight-stm.h
  CC      /kernel/build64/tools/objtool/objtool.o
  HDRINST usr/include/linux/cxl_mem.h
  HDRINST usr/include/linux/iso_fs.h
  CC      /kernel/build64/tools/objtool/orc_gen.o
  HDRINST usr/include/linux/virtio_blk.h
  MKDIR   /kernel/build64/tools/objtool/arch/x86/lib/
  HDRINST usr/include/linux/udf_fs_i.h
  CC      /kernel/build64/tools/objtool/orc_dump.o
  HDRINST usr/include/linux/coff.h
  CC      /kernel/build64/tools/objtool/arch/x86/special.o
  CC      /kernel/build64/tools/objtool/libstring.o
  HDRINST usr/include/linux/dma-buf.h
  HDRINST usr/include/linux/ife.h
  CC      /kernel/build64/tools/objtool/libctype.o
  GEN     /kernel/build64/tools/objtool/arch/x86/lib/inat-tables.c
  HDRINST usr/include/linux/agpgart.h
  HDRINST usr/include/linux/socket.h
  CC      /kernel/build64/tools/objtool/str_error_r.o
  HDRINST usr/include/linux/nilfs2_ondisk.h
  HDRINST usr/include/linux/connector.h
  HDRINST usr/include/linux/auto_fs4.h
  CC      /kernel/build64/tools/objtool/librbtree.o
  HDRINST usr/include/linux/bt-bmc.h
  HDRINST usr/include/linux/map_to_7segment.h
  HDRINST usr/include/linux/tc_act/tc_skbedit.h
  HDRINST usr/include/linux/tc_act/tc_ctinfo.h
  HDRINST usr/include/linux/tc_act/tc_defact.h
  HDRINST usr/include/linux/tc_act/tc_gact.h
  HDRINST usr/include/linux/tc_act/tc_vlan.h
  HDRINST usr/include/linux/tc_act/tc_skbmod.h
  HDRINST usr/include/linux/tc_act/tc_sample.h
  HDRINST usr/include/linux/tc_act/tc_tunnel_key.h
  HDRINST usr/include/linux/tc_act/tc_gate.h
  HDRINST usr/include/linux/tc_act/tc_mirred.h
  HDRINST usr/include/linux/tc_act/tc_nat.h
  HDRINST usr/include/linux/tc_act/tc_csum.h
  HDRINST usr/include/linux/tc_act/tc_connmark.h
  HDRINST usr/include/linux/tc_act/tc_ife.h
  HDRINST usr/include/linux/tc_act/tc_mpls.h
  HDRINST usr/include/linux/tc_act/tc_ct.h
  HDRINST usr/include/linux/tc_act/tc_pedit.h
  HDRINST usr/include/linux/tc_act/tc_bpf.h
  HDRINST usr/include/linux/tc_act/tc_ipt.h
  HDRINST usr/include/linux/netrom.h
  HDRINST usr/include/linux/joystick.h
  HDRINST usr/include/linux/falloc.h
  HDRINST usr/include/linux/cycx_cfm.h
  HDRINST usr/include/linux/omapfb.h
  HDRINST usr/include/linux/msdos_fs.h
  HDRINST usr/include/linux/virtio_types.h
  HDRINST usr/include/linux/mroute.h
  HDRINST usr/include/linux/psample.h
  HDRINST usr/include/linux/ipv6.h
  HDRINST usr/include/linux/dw100.h
  HDRINST usr/include/linux/psp-sev.h
  HDRINST usr/include/linux/vfio.h
  HDRINST usr/include/linux/if_ppp.h
  HDRINST usr/include/linux/byteorder/big_endian.h
  HDRINST usr/include/linux/byteorder/little_endian.h
  HDRINST usr/include/linux/comedi.h
  HDRINST usr/include/linux/scif_ioctl.h
  HDRINST usr/include/linux/timerfd.h
  HDRINST usr/include/linux/time_types.h
  HDRINST usr/include/linux/firewire-constants.h
  HDRINST usr/include/linux/virtio_snd.h
  HDRINST usr/include/linux/ppp-ioctl.h
  HDRINST usr/include/linux/fib_rules.h
  HDRINST usr/include/linux/gen_stats.h
  HDRINST usr/include/linux/virtio_iommu.h
  HDRINST usr/include/linux/genetlink.h
  HDRINST usr/include/linux/uvcvideo.h
  HDRINST usr/include/linux/pfkeyv2.h
  HDRINST usr/include/linux/soundcard.h
  HDRINST usr/include/linux/times.h
  HDRINST usr/include/linux/nfc.h
  HDRINST usr/include/linux/affs_hardblocks.h
  HDRINST usr/include/linux/nilfs2_api.h
  HDRINST usr/include/linux/rseq.h
  HDRINST usr/include/linux/caif/caif_socket.h
  HDRINST usr/include/linux/caif/if_caif.h
  HDRINST usr/include/linux/i2c-dev.h
  HDRINST usr/include/linux/cuda.h
  HDRINST usr/include/linux/cn_proc.h
  HDRINST usr/include/linux/parport.h
  HDRINST usr/include/linux/v4l2-controls.h
  HDRINST usr/include/linux/hsi/cs-protocol.h
  HDRINST usr/include/linux/hsi/hsi_char.h
  HDRINST usr/include/linux/seg6_genl.h
  HDRINST usr/include/linux/am437x-vpfe.h
  HDRINST usr/include/linux/amt.h
  HDRINST usr/include/linux/netconf.h
  HDRINST usr/include/linux/erspan.h
  HDRINST usr/include/linux/nsfs.h
  HDRINST usr/include/linux/xilinx-v4l2-controls.h
  HDRINST usr/include/linux/aspeed-p2a-ctrl.h
  HDRINST usr/include/linux/vfio_zdev.h
  HDRINST usr/include/linux/serio.h
  CC      /kernel/build64/tools/objtool/arch/x86/decode.o
  HDRINST usr/include/linux/acrn.h
  HDRINST usr/include/linux/nfs2.h
  HDRINST usr/include/linux/virtio_pci.h
  HDRINST usr/include/linux/ipc.h
  HDRINST usr/include/linux/ethtool_netlink.h
  HDRINST usr/include/linux/kd.h
  HDRINST usr/include/linux/elf.h
  HDRINST usr/include/linux/videodev2.h
  HDRINST usr/include/linux/if_alg.h
  HDRINST usr/include/linux/sonypi.h
  HDRINST usr/include/linux/fsverity.h
  HDRINST usr/include/linux/if.h
  HDRINST usr/include/linux/btrfs.h
  HDRINST usr/include/linux/vm_sockets_diag.h
  HDRINST usr/include/linux/packet_diag.h
  HDRINST usr/include/linux/netfilter_bridge.h
  HDRINST usr/include/linux/netfilter_ipv4.h
  HDRINST usr/include/linux/kvm.h
  HDRINST usr/include/linux/pci.h
  HDRINST usr/include/linux/if_addrlabel.h
  HDRINST usr/include/linux/hdlcdrv.h
  HDRINST usr/include/linux/cfm_bridge.h
  HDRINST usr/include/linux/fiemap.h
  HDRINST usr/include/linux/dm-ioctl.h
  HDRINST usr/include/linux/aspeed-lpc-ctrl.h
  HDRINST usr/include/linux/atmioc.h
  HDRINST usr/include/linux/dlm.h
  HDRINST usr/include/linux/pci_regs.h
  HDRINST usr/include/linux/cachefiles.h
  HDRINST usr/include/linux/membarrier.h
  HDRINST usr/include/linux/nfs_idmap.h
  HDRINST usr/include/linux/ip.h
  HDRINST usr/include/linux/atm_he.h
  HDRINST usr/include/linux/nfsd/export.h
  HDRINST usr/include/linux/nfsd/stats.h
  HDRINST usr/include/linux/nfsd/debug.h
  HDRINST usr/include/linux/nfsd/cld.h
  HDRINST usr/include/linux/ip_vs.h
  HDRINST usr/include/linux/vmcore.h
  HDRINST usr/include/linux/vbox_vmmdev_types.h
  HDRINST usr/include/linux/dvb/osd.h
  HDRINST usr/include/linux/dvb/dmx.h
  HDRINST usr/include/linux/dvb/net.h
  HDRINST usr/include/linux/dvb/frontend.h
  HDRINST usr/include/linux/dvb/ca.h
  HDRINST usr/include/linux/dvb/version.h
  HDRINST usr/include/linux/dvb/video.h
  HDRINST usr/include/linux/dvb/audio.h
  HDRINST usr/include/linux/nfs.h
  HDRINST usr/include/linux/if_link.h
  HDRINST usr/include/linux/wait.h
  HDRINST usr/include/linux/icmpv6.h
  HDRINST usr/include/linux/media.h
  HDRINST usr/include/linux/seg6_local.h
  HDRINST usr/include/linux/openvswitch.h
  HDRINST usr/include/linux/atmsap.h
  HDRINST usr/include/linux/bpfilter.h
  HDRINST usr/include/linux/fpga-dfl.h
  HDRINST usr/include/linux/userio.h
  HDRINST usr/include/linux/signal.h
  HDRINST usr/include/linux/map_to_14segment.h
  HDRINST usr/include/linux/hdreg.h
  HDRINST usr/include/linux/utime.h
  HDRINST usr/include/linux/usbdevice_fs.h
  HDRINST usr/include/linux/timex.h
  HDRINST usr/include/linux/if_fc.h
  HDRINST usr/include/linux/reiserfs_xattr.h
  HDRINST usr/include/linux/hw_breakpoint.h
  HDRINST usr/include/linux/quota.h
  HDRINST usr/include/linux/ioprio.h
  HDRINST usr/include/linux/eventpoll.h
  HDRINST usr/include/linux/atmclip.h
  HDRINST usr/include/linux/can.h
  HDRINST usr/include/linux/if_team.h
  HDRINST usr/include/linux/usbip.h
  HDRINST usr/include/linux/stat.h
  HDRINST usr/include/linux/fou.h
  HDRINST usr/include/linux/hash_info.h
  HDRINST usr/include/linux/ppp-comp.h
  HDRINST usr/include/linux/ip6_tunnel.h
  HDRINST usr/include/linux/tipc_netlink.h
  HDRINST usr/include/linux/in.h
  HDRINST usr/include/linux/wireguard.h
  HDRINST usr/include/linux/btf.h
  HDRINST usr/include/linux/batman_adv.h
  HDRINST usr/include/linux/fcntl.h
  HDRINST usr/include/linux/if_ltalk.h
  HDRINST usr/include/linux/i2c.h
  HDRINST usr/include/linux/atm_idt77105.h
  HDRINST usr/include/linux/kexec.h
  HDRINST usr/include/linux/arm_sdei.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6_tables.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_ah.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_NPT.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_rt.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_REJECT.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_opts.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_srh.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_LOG.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_mh.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_HL.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_hl.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_frag.h
  HDRINST usr/include/linux/netfilter_ipv6/ip6t_ipv6header.h
  HDRINST usr/include/linux/minix_fs.h
  HDRINST usr/include/linux/aio_abi.h
  HDRINST usr/include/linux/pktcdvd.h
  HDRINST usr/include/linux/libc-compat.h
  HDRINST usr/include/linux/atmlec.h
  HDRINST usr/include/linux/signalfd.h
  HDRINST usr/include/linux/bpf_common.h
  HDRINST usr/include/linux/seg6_iptunnel.h
  HDRINST usr/include/linux/synclink.h
  HDRINST usr/include/linux/mpls_iptunnel.h
  HDRINST usr/include/linux/mctp.h
  HDRINST usr/include/linux/if_xdp.h
  HDRINST usr/include/linux/llc.h
  HDRINST usr/include/linux/atmsvc.h
  HDRINST usr/include/linux/sed-opal.h
  HDRINST usr/include/linux/sock_diag.h
  HDRINST usr/include/linux/time.h
  HDRINST usr/include/linux/securebits.h
  HDRINST usr/include/linux/fsl_hypervisor.h
  HDRINST usr/include/linux/if_hippi.h
  HDRINST usr/include/linux/dlm_netlink.h
  HDRINST usr/include/linux/seccomp.h
  HDRINST usr/include/linux/oom.h
  HDRINST usr/include/linux/filter.h
  HDRINST usr/include/linux/inotify.h
  HDRINST usr/include/linux/rfkill.h
  HDRINST usr/include/linux/reboot.h
  HDRINST usr/include/linux/can/vxcan.h
  HDRINST usr/include/linux/can/j1939.h
  HDRINST usr/include/linux/can/netlink.h
  HDRINST usr/include/linux/can/bcm.h
  HDRINST usr/include/linux/can/raw.h
  HDRINST usr/include/linux/can/gw.h
  HDRINST usr/include/linux/can/error.h
  HDRINST usr/include/linux/can/isotp.h
  HDRINST usr/include/linux/if_eql.h
  HDRINST usr/include/linux/hiddev.h
  HDRINST usr/include/linux/blktrace_api.h
  HDRINST usr/include/linux/ccs.h
  HDRINST usr/include/linux/ioam6.h
  HDRINST usr/include/linux/hsr_netlink.h
  HDRINST usr/include/linux/mmc/ioctl.h
  HDRINST usr/include/linux/bfs_fs.h
  HDRINST usr/include/linux/rio_cm_cdev.h
  HDRINST usr/include/linux/uleds.h
  HDRINST usr/include/linux/mrp_bridge.h
  HDRINST usr/include/linux/adb.h
  HDRINST usr/include/linux/pmu.h
  HDRINST usr/include/linux/udmabuf.h
  HDRINST usr/include/linux/kcmp.h
  HDRINST usr/include/linux/dma-heap.h
  HDRINST usr/include/linux/userfaultfd.h
  HDRINST usr/include/linux/netfilter_arp/arpt_mangle.h
  HDRINST usr/include/linux/netfilter_arp/arp_tables.h
  HDRINST usr/include/linux/tipc.h
  HDRINST usr/include/linux/virtio_ids.h
  HDRINST usr/include/linux/l2tp.h
  HDRINST usr/include/linux/devlink.h
  HDRINST usr/include/linux/virtio_gpio.h
  HDRINST usr/include/linux/dcbnl.h
  HDRINST usr/include/linux/cyclades.h
  HDRINST usr/include/sound/sof/fw.h
  HDRINST usr/include/sound/intel/avs/tokens.h
  HDRINST usr/include/sound/sof/abi.h
  HDRINST usr/include/sound/sof/tokens.h
  HDRINST usr/include/sound/sof/header.h
  HDRINST usr/include/sound/usb_stream.h
  HDRINST usr/include/sound/sfnt_info.h
  HDRINST usr/include/sound/asequencer.h
  HDRINST usr/include/sound/tlv.h
  HDRINST usr/include/sound/asound.h
  HDRINST usr/include/sound/asoc.h
  HDRINST usr/include/sound/sb16_csp.h
  HDRINST usr/include/sound/compress_offload.h
  HDRINST usr/include/sound/hdsp.h
  HDRINST usr/include/sound/emu10k1.h
  HDRINST usr/include/sound/snd_ar_tokens.h
  HDRINST usr/include/sound/snd_sst_tokens.h
  HDRINST usr/include/sound/asound_fm.h
  HDRINST usr/include/sound/hdspm.h
  HDRINST usr/include/sound/compress_params.h
  HDRINST usr/include/sound/firewire.h
  HDRINST usr/include/sound/skl-tplg-interface.h
  HDRINST usr/include/scsi/scsi_bsg_ufs.h
  HDRINST usr/include/scsi/scsi_netlink_fc.h
  HDRINST usr/include/scsi/scsi_bsg_mpi3mr.h
  HDRINST usr/include/scsi/fc/fc_ns.h
  HDRINST usr/include/scsi/fc/fc_fs.h
  HDRINST usr/include/scsi/fc/fc_els.h
  HDRINST usr/include/scsi/fc/fc_gs.h
  HDRINST usr/include/scsi/scsi_bsg_fc.h
  HDRINST usr/include/scsi/cxlflash_ioctl.h
  HDRINST usr/include/scsi/scsi_netlink.h
  HDRINST usr/include/linux/version.h
  HDRINST usr/include/asm/processor-flags.h
  HDRINST usr/include/asm/auxvec.h
  HDRINST usr/include/asm/svm.h
  HDRINST usr/include/asm/bitsperlong.h
  HDRINST usr/include/asm/kvm_perf.h
  HDRINST usr/include/asm/mce.h
  HDRINST usr/include/asm/posix_types.h
  HDRINST usr/include/asm/msr.h
  HDRINST usr/include/asm/sigcontext32.h
  HDRINST usr/include/asm/mman.h
  HDRINST usr/include/asm/shmbuf.h
  HDRINST usr/include/asm/e820.h
  HDRINST usr/include/asm/posix_types_64.h
  HDRINST usr/include/asm/vsyscall.h
  HDRINST usr/include/asm/msgbuf.h
  HDRINST usr/include/asm/swab.h
  HDRINST usr/include/asm/statfs.h
  HDRINST usr/include/asm/posix_types_x32.h
  HDRINST usr/include/asm/ptrace.h
  HDRINST usr/include/asm/unistd.h
  HDRINST usr/include/asm/ist.h
  HDRINST usr/include/asm/prctl.h
  HDRINST usr/include/asm/boot.h
  HDRINST usr/include/asm/sigcontext.h
  HDRINST usr/include/asm/posix_types_32.h
  HDRINST usr/include/asm/kvm_para.h
  HDRINST usr/include/asm/a.out.h
  HDRINST usr/include/asm/mtrr.h
  HDRINST usr/include/asm/amd_hsmp.h
  HDRINST usr/include/asm/hwcap2.h
  HDRINST usr/include/asm/ptrace-abi.h
  HDRINST usr/include/asm/vm86.h
  HDRINST usr/include/asm/vmx.h
  HDRINST usr/include/asm/ldt.h
  HDRINST usr/include/asm/perf_regs.h
  HDRINST usr/include/asm/kvm.h
  HDRINST usr/include/asm/debugreg.h
  HDRINST usr/include/asm/signal.h
  HDRINST usr/include/asm/bootparam.h
  HDRINST usr/include/asm/siginfo.h
  HDRINST usr/include/asm/hw_breakpoint.h
  HDRINST usr/include/asm/stat.h
  HDRINST usr/include/asm/setup.h
  HDRINST usr/include/asm/sembuf.h
  HDRINST usr/include/asm/sgx.h
  HDRINST usr/include/asm/ucontext.h
  HDRINST usr/include/asm/byteorder.h
  HDRINST usr/include/asm/unistd_64.h
  HDRINST usr/include/asm/ioctls.h
  HDRINST usr/include/asm/bpf_perf_event.h
  HDRINST usr/include/asm/types.h
  HDRINST usr/include/asm/poll.h
  HDRINST usr/include/asm/resource.h
  HDRINST usr/include/asm/param.h
  HDRINST usr/include/asm/sockios.h
  HDRINST usr/include/asm/errno.h
  HDRINST usr/include/asm/unistd_x32.h
  HDRINST usr/include/asm/termios.h
  HDRINST usr/include/asm/ioctl.h
  HDRINST usr/include/asm/socket.h
  HDRINST usr/include/asm/unistd_32.h
  HDRINST usr/include/asm/termbits.h
  HDRINST usr/include/asm/fcntl.h
  HDRINST usr/include/asm/ipcbuf.h
  HOSTLD  scripts/mod/modpost
  CC      kernel/bounds.s
  CHKSHA1 ../include/linux/atomic/atomic-arch-fallback.h
  CHKSHA1 ../include/linux/atomic/atomic-instrumented.h
  CHKSHA1 ../include/linux/atomic/atomic-long.h
  UPD     include/generated/timeconst.h
  UPD     include/generated/bounds.h
  CC      arch/x86/kernel/asm-offsets.s
  LD      /kernel/build64/tools/objtool/arch/x86/objtool-in.o
  UPD     include/generated/asm-offsets.h
  CALL    ../scripts/checksyscalls.sh
  LD      /kernel/build64/tools/objtool/objtool-in.o
  LINK    /kernel/build64/tools/objtool/objtool
  LDS     scripts/module.lds
  CC      ipc/compat.o
  CC      ipc/util.o
  AR      certs/built-in.a
  HOSTCC  usr/gen_init_cpio
  CC      ipc/msgutil.o
  CC      ipc/msg.o
  CC      io_uring/io_uring.o
  CC      ipc/sem.o
  CC      io_uring/xattr.o
  CC      init/main.o
  AS      arch/x86/lib/clear_page_64.o
  CC      ipc/shm.o
  CC      io_uring/nop.o
  CC      security/commoncap.o
  CC      arch/x86/lib/cmdline.o
  AS      arch/x86/lib/cmpxchg16b_emu.o
  CC      init/do_mounts.o
  CC      block/bdev.o
  CC      security/min_addr.o
  CC      ipc/syscall.o
  CC      arch/x86/lib/copy_mc.o
  AR      arch/x86/video/built-in.a
  CC      io_uring/fs.o
  CC      arch/x86/power/cpu.o
  CC      arch/x86/pci/i386.o
  UPD     init/utsversion-tmp.h
  CC      block/partitions/core.o
  CC      arch/x86/realmode/init.o
  CC [M]  arch/x86/video/fbdev.o
  AR      arch/x86/ia32/built-in.a
  AR      arch/x86/net/built-in.a
  AS      arch/x86/crypto/aesni-intel_asm.o
  CC      security/keys/gc.o
  CC      net/ethernet/eth.o
  CC      net/llc/llc_core.o
  AR      virt/lib/built-in.a
  CC [M]  arch/x86/kvm/../../../virt/kvm/kvm_main.o
  CC      net/core/sock.o
  CC [M]  virt/lib/irqbypass.o
  CC      arch/x86/mm/pat/set_memory.o
  CC      net/core/request_sock.o
  AR      drivers/irqchip/built-in.a
  CC      sound/core/seq/seq.o
  CC      arch/x86/kernel/fpu/init.o
  CC      fs/notify/dnotify/dnotify.o
  CC      arch/x86/events/amd/core.o
  AR      arch/x86/platform/atom/built-in.a
  CC      sound/core/seq/seq_lock.o
  AR      arch/x86/platform/ce4100/built-in.a
  CC      arch/x86/entry/vsyscall/vsyscall_64.o
  AS      arch/x86/entry/vsyscall/vsyscall_emu_64.o
  CC      arch/x86/platform/efi/memmap.o
  CC      arch/x86/entry/vdso/vma.o
  AR      drivers/bus/mhi/built-in.a
  AS      arch/x86/lib/copy_mc_64.o
  AR      drivers/bus/built-in.a
  CC      lib/kunit/test.o
  CC      kernel/sched/core.o
  CC      mm/kasan/common.o
  CC      arch/x86/crypto/aesni-intel_glue.o
  CC      crypto/api.o
  AR      drivers/phy/allwinner/built-in.a
  CC      sound/core/seq/seq_clientmgr.o
  AR      drivers/phy/amlogic/built-in.a
  AS      arch/x86/entry/entry.o
  AR      drivers/phy/broadcom/built-in.a
  AR      drivers/phy/cadence/built-in.a
  CC      arch/x86/power/hibernate_64.o
  AR      drivers/phy/freescale/built-in.a
  CC      crypto/cipher.o
  AR      drivers/phy/hisilicon/built-in.a
  AR      drivers/phy/ingenic/built-in.a
  AR      drivers/phy/intel/built-in.a
  AR      drivers/phy/lantiq/built-in.a
  AR      drivers/phy/marvell/built-in.a
  AR      drivers/phy/mediatek/built-in.a
  AR      drivers/phy/microchip/built-in.a
  AR      drivers/phy/motorola/built-in.a
  AR      drivers/phy/mscc/built-in.a
  AR      drivers/phy/qualcomm/built-in.a
  AR      drivers/phy/ralink/built-in.a
  AS      arch/x86/lib/copy_page_64.o
  AR      drivers/phy/renesas/built-in.a
  AR      drivers/phy/rockchip/built-in.a
  AR      drivers/phy/samsung/built-in.a
  AS      arch/x86/lib/copy_user_64.o
  GEN     usr/initramfs_data.cpio
  AR      drivers/phy/socionext/built-in.a
  COPY    usr/initramfs_inc_data
  AR      drivers/phy/st/built-in.a
  AS      usr/initramfs_data.o
  AR      drivers/phy/sunplus/built-in.a
  CC      arch/x86/lib/cpu.o
  AR      drivers/phy/tegra/built-in.a
  AR      usr/built-in.a
  AR      drivers/phy/ti/built-in.a
  AR      drivers/phy/xilinx/built-in.a
  CC      arch/x86/platform/efi/quirks.o
  CC      drivers/phy/phy-core.o
  AS      arch/x86/entry/entry_64.o
  AR      virt/built-in.a
  CC      mm/filemap.o
  CC      arch/x86/events/intel/core.o
  CC      ipc/ipc_sysctl.o
  CC      arch/x86/events/intel/bts.o
  CC      ipc/mqueue.o
  CC      arch/x86/kernel/fpu/bugs.o
  AS      arch/x86/realmode/rm/header.o
  CC      arch/x86/kernel/fpu/core.o
  AS      arch/x86/realmode/rm/trampoline_64.o
  CC      ipc/namespace.o
  CC      security/keys/key.o
  AS      arch/x86/realmode/rm/stack.o
  AS      arch/x86/realmode/rm/reboot.o
  CC      arch/x86/kernel/fpu/regset.o
  AR      fs/notify/dnotify/built-in.a
  CC      mm/mempool.o
  CC      fs/notify/inotify/inotify_fsnotify.o
  AS      arch/x86/realmode/rm/wakeup_asm.o
  CC      arch/x86/pci/init.o
  CC      arch/x86/lib/delay.o
  CC      arch/x86/realmode/rm/wakemain.o
  CC      fs/notify/inotify/inotify_user.o
  CC      arch/x86/platform/efi/efi.o
  CC      security/keys/keyring.o
  CC      arch/x86/realmode/rm/video-mode.o
  CC      lib/kunit/resource.o
  CC      net/llc/llc_input.o
  CC      io_uring/splice.o
  CC      crypto/compress.o
  AS      arch/x86/power/hibernate_asm_64.o
  CC      net/802/p8022.o
  CC      net/802/psnap.o
  AR      arch/x86/entry/vsyscall/built-in.a
  CC      arch/x86/entry/vdso/extable.o
  CC      net/802/stp.o
  CC      arch/x86/power/hibernate.o
  CC      block/partitions/ldm.o
  CC      mm/kasan/report.o
  AS      arch/x86/realmode/rm/copy.o
  CC      io_uring/sync.o
  AS      arch/x86/realmode/rm/bioscall.o
  CC      arch/x86/realmode/rm/regs.o
  CC      arch/x86/realmode/rm/video-vga.o
  AS      arch/x86/lib/getuser.o
  AS      arch/x86/crypto/aesni-intel_avx-x86_64.o
  GEN     arch/x86/lib/inat-tables.c
  CC      arch/x86/events/amd/lbr.o
  CC      security/keys/keyctl.o
  CC      arch/x86/realmode/rm/video-vesa.o
  CC      arch/x86/lib/insn-eval.o
  CC      security/inode.o
  CC      net/llc/llc_output.o
  CC      init/do_mounts_initrd.o
  CC      arch/x86/realmode/rm/video-bios.o
  CC      init/initramfs.o
  CC      ipc/mq_sysctl.o
  CC      init/calibrate.o
  CC      arch/x86/events/amd/ibs.o
  CC      crypto/algapi.o
  AR      net/ethernet/built-in.a
  CC      io_uring/advise.o
  CC      arch/x86/events/amd/uncore.o
  CC      lib/kunit/static_stub.o
  PASYMS  arch/x86/realmode/rm/pasyms.h
  LDS     arch/x86/realmode/rm/realmode.lds
  LD      arch/x86/realmode/rm/realmode.elf
  RELOCS  arch/x86/realmode/rm/realmode.relocs
  CC      arch/x86/pci/mmconfig_64.o
  OBJCOPY arch/x86/realmode/rm/realmode.bin
  AS      arch/x86/realmode/rmpiggy.o
  AS      arch/x86/crypto/aes_ctrby8_avx-x86_64.o
  AR      arch/x86/realmode/built-in.a
  AR      drivers/phy/built-in.a
  CC      arch/x86/platform/efi/efi_64.o
  AR      drivers/pinctrl/actions/built-in.a
  AR      drivers/pinctrl/bcm/built-in.a
  AR      drivers/pinctrl/cirrus/built-in.a
  CC      arch/x86/kernel/fpu/signal.o
  CC      crypto/scatterwalk.o
  AR      drivers/pinctrl/freescale/built-in.a
  CC      drivers/pinctrl/intel/pinctrl-baytrail.o
  CC      drivers/pinctrl/intel/pinctrl-intel.o
  AS [M]  arch/x86/crypto/ghash-clmulni-intel_asm.o
  CC      sound/core/seq/seq_memory.o
  CC [M]  arch/x86/crypto/ghash-clmulni-intel_glue.o
  CC      arch/x86/entry/vdso/vdso32-setup.o
  CC      init/init_task.o
  CC      security/device_cgroup.o
  CC      crypto/proc.o
  AR      arch/x86/power/built-in.a
  CC      sound/core/seq/seq_queue.o
  CC      arch/x86/pci/direct.o
  CC      net/core/skbuff.o
  CC      security/keys/permission.o
  CC      arch/x86/events/intel/ds.o
  CC      arch/x86/mm/pat/memtype.o
  CC      lib/math/div64.o
  CC      lib/crypto/memneq.o
  AR      net/802/built-in.a
  CC      mm/kasan/init.o
  CC      net/sched/sch_generic.o
  CC      arch/x86/events/intel/knc.o
  CC      kernel/locking/mutex.o
  CC      lib/math/gcd.o
  LDS     arch/x86/entry/vdso/vdso.lds
  AR      fs/notify/inotify/built-in.a
  AS      arch/x86/entry/vdso/vdso-note.o
  CC      fs/notify/fanotify/fanotify.o
  CC      arch/x86/entry/vdso/vclock_gettime.o
  CC      lib/math/lcm.o
  CC      fs/notify/fanotify/fanotify_user.o
  CC      lib/kunit/string-stream.o
  CC      lib/math/int_pow.o
  CC      lib/crypto/utils.o
  CC      io_uring/filetable.o
  CC      lib/math/int_sqrt.o
  CC      kernel/locking/semaphore.o
  AR      net/llc/built-in.a
  CC      lib/crypto/chacha.o
  CC      lib/math/reciprocal_div.o
  CC      block/partitions/msdos.o
  CC      lib/math/rational.o
  CC      net/sched/sch_mq.o
  CC      crypto/aead.o
  CC      arch/x86/lib/insn.o
  CC      security/keys/process_keys.o
  AS [M]  arch/x86/crypto/crc32-pclmul_asm.o
  CC [M]  arch/x86/crypto/crc32-pclmul_glue.o
  CC      lib/kunit/assert.o
  CC      net/netlink/af_netlink.o
  CC      drivers/gpio/gpiolib.o
  CC      net/netlink/genetlink.o
  CC      mm/oom_kill.o
  CC      init/version.o
  CC      lib/kunit/try-catch.o
  CC      net/netlink/policy.o
  CC      net/netlink/diag.o
  AS      arch/x86/platform/efi/efi_stub_64.o
  AR      arch/x86/platform/efi/built-in.a
  AR      arch/x86/platform/geode/built-in.a
  AR      arch/x86/platform/iris/built-in.a
  CC      arch/x86/platform/intel/iosf_mbi.o
  CC      arch/x86/pci/mmconfig-shared.o
  CC      sound/core/seq/seq_fifo.o
  CC      arch/x86/entry/vdso/vgetcpu.o
  CC      lib/crypto/aes.o
  CC      arch/x86/kernel/fpu/xstate.o
  CC      fs/notify/fsnotify.o
  CC      security/keys/request_key.o
  HOSTCC  arch/x86/entry/vdso/vdso2c
  AR      ipc/built-in.a
  CC      io_uring/openclose.o
  AR      arch/x86/events/amd/built-in.a
  CC      mm/fadvise.o
  AR      arch/x86/platform/intel-mid/built-in.a
  CC [M]  lib/math/prime_numbers.o
  CC      block/partitions/efi.o
  CC      fs/nfs_common/grace.o
  AR      init/built-in.a
  AS      arch/x86/lib/memcpy_64.o
  AR      net/bpf/built-in.a
  CC      net/ethtool/ioctl.o
  AS      arch/x86/lib/memmove_64.o
  AS      arch/x86/lib/memset_64.o
  CC      io_uring/uring_cmd.o
  CC      arch/x86/lib/misc.o
  AS [M]  arch/x86/crypto/crct10dif-pcl-asm_64.o
  CC      mm/kasan/generic.o
  CC      arch/x86/lib/pc-conf-reg.o
  CC      arch/x86/mm/pat/memtype_interval.o
  CC [M]  arch/x86/crypto/crct10dif-pclmul_glue.o
  CC      fs/notify/notification.o
  CC      mm/kasan/report_generic.o
  CC      lib/kunit/executor.o
  CC      kernel/locking/rwsem.o
  CC      kernel/locking/percpu-rwsem.o
  AS      arch/x86/lib/putuser.o
  LDS     arch/x86/entry/vdso/vdso32/vdso32.lds
  AS      arch/x86/entry/vdso/vdso32/note.o
  AS      arch/x86/lib/retpoline.o
  CC [M]  drivers/pinctrl/intel/pinctrl-cherryview.o
  AS      arch/x86/entry/vdso/vdso32/system_call.o
  CC      crypto/geniv.o
  AS      arch/x86/entry/vdso/vdso32/sigreturn.o
  CC      arch/x86/lib/usercopy.o
  CC      security/keys/request_key_auth.o
  CC      arch/x86/lib/usercopy_64.o
  CC      arch/x86/entry/vdso/vdso32/vclock_gettime.o
  CC      sound/core/seq/seq_prioq.o
  CC      arch/x86/pci/fixup.o
  CC      net/core/datagram.o
  AR      lib/math/built-in.a
  CC      net/core/stream.o
  CC      net/core/scm.o
  CC      lib/crypto/gf128mul.o
  AR      arch/x86/platform/intel/built-in.a
  AR      arch/x86/platform/intel-quark/built-in.a
  CC      arch/x86/events/intel/lbr.o
  AR      arch/x86/platform/olpc/built-in.a
  LD [M]  arch/x86/crypto/ghash-clmulni-intel.o
  AR      arch/x86/platform/scx200/built-in.a
  LD [M]  arch/x86/crypto/crc32-pclmul.o
  AR      arch/x86/platform/ts5500/built-in.a
  LD [M]  arch/x86/crypto/crct10dif-pclmul.o
  AR      arch/x86/platform/uv/built-in.a
  AR      arch/x86/platform/built-in.a
  AR      arch/x86/crypto/built-in.a
  CC      arch/x86/entry/vdso/vdso32/vgetcpu.o
  CC      arch/x86/lib/msr-smp.o
  CC      lib/kunit/hooks.o
  CC      mm/maccess.o
  AR      fs/nfs_common/built-in.a
  CC      security/keys/user_defined.o
  CC      net/core/gen_stats.o
  CC      net/core/gen_estimator.o
  AR      arch/x86/mm/pat/built-in.a
  CC      arch/x86/mm/init.o
  CC      fs/notify/group.o
  CC      mm/page-writeback.o
  CC      security/keys/compat.o
  AR      fs/notify/fanotify/built-in.a
  CC      mm/kasan/shadow.o
  CC      mm/kasan/quarantine.o
  CC      arch/x86/lib/cache-smp.o
  AR      block/partitions/built-in.a
  CC      block/fops.o
  CC      fs/notify/mark.o
  CC      block/bio.o
  AR      lib/kunit/built-in.a
  CC      arch/x86/kernel/cpu/mce/core.o
  CC      mm/folio-compat.o
  CC      arch/x86/kernel/cpu/cacheinfo.o
  CC      arch/x86/kernel/cpu/mtrr/mtrr.o
  CC      arch/x86/kernel/cpu/scattered.o
  CC      arch/x86/kernel/cpu/topology.o
  CC      arch/x86/lib/msr.o
  CC      arch/x86/kernel/cpu/mce/severity.o
  VDSO    arch/x86/entry/vdso/vdso64.so.dbg
  CC      security/keys/proc.o
  VDSO    arch/x86/entry/vdso/vdso32.so.dbg
  OBJCOPY arch/x86/entry/vdso/vdso64.so
  OBJCOPY arch/x86/entry/vdso/vdso32.so
  VDSO2C  arch/x86/entry/vdso/vdso-image-64.c
  VDSO2C  arch/x86/entry/vdso/vdso-image-32.c
  CC      arch/x86/kernel/cpu/common.o
  CC      arch/x86/entry/vdso/vdso-image-64.o
  CC      net/ethtool/common.o
  AR      arch/x86/kernel/fpu/built-in.a
  CC      arch/x86/events/zhaoxin/core.o
  CC      sound/core/seq/seq_timer.o
  CC      crypto/skcipher.o
  CC      lib/crypto/blake2s.o
  AR      sound/i2c/other/built-in.a
  AR      sound/i2c/built-in.a
  AR      sound/drivers/opl3/built-in.a
  AR      sound/drivers/opl4/built-in.a
  CC      security/keys/sysctl.o
  AR      sound/drivers/mpu401/built-in.a
  AR      sound/drivers/vx/built-in.a
  CC      kernel/locking/irqflag-debug.o
  CC      arch/x86/entry/vdso/vdso-image-32.o
  CC      arch/x86/pci/acpi.o
  AR      sound/drivers/pcsp/built-in.a
  AR      sound/isa/ad1816a/built-in.a
  CC      net/sched/sch_frag.o
  AR      sound/drivers/built-in.a
  AR      sound/isa/ad1848/built-in.a
  CC      net/sched/sch_api.o
  AR      sound/isa/cs423x/built-in.a
  CC      arch/x86/kernel/cpu/rdrand.o
  AR      sound/isa/es1688/built-in.a
  AR      sound/isa/galaxy/built-in.a
  AR      sound/isa/gus/built-in.a
  AR      sound/isa/msnd/built-in.a
  CC      lib/crypto/blake2s-generic.o
  CC      arch/x86/kernel/cpu/match.o
  AR      sound/isa/opti9xx/built-in.a
  AR      sound/isa/sb/built-in.a
  CC      fs/notify/fdinfo.o
  AR      sound/isa/wavefront/built-in.a
  CC      fs/iomap/trace.o
  AR      sound/isa/wss/built-in.a
  AR      sound/isa/built-in.a
  CC      fs/iomap/iter.o
  CC [M]  drivers/pinctrl/intel/pinctrl-broxton.o
  AR      sound/pci/ac97/built-in.a
  AR      sound/pci/ali5451/built-in.a
  AR      sound/pci/asihpi/built-in.a
  AR      sound/pci/au88x0/built-in.a
  AR      mm/kasan/built-in.a
  CC      arch/x86/kernel/cpu/bugs.o
  AR      sound/ppc/built-in.a
  AS      arch/x86/lib/msr-reg.o
  AR      sound/pci/aw2/built-in.a
  AR      arch/x86/entry/vdso/built-in.a
  CC      arch/x86/lib/msr-reg-export.o
  CC      arch/x86/entry/syscall_64.o
  AR      sound/pci/ctxfi/built-in.a
  AR      sound/pci/ca0106/built-in.a
  CC      lib/zlib_inflate/inffast.o
  CC      lib/zlib_deflate/deflate.o
  CC      lib/zlib_inflate/inflate.o
  AR      sound/pci/cs46xx/built-in.a
  AR      sound/pci/cs5535audio/built-in.a
  AR      sound/pci/lola/built-in.a
  CC      kernel/locking/mutex-debug.o
  AR      sound/pci/lx6464es/built-in.a
  CC      lib/zlib_inflate/infutil.o
  CC      lib/crypto/blake2s-selftest.o
  AR      sound/pci/echoaudio/built-in.a
  CC      lib/crypto/des.o
  AR      sound/pci/emu10k1/built-in.a
  AR      sound/pci/hda/built-in.a
  CC      lib/zlib_deflate/deftree.o
  CC      lib/zlib_deflate/deflate_syms.o
  CC [M]  sound/pci/hda/hda_bind.o
  CC      arch/x86/mm/init_64.o
  CC      arch/x86/kernel/cpu/mtrr/if.o
  CC [M]  sound/pci/hda/hda_codec.o
  CC [M]  net/netfilter/ipvs/ip_vs_conn.o
  CC      net/netfilter/core.o
  AS      arch/x86/lib/hweight.o
  CC      mm/readahead.o
  CC      arch/x86/events/intel/p4.o
  AR      security/keys/built-in.a
  AR      security/built-in.a
  CC      arch/x86/lib/iomem.o
  AR      drivers/pwm/built-in.a
  AR      net/ipv4/netfilter/built-in.a
  CC      arch/x86/kernel/cpu/aperfmperf.o
  CC      net/xfrm/xfrm_policy.o
  CC [M]  net/ipv4/netfilter/nf_defrag_ipv4.o
  CC      arch/x86/kernel/cpu/cpuid-deps.o
  CC      sound/core/seq/seq_system.o
  CC      net/unix/af_unix.o
  CC      sound/core/seq/seq_ports.o
  AR      arch/x86/events/zhaoxin/built-in.a
  CC      net/xfrm/xfrm_state.o
  CC      net/core/net_namespace.o
  CC      lib/lzo/lzo1x_compress.o
  CC      arch/x86/events/core.o
  CC [M]  drivers/pinctrl/intel/pinctrl-geminilake.o
  CC [M]  net/ipv4/netfilter/nf_reject_ipv4.o
  AR      fs/notify/built-in.a
  CC      arch/x86/pci/legacy.o
  CC      lib/lzo/lzo1x_decompress_safe.o
  CC      net/ipv4/route.o
  CC      net/core/secure_seq.o
  CC      arch/x86/entry/common.o
  AS      arch/x86/lib/iomap_copy_64.o
  CC      kernel/locking/lockdep.o
  CC      arch/x86/lib/inat.o
  CC      io_uring/epoll.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/eventfd.o
  CC      net/core/flow_dissector.o
  CC      lib/zlib_inflate/inftrees.o
  CC      crypto/seqiv.o
  CC      arch/x86/kernel/cpu/mtrr/generic.o
  CC      arch/x86/kernel/acpi/boot.o
  AR      arch/x86/lib/built-in.a
  AR      arch/x86/lib/lib.a
  CC      fs/iomap/buffered-io.o
  CC      drivers/gpio/gpiolib-devres.o
  CC      fs/iomap/direct-io.o
  CC      fs/iomap/fiemap.o
  AR      fs/quota/built-in.a
  CC      fs/iomap/seek.o
  CC      net/ethtool/netlink.o
  AR      net/ipv6/netfilter/built-in.a
  CC      net/packet/af_packet.o
  CC [M]  net/ipv6/netfilter/nf_defrag_ipv6_hooks.o
  AR      lib/zlib_deflate/built-in.a
  CC [M]  net/ipv6/netfilter/nf_conntrack_reasm.o
  AR      net/netlink/built-in.a
  CC      arch/x86/kernel/acpi/sleep.o
  CC      lib/zlib_inflate/inflate_syms.o
  CC      drivers/gpio/gpiolib-legacy.o
  AR      lib/lzo/built-in.a
  CC      sound/core/seq/seq_info.o
  CC [M]  drivers/pinctrl/intel/pinctrl-sunrisepoint.o
  CC      net/core/sysctl_net_core.o
  CC      arch/x86/kernel/cpu/mce/genpool.o
  CC      block/elevator.o
  CC      arch/x86/pci/irq.o
  CC      net/packet/diag.o
  CC      lib/crypto/sha1.o
  CC      arch/x86/events/intel/p6.o
  CC      mm/swap.o
  AR      lib/zlib_inflate/built-in.a
  CC      lib/lz4/lz4_compress.o
  CC [M]  net/ipv4/netfilter/ip_tables.o
  CC      lib/zstd/zstd_compress_module.o
  AS      arch/x86/entry/thunk_64.o
  CC      crypto/echainiv.o
  AS      arch/x86/entry/entry_64_compat.o
  CC      arch/x86/kernel/cpu/umwait.o
  AR      drivers/pinctrl/intel/built-in.a
  CC      io_uring/statx.o
  CC      arch/x86/kernel/cpu/proc.o
  CC      arch/x86/entry/syscall_32.o
  CC      drivers/gpio/gpiolib-cdev.o
  CC      lib/lz4/lz4hc_compress.o
  CC      arch/x86/mm/fault.o
  AR      sound/core/seq/built-in.a
  CC      sound/core/sound.o
  AR      drivers/pinctrl/mediatek/built-in.a
  CC      sound/core/init.o
  AR      drivers/pinctrl/mvebu/built-in.a
  AR      drivers/pinctrl/nomadik/built-in.a
  CC      lib/crypto/sha256.o
  AR      drivers/pinctrl/nuvoton/built-in.a
  CC      mm/truncate.o
  AR      drivers/pinctrl/sprd/built-in.a
  AR      drivers/pinctrl/sunplus/built-in.a
  AR      drivers/pinctrl/ti/built-in.a
  CC      drivers/pinctrl/core.o
  CC      arch/x86/kernel/cpu/mce/intel.o
  CC      arch/x86/kernel/cpu/mtrr/cleanup.o
  CC [M]  net/ipv4/netfilter/iptable_filter.o
  CC      lib/zstd/compress/fse_compress.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/binary_stats.o
  CC      arch/x86/kernel/apic/apic.o
  CC      arch/x86/events/intel/pt.o
  CC      arch/x86/kernel/apic/apic_common.o
  CC      net/sched/sch_blackhole.o
  AS      arch/x86/kernel/acpi/wakeup_64.o
  CC      fs/iomap/swapfile.o
  CC      net/sched/sch_fifo.o
  CC      arch/x86/kernel/acpi/apei.o
  CC [M]  net/netfilter/ipvs/ip_vs_core.o
  CC      crypto/ahash.o
  CC      crypto/shash.o
  CC      arch/x86/kernel/acpi/cppc.o
  AR      arch/x86/entry/built-in.a
  CC [M]  net/netfilter/ipvs/ip_vs_ctl.o
  CC      lib/zstd/compress/hist.o
  CC      net/core/dev.o
  CC      io_uring/net.o
  CC      lib/zstd/compress/huf_compress.o
  CC      block/blk-core.o
  CC      arch/x86/events/intel/uncore.o
  CC [M]  lib/crypto/arc4.o
  CC      net/ethtool/bitset.o
  CC [M]  sound/pci/hda/hda_jack.o
  CC      arch/x86/pci/common.o
  CC      arch/x86/kernel/cpu/mce/threshold.o
  CC      crypto/akcipher.o
  CC      net/ethtool/strset.o
  CC      kernel/sched/fair.o
  LD [M]  net/ipv6/netfilter/nf_defrag_ipv6.o
  CC      net/ipv6/af_inet6.o
  CC      kernel/locking/lockdep_proc.o
  CC      net/netfilter/nf_log.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/vfio.o
  CC      sound/core/memory.o
  AR      lib/crypto/built-in.a
  LD [M]  lib/crypto/libarc4.o
  AR      sound/pci/ice1712/built-in.a
  AR      sound/pci/korg1212/built-in.a
  AR      sound/pci/mixart/built-in.a
  AR      sound/pci/nm256/built-in.a
  CC      arch/x86/kernel/acpi/cstate.o
  CC      kernel/locking/spinlock.o
  CC      lib/xz/xz_dec_syms.o
  AR      arch/x86/kernel/cpu/mtrr/built-in.a
  CC      mm/vmscan.o
  CC      mm/shmem.o
  CC [M]  net/ipv4/netfilter/iptable_mangle.o
  CC      lib/raid6/algos.o
  CC      lib/raid6/recov.o
  CC      lib/xz/xz_dec_stream.o
  HOSTCC  lib/raid6/mktables
  CC      lib/xz/xz_dec_lzma2.o
  CC      arch/x86/mm/ioremap.o
  AR      fs/iomap/built-in.a
  UNROLL  lib/raid6/int1.c
  UNROLL  lib/raid6/int2.c
  UNROLL  lib/raid6/int4.c
  UNROLL  lib/raid6/int8.c
  CC      fs/proc/task_mmu.o
  CC      net/unix/garbage.o
  CC [M]  sound/pci/hda/hda_auto_parser.o
  AR      net/sched/built-in.a
  CC      net/unix/sysctl_net_unix.o
  CC      drivers/pinctrl/pinctrl-utils.o
  CC      lib/xz/xz_dec_bcj.o
  CC      lib/lz4/lz4_decompress.o
  CC [M]  sound/pci/hda/hda_sysfs.o
  CC      sound/core/control.o
  UNROLL  lib/raid6/int16.c
  CC      arch/x86/events/probe.o
  CC      crypto/kpp.o
  CC      arch/x86/kernel/kprobes/core.o
  CC      arch/x86/pci/early.o
  CC      arch/x86/kernel/cpu/mce/apei.o
  CC      arch/x86/kernel/kprobes/opt.o
  CC      drivers/gpio/gpiolib-sysfs.o
  CC [M]  sound/pci/hda/hda_controller.o
  AR      arch/x86/kernel/acpi/built-in.a
  CC      mm/util.o
  CC      io_uring/msg_ring.o
  CC      kernel/locking/osq_lock.o
  CC      net/netfilter/nf_queue.o
  CC      net/netfilter/nf_sockopt.o
  UNROLL  lib/raid6/int32.c
  CC      lib/raid6/recov_ssse3.o
  CC      net/ipv4/inetpeer.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/coalesced_mmio.o
  CC      net/ethtool/linkinfo.o
  CC      block/blk-sysfs.o
  CC      drivers/pinctrl/pinmux.o
  CC      fs/kernfs/mount.o
  CC      drivers/pinctrl/pinconf.o
  AR      lib/xz/built-in.a
  CC      fs/kernfs/inode.o
  CC      kernel/power/qos.o
  CC      lib/zstd/compress/zstd_compress.o
  CC      net/core/dev_addr_lists.o
  CC      arch/x86/kernel/apic/apic_noop.o
  CC      net/core/dst.o
  CC [M]  net/ipv4/netfilter/iptable_nat.o
  CC      arch/x86/pci/bus_numa.o
  CC      lib/zstd/compress/zstd_compress_literals.o
  CC      arch/x86/mm/extable.o
  CC      crypto/acompress.o
  AR      arch/x86/kernel/cpu/mce/built-in.a
  MKCAP   arch/x86/kernel/cpu/capflags.c
  CC      crypto/scompress.o
  CC      arch/x86/events/intel/uncore_nhmex.o
  CC      arch/x86/kernel/apic/ipi.o
  CC      drivers/gpio/gpiolib-acpi.o
  CC      net/core/netevent.o
  CC      kernel/printk/printk.o
  CC      kernel/locking/qspinlock.o
  CC      kernel/printk/printk_safe.o
  CC      lib/raid6/recov_avx2.o
  CC      net/unix/diag.o
  CC      arch/x86/kernel/kprobes/ftrace.o
  CC      lib/raid6/mmx.o
  CC      net/ipv6/anycast.o
  AR      lib/lz4/built-in.a
  CC      net/ipv4/protocol.o
  CC      drivers/pinctrl/pinconf-generic.o
  CC      lib/fonts/fonts.o
  CC      lib/argv_split.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/async_pf.o
  CC      lib/fonts/font_8x8.o
  CC      arch/x86/kernel/apic/vector.o
  CC      lib/fonts/font_8x16.o
  CC [M]  net/netfilter/ipvs/ip_vs_sched.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/irqchip.o
  CC      lib/raid6/sse1.o
  CC      fs/kernfs/dir.o
  CC      io_uring/timeout.o
  CC      net/ethtool/linkmodes.o
  CC      fs/kernfs/file.o
  CC      arch/x86/pci/amd_bus.o
  CC      net/ipv6/ip6_output.o
  AR      net/packet/built-in.a
  CC      block/blk-flush.o
  CC [M]  net/netfilter/ipvs/ip_vs_xmit.o
  AR      sound/arm/built-in.a
  CC      arch/x86/mm/mmap.o
  CC      net/ethtool/rss.o
  CC      net/ipv6/ip6_input.o
  CC      net/ipv6/addrconf.o
  CC      crypto/algboss.o
  CC      block/blk-settings.o
  CC      kernel/locking/rtmutex_api.o
  CC      net/xfrm/xfrm_hash.o
  CC      kernel/printk/printk_ringbuffer.o
  CC      kernel/power/main.o
  CC      kernel/sched/build_policy.o
  CC      kernel/power/console.o
  AR      lib/fonts/built-in.a
  CC      lib/raid6/sse2.o
  CC      crypto/testmgr.o
  CC [M]  sound/pci/hda/hda_proc.o
  AR      arch/x86/kernel/kprobes/built-in.a
  CC      crypto/cmac.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/dirty_ring.o
  AR      drivers/pinctrl/built-in.a
  CC      fs/proc/inode.o
  CC [M]  net/ipv4/netfilter/ipt_REJECT.o
  CC      fs/proc/root.o
  CC      net/ipv6/addrlabel.o
  CC      sound/core/misc.o
  CC      net/core/neighbour.o
  CC      arch/x86/events/intel/uncore_snb.o
  CC      arch/x86/events/intel/uncore_snbep.o
  CC      drivers/gpio/gpiolib-swnode.o
  CC      net/unix/scm.o
  CC      lib/raid6/avx2.o
  AR      arch/x86/pci/built-in.a
  AR      sound/sh/built-in.a
  CC      sound/core/device.o
  CC      net/xfrm/xfrm_input.o
  CC [M]  arch/x86/kvm/../../../virt/kvm/pfncache.o
  CC      arch/x86/mm/pgtable.o
  CC      net/netfilter/utils.o
  CC      net/core/rtnetlink.o
  CC      arch/x86/events/intel/uncore_discovery.o
  CC      kernel/printk/sysctl.o
  CC      arch/x86/mm/physaddr.o
  CC      arch/x86/events/intel/cstate.o
  CC      kernel/power/process.o
  AR      sound/synth/emux/built-in.a
  CC      net/ipv6/route.o
  CC      kernel/power/suspend.o
  AR      sound/synth/built-in.a
  CC      io_uring/sqpoll.o
  CC      kernel/power/hibernate.o
  AR      sound/usb/misc/built-in.a
  CC      net/ethtool/linkstate.o
  CC      net/ethtool/debug.o
  AR      sound/usb/usx2y/built-in.a
  CC      block/blk-ioc.o
  CC      block/blk-map.o
  AR      sound/usb/caiaq/built-in.a
  AR      sound/usb/6fire/built-in.a
  CC      arch/x86/mm/tlb.o
  AR      sound/usb/hiface/built-in.a
  AR      sound/usb/bcd2000/built-in.a
  AR      sound/usb/built-in.a
  CC      fs/kernfs/symlink.o
  AR      sound/firewire/built-in.a
  CC      net/core/utils.o
  AR      drivers/gpio/built-in.a
  AR      sound/sparc/built-in.a
  CC      drivers/video/console/dummycon.o
  CC      drivers/pci/msi/pcidev_msi.o
  CC      fs/proc/base.o
  CC      drivers/pci/pcie/portdrv.o
  CC [M]  net/netfilter/nfnetlink.o
  CC      kernel/locking/spinlock_debug.o
  CC      sound/core/info.o
  CC      net/ipv4/ip_input.o
  CC [M]  sound/pci/hda/hda_hwdep.o
  CC      arch/x86/kernel/apic/hw_nmi.o
  CC      lib/raid6/avx512.o
  CC      arch/x86/mm/cpu_entry_area.o
  CC      kernel/sched/build_utility.o
  AR      net/unix/built-in.a
  CC      net/key/af_key.o
  CC      net/ipv6/ip6_fib.o
  AR      kernel/printk/built-in.a
  CC      kernel/irq/irqdesc.o
  CC [M]  arch/x86/kvm/x86.o
  CC [M]  arch/x86/kvm/emulate.o
  AR      fs/kernfs/built-in.a
  CC      fs/sysfs/file.o
  CC      kernel/irq/handle.o
  CC      drivers/video/console/vgacon.o
  CC      kernel/locking/qrwlock.o
  CC      drivers/pci/msi/api.o
  CC      drivers/pci/msi/msi.o
  CC [M]  net/netfilter/ipvs/ip_vs_app.o
  CC      lib/raid6/recov_avx512.o
  CC      drivers/pci/msi/irqdomain.o
  CC      net/ipv4/ip_fragment.o
  CC      kernel/irq/manage.o
  CC      drivers/pci/pcie/rcec.o
  CC      net/ethtool/wol.o
  CC      arch/x86/kernel/apic/io_apic.o
  CC      net/ethtool/features.o
  CC [M]  sound/pci/hda/hda_generic.o
  CC      block/blk-merge.o
  CC      io_uring/fdinfo.o
  CC      arch/x86/mm/maccess.o
  CC      kernel/power/snapshot.o
  CC      kernel/irq/spurious.o
  CC      arch/x86/mm/pgprot.o
  CC      arch/x86/kernel/apic/msi.o
  CC      fs/sysfs/dir.o
  CC      sound/core/isadma.o
  AR      kernel/locking/built-in.a
  CC      net/ipv4/ip_forward.o
  CC      mm/mmzone.o
  CC      net/ipv4/ip_options.o
  CC      drivers/pci/pcie/aspm.o
  CC      lib/zstd/compress/zstd_compress_sequences.o
  CC      net/xfrm/xfrm_output.o
  CC      arch/x86/mm/hugetlbpage.o
  CC [M]  sound/pci/hda/patch_realtek.o
  TABLE   lib/raid6/tables.c
  CC      lib/raid6/int1.o
  AR      net/bridge/netfilter/built-in.a
  CC      net/bridge/br.o
  CC      arch/x86/kernel/cpu/powerflags.o
  CC      crypto/hmac.o
  CC      kernel/irq/resend.o
  CC      sound/core/vmaster.o
  CC      net/bridge/br_device.o
  CC      fs/sysfs/symlink.o
  CC      fs/sysfs/mount.o
  CC      io_uring/tctx.o
  CC      io_uring/poll.o
  CC      arch/x86/kernel/cpu/feat_ctl.o
  AR      drivers/video/console/built-in.a
  CC      lib/zstd/compress/zstd_compress_superblock.o
  CC      drivers/video/logo/logo.o
  AR      arch/x86/events/intel/built-in.a
  AR      drivers/pci/msi/built-in.a
  CC      mm/vmstat.o
  CC      arch/x86/events/utils.o
  CC      mm/backing-dev.o
  CC      kernel/rcu/update.o
  CC      kernel/rcu/sync.o
  CC      arch/x86/kernel/apic/x2apic_phys.o
  CC      net/ethtool/privflags.o
  CC      lib/zstd/compress/zstd_double_fast.o
  HOSTCC  drivers/video/logo/pnmtologo
  CC      arch/x86/mm/kasan_init_64.o
  CC      lib/raid6/int2.o
  CC [M]  net/netfilter/ipvs/ip_vs_sync.o
  CC      arch/x86/mm/pkeys.o
  CC      lib/zstd/compress/zstd_fast.o
  CC      arch/x86/kernel/cpu/intel.o
  CC      lib/zstd/compress/zstd_lazy.o
  AR      kernel/livepatch/built-in.a
  CC      arch/x86/mm/pti.o
  CC      sound/core/ctljack.o
  CC      arch/x86/kernel/cpu/intel_pconfig.o
  CC      crypto/vmac.o
  CC [M]  net/netfilter/ipvs/ip_vs_est.o
  CC      kernel/irq/chip.o
  CC      fs/sysfs/group.o
  LOGO    drivers/video/logo/logo_linux_clut224.c
  CC      kernel/irq/dummychip.o
  CC      drivers/video/logo/logo_linux_clut224.o
  CC      fs/proc/generic.o
  CC      arch/x86/kernel/cpu/tsx.o
  CC      drivers/pci/pcie/aer.o
  AR      drivers/video/logo/built-in.a
  CC      arch/x86/events/rapl.o
  CC      kernel/power/swap.o
  CC      net/ipv6/ipv6_sockglue.o
  CC      drivers/video/backlight/backlight.o
  CC      drivers/video/fbdev/core/fb_notify.o
  AR      drivers/video/fbdev/omap/built-in.a
  CC      block/blk-timeout.o
  CC      io_uring/cancel.o
  CC      net/ipv4/ip_output.o
  CC      net/bridge/br_fdb.o
  CC      sound/core/jack.o
  CC      arch/x86/kernel/apic/x2apic_cluster.o
  CC      lib/raid6/int4.o
  CC      fs/configfs/inode.o
  CC      net/bridge/br_forward.o
  CC      fs/devpts/inode.o
  CC      net/ethtool/rings.o
  CC      net/bridge/br_if.o
  CC      net/core/link_watch.o
  CC      net/core/filter.o
  CC      drivers/video/aperture.o
  CC      io_uring/kbuf.o
  AR      arch/x86/mm/built-in.a
  AR      fs/sysfs/built-in.a
  AR      drivers/video/fbdev/omap2/omapfb/dss/built-in.a
  CC      lib/bug.o
  AR      drivers/video/fbdev/omap2/omapfb/displays/built-in.a
  AR      drivers/video/fbdev/omap2/omapfb/built-in.a
  AR      drivers/video/fbdev/omap2/built-in.a
  CC      block/blk-lib.o
  CC      net/xfrm/xfrm_sysctl.o
  AR      net/key/built-in.a
  CC      crypto/xcbc.o
  CC [M]  net/sunrpc/auth_gss/auth_gss.o
  CC      net/8021q/vlan_core.o
  CC      mm/mm_init.o
  CC [M]  net/sunrpc/auth_gss/gss_generic_token.o
  CC [M]  drivers/video/fbdev/core/fbmem.o
  CC      fs/proc/array.o
  CC      arch/x86/events/msr.o
  CC      arch/x86/kernel/cpu/intel_epb.o
  CC      kernel/rcu/srcutree.o
  CC      sound/core/timer.o
  CC      arch/x86/kernel/apic/apic_flat_64.o
  AR      drivers/video/backlight/built-in.a
  CC      kernel/irq/devres.o
  CC      net/sunrpc/clnt.o
  CC      lib/raid6/int8.o
  CC      fs/configfs/file.o
  CC      drivers/pci/pcie/err.o
  AR      fs/devpts/built-in.a
  CC      fs/ext4/balloc.o
  CC      fs/ext4/bitmap.o
  CC      crypto/crypto_null.o
  CC      io_uring/rsrc.o
  CC      drivers/video/cmdline.o
  CC      arch/x86/kernel/cpu/amd.o
  CC      arch/x86/kernel/apic/probe_64.o
  CC      kernel/power/user.o
  CC      block/blk-mq.o
  CC      net/ethtool/channels.o
  CC      fs/proc/fd.o
  CC [M]  net/netfilter/ipvs/ip_vs_proto.o
  CC      kernel/irq/autoprobe.o
  CC      mm/percpu.o
  CC      net/ipv6/ndisc.o
  CC      net/ipv6/udp.o
  CC      lib/raid6/int16.o
  CC      fs/proc/proc_tty.o
  CC      fs/proc/cmdline.o
  AR      arch/x86/events/built-in.a
  CC      kernel/power/poweroff.o
  CC      net/dcb/dcbnl.o
  AR      arch/x86/kernel/apic/built-in.a
  CC      net/xfrm/xfrm_replay.o
  CC      net/l3mdev/l3mdev.o
  LDS     arch/x86/kernel/vmlinux.lds
  CC      kernel/irq/irqdomain.o
  CC [M]  net/8021q/vlan.o
  CC      fs/proc/consoles.o
  CC      fs/configfs/dir.o
  CC      drivers/pci/pcie/aer_inject.o
  CC      net/core/sock_diag.o
  CC      io_uring/rw.o
  CC      crypto/md5.o
  CC      net/xfrm/xfrm_device.o
  CC      kernel/irq/proc.o
  CC [M]  net/sunrpc/auth_gss/gss_mech_switch.o
  CC      io_uring/opdef.o
  CC      net/ethtool/coalesce.o
  CC      block/blk-mq-tag.o
  CC      net/ipv6/udplite.o
  CC      drivers/video/nomodeset.o
  CC      lib/raid6/int32.o
  CC [M]  sound/pci/hda/patch_analog.o
  AR      kernel/power/built-in.a
  CC      kernel/rcu/tree.o
  CC      kernel/dma/mapping.o
  CC [M]  net/sunrpc/auth_gss/svcauth_gss.o
  CC      arch/x86/kernel/cpu/hygon.o
  CC      kernel/irq/migration.o
  CC      fs/proc/cpuinfo.o
  CC [M]  drivers/video/fbdev/core/fbmon.o
  CC      fs/configfs/symlink.o
  CC      drivers/pci/pcie/pme.o
  CC      crypto/sha1_generic.o
  CC [M]  drivers/video/fbdev/uvesafb.o
  CC      net/bridge/br_input.o
  AR      net/l3mdev/built-in.a
  CC      fs/ext4/block_validity.o
  CC      sound/core/hrtimer.o
  CC      drivers/pci/pcie/dpc.o
  CC      lib/raid6/tables.o
  CC      kernel/irq/cpuhotplug.o
  CC [M]  net/netfilter/ipvs/ip_vs_pe.o
  CC      fs/configfs/mount.o
  CC      net/ipv4/ip_sockglue.o
  CC      arch/x86/kernel/cpu/centaur.o
  CC      fs/proc/devices.o
  CC [M]  net/8021q/vlan_dev.o
  CC      kernel/irq/pm.o
  CC      arch/x86/kernel/cpu/zhaoxin.o
  CC [M]  drivers/video/fbdev/simplefb.o
  CC [M]  sound/pci/hda/patch_hdmi.o
  CC      fs/proc/interrupts.o
  CC      drivers/video/hdmi.o
  CC [M]  drivers/video/fbdev/core/fbcmap.o
  CC [M]  net/8021q/vlan_netlink.o
  CC [M]  net/8021q/vlanproc.o
  CC      net/core/dev_ioctl.o
  CC      crypto/sha256_generic.o
  CC      sound/core/seq_device.o
  CC      net/xfrm/xfrm_algo.o
  CC      block/blk-stat.o
  CC      net/ethtool/pause.o
  CC [M]  drivers/video/fbdev/core/fbsysfs.o
  CC      fs/proc/loadavg.o
  CC      net/core/tso.o
  CC      kernel/dma/direct.o
  CC [M]  net/bluetooth/af_bluetooth.o
  AR      kernel/sched/built-in.a
  CC      kernel/dma/ops_helpers.o
  CC [M]  net/dns_resolver/dns_key.o
  CC      drivers/idle/intel_idle.o
  CC      arch/x86/kernel/cpu/perfctr-watchdog.o
  AR      drivers/char/ipmi/built-in.a
  CC      lib/zstd/compress/zstd_ldm.o
  CC      fs/configfs/item.o
  CC      fs/ext4/dir.o
  AR      lib/raid6/built-in.a
  CC      net/core/sock_reuseport.o
  AR      drivers/pci/pcie/built-in.a
  CC      block/blk-mq-sysfs.o
  CC      drivers/pci/hotplug/pci_hotplug_core.o
  CC      io_uring/notif.o
  CC      lib/zstd/compress/zstd_opt.o
  CC      net/dcb/dcbevent.o
  CC      crypto/sha512_generic.o
  CC      drivers/pci/hotplug/acpi_pcihp.o
  CC      fs/proc/meminfo.o
  CC      kernel/irq/msi.o
  CC [M]  sound/core/control_led.o
  CC [M]  net/netfilter/ipvs/ip_vs_proto_tcp.o
  CC [M]  net/netfilter/ipvs/ip_vs_proto_udp.o
  CC      io_uring/io-wq.o
  CC [M]  drivers/video/fbdev/core/modedb.o
  CC      net/bridge/br_ioctl.o
  CC      kernel/irq/affinity.o
  CC [M]  drivers/video/fbdev/core/fbcvt.o
  CC      net/core/fib_notifier.o
  CC      drivers/pci/hotplug/pciehp_core.o
  AR      fs/configfs/built-in.a
  CC      fs/proc/stat.o
  CC [M]  sound/pci/hda/hda_eld.o
  CC      net/core/xdp.o
  CC [M]  sound/pci/hda/hda_intel.o
  CC [M]  net/dns_resolver/dns_query.o
  CC      arch/x86/kernel/cpu/vmware.o
  CC      mm/slab_common.o
  CC      fs/jbd2/transaction.o
  AR      net/8021q/built-in.a
  LD [M]  net/8021q/8021q.o
  CC      net/ethtool/eee.o
  CC      kernel/dma/dummy.o
  CC      fs/jbd2/commit.o
  CC      kernel/dma/contiguous.o
  CC      net/xfrm/xfrm_user.o
  CC      fs/jbd2/recovery.o
  CC      net/ipv6/raw.o
  CC      net/devres.o
  CC      drivers/pci/hotplug/pciehp_ctrl.o
  CC      net/bridge/br_stp.o
  CC      kernel/rcu/rcu_segcblist.o
  AR      net/dcb/built-in.a
  CC      net/socket.o
  CC      crypto/blake2b_generic.o
  CC [M]  net/sunrpc/auth_gss/gss_rpc_upcall.o
  CC      fs/ext4/ext4_jbd2.o
  CC      fs/ramfs/inode.o
  CC      fs/ramfs/file-mmu.o
  CC      fs/ext4/extents.o
  CC      net/core/flow_offload.o
  CC [M]  sound/core/hwdep.o
  AR      drivers/idle/built-in.a
  CC      lib/zstd/zstd_decompress_module.o
  CC      fs/proc/uptime.o
  CC      arch/x86/kernel/cpu/hypervisor.o
  CC      drivers/pci/hotplug/pciehp_pci.o
  CC [M]  net/bluetooth/hci_core.o
  LD [M]  net/dns_resolver/dns_resolver.o
  CC      kernel/dma/swiotlb.o
  CC      lib/zstd/decompress/huf_decompress.o
  CC      net/compat.o
  AR      drivers/pci/controller/dwc/built-in.a
  AR      drivers/pci/controller/mobiveil/built-in.a
  CC      drivers/pci/controller/vmd.o
  AR      drivers/pci/switch/built-in.a
  CC      net/ipv4/inet_hashtables.o
  CC      lib/zstd/decompress/zstd_ddict.o
  CC      kernel/dma/remap.o
  CC [M]  drivers/video/fbdev/core/fb_cmdline.o
  CC      drivers/pci/access.o
  CC      kernel/irq/matrix.o
  CC      arch/x86/kernel/cpu/mshyperv.o
  CC [M]  drivers/video/fbdev/core/fb_defio.o
  CC      drivers/pci/hotplug/pciehp_hpc.o
  CC      lib/zstd/decompress/zstd_decompress.o
  CC      lib/buildid.o
  CC      lib/cmdline.o
  CC      net/core/gro.o
  CC      net/core/netdev-genl.o
  CC      drivers/pci/hotplug/acpiphp_core.o
  CC      net/ethtool/tsinfo.o
  CC      fs/proc/util.o
  AR      fs/ramfs/built-in.a
  CC [M]  net/bluetooth/hci_conn.o
  LD [M]  sound/pci/hda/snd-hda-codec.o
  CC      block/blk-mq-cpumap.o
  CC      lib/cpumask.o
  CC [M]  net/netfilter/ipvs/ip_vs_nfct.o
  CC      net/bridge/br_stp_bpdu.o
  LD [M]  sound/pci/hda/snd-hda-codec-generic.o
  LD [M]  sound/pci/hda/snd-hda-codec-realtek.o
  CC      lib/zstd/decompress/zstd_decompress_block.o
  CC      crypto/ecb.o
  CC [M]  sound/core/pcm.o
  CC      drivers/pci/bus.o
  CC [M]  net/sunrpc/auth_gss/gss_rpc_xdr.o
  AR      io_uring/built-in.a
  CC      net/sunrpc/xprt.o
  CC      kernel/entry/common.o
  CC [M]  net/sunrpc/auth_gss/trace.o
  CC      fs/proc/version.o
  CC      kernel/entry/syscall_user_dispatch.o
  CC      net/ethtool/cabletest.o
  LD [M]  sound/pci/hda/snd-hda-codec-analog.o
  CC      net/bridge/br_stp_if.o
  CC      fs/hugetlbfs/inode.o
  LD [M]  sound/pci/hda/snd-hda-codec-hdmi.o
  LD [M]  sound/pci/hda/snd-hda-intel.o
  CC      net/core/netdev-genl-gen.o
  AR      sound/pci/oxygen/built-in.a
  AR      sound/pci/pcxhr/built-in.a
  CC      arch/x86/kernel/cpu/capflags.o
  AR      sound/pci/riptide/built-in.a
  CC      lib/ctype.o
  CC      mm/compaction.o
  AR      sound/pci/rme9652/built-in.a
  CC      crypto/cbc.o
  CC      lib/dec_and_lock.o
  AR      sound/pci/trident/built-in.a
  CC      crypto/pcbc.o
  AR      sound/pci/ymfpci/built-in.a
  CC      crypto/cts.o
  AR      arch/x86/kernel/cpu/built-in.a
  AR      sound/pci/vx222/built-in.a
  CC      crypto/lrw.o
  AR      sound/pci/built-in.a
  AS      arch/x86/kernel/head_64.o
  CC      block/blk-mq-sched.o
  CC      fs/jbd2/checkpoint.o
  CC [M]  drivers/video/fbdev/core/fbcon.o
  AR      drivers/pci/controller/built-in.a
  CC      fs/fat/cache.o
  CC      arch/x86/kernel/head64.o
  CC [M]  net/sunrpc/auth_gss/gss_krb5_mech.o
  AR      sound/spi/built-in.a
  CC      lib/zstd/zstd_common_module.o
  AR      kernel/dma/built-in.a
  CC      fs/ext4/extents_status.o
  CC      drivers/acpi/acpica/dsargs.o
  CC      fs/ext4/file.o
  CC      kernel/entry/kvm.o
  CC      net/sysctl_net.o
  CC      drivers/pci/hotplug/acpiphp_glue.o
  CC      drivers/acpi/acpica/dscontrol.o
  CC      fs/proc/softirqs.o
  CC      drivers/pci/probe.o
  AR      kernel/irq/built-in.a
  CC [M]  sound/core/pcm_native.o
  CC [M]  sound/core/pcm_lib.o
  CC      net/ipv6/icmp.o
  CC      net/ipv6/mcast.o
  CC      fs/ext4/fsmap.o
  CC [M]  net/netfilter/ipvs/ip_vs_rr.o
  CC      fs/fat/dir.o
  CC      lib/zstd/common/debug.o
  AR      sound/parisc/built-in.a
  AR      sound/mips/built-in.a
  AR      sound/pcmcia/vx/built-in.a
  CC      crypto/xts.o
  AR      sound/pcmcia/pdaudiocf/built-in.a
  CC      crypto/ctr.o
  AR      sound/pcmcia/built-in.a
  CC      drivers/acpi/acpica/dsdebug.o
  CC      drivers/pci/host-bridge.o
  CC      drivers/acpi/acpica/dsfield.o
  CC      fs/fat/fatent.o
  CC      drivers/acpi/apei/apei-base.o
  CC      arch/x86/kernel/ebda.o
  CC      fs/proc/namespaces.o
  CC      drivers/acpi/apei/hest.o
  CC      arch/x86/kernel/platform-quirks.o
  CC      net/core/net-sysfs.o
  CC      net/core/net-procfs.o
  CC      net/ethtool/tunnels.o
  AR      net/xfrm/built-in.a
  CC      fs/nfs/client.o
  AR      kernel/entry/built-in.a
  CC      net/ethtool/fec.o
  CC      net/ethtool/eeprom.o
  CC      fs/jbd2/revoke.o
  CC [M]  drivers/video/fbdev/core/bitblit.o
  CC      net/bridge/br_stp_timer.o
  CC      block/ioctl.o
  AR      kernel/rcu/built-in.a
  CC      kernel/module/main.o
  CC      block/genhd.o
  CC      drivers/acpi/apei/erst.o
  CC [M]  sound/core/pcm_misc.o
  CC      net/ipv4/inet_timewait_sock.o
  CC      net/ipv4/inet_connection_sock.o
  AR      fs/hugetlbfs/built-in.a
  CC      fs/proc/self.o
  CC [M]  sound/core/pcm_memory.o
  CC      drivers/acpi/acpica/dsinit.o
  CC      arch/x86/kernel/process_64.o
  CC      lib/zstd/common/entropy_common.o
  AR      drivers/pci/hotplug/built-in.a
  CC      net/bridge/br_netlink.o
  CC      crypto/gcm.o
  CC      fs/proc/thread_self.o
  CC      drivers/acpi/apei/bert.o
  LD [M]  net/netfilter/ipvs/ip_vs.o
  CC      fs/proc/proc_sysctl.o
  CC [M]  net/netfilter/nf_conntrack_core.o
  CC      drivers/acpi/apei/ghes.o
  CC      drivers/acpi/acpica/dsmethod.o
  CC [M]  net/sunrpc/auth_gss/gss_krb5_seal.o
  CC      fs/proc/proc_net.o
  CC      kernel/module/strict_rwx.o
  CC      net/ipv4/tcp.o
  CC [M]  drivers/video/fbdev/core/softcursor.o
  CC      net/core/netpoll.o
  CC      fs/jbd2/journal.o
  CC [M]  net/bluetooth/hci_event.o
  CC      lib/decompress.o
  CC      net/sunrpc/socklib.o
  CC      net/sunrpc/xprtsock.o
  CC      net/ipv4/tcp_input.o
  CC      kernel/module/tree_lookup.o
  CC      lib/decompress_bunzip2.o
  CC      fs/fat/file.o
  CC [M]  sound/core/memalloc.o
  CC      lib/decompress_inflate.o
  CC      fs/exportfs/expfs.o
  CC      drivers/pci/remove.o
  CC [M]  sound/core/pcm_timer.o
  CC      net/ethtool/stats.o
  CC      net/ipv6/reassembly.o
  CC      fs/lockd/clntlock.o
  CC      block/ioprio.o
  CC      kernel/module/debug_kmemleak.o
  CC      drivers/acpi/acpica/dsmthdat.o
  CC      fs/nls/nls_base.o
  CC      crypto/pcrypt.o
  CC      fs/nls/nls_cp437.o
  CC      net/sunrpc/sched.o
  CC      fs/proc/kcore.o
  CC      kernel/module/kallsyms.o
  CC      arch/x86/kernel/signal.o
  CC      fs/nls/nls_ascii.o
  CC [M]  net/sunrpc/auth_gss/gss_krb5_unseal.o
  CC      fs/lockd/clntproc.o
  CC      fs/lockd/clntxdr.o
  CC      fs/ext4/fsync.o
  CC      net/ethtool/phc_vclocks.o
  CC      fs/fat/inode.o
  CC [M]  drivers/video/fbdev/core/tileblit.o
  CC [M]  drivers/video/fbdev/core/cfbfillrect.o
  CC      net/ethtool/mm.o
  CC      fs/proc/kmsg.o
  AR      fs/exportfs/built-in.a
  CC      drivers/pci/pci.o
  CC      arch/x86/kernel/signal_64.o
  CC      drivers/acpi/acpica/dsobject.o
  CC      fs/nls/nls_iso8859-1.o
  CC      lib/decompress_unlz4.o
  CC      fs/proc/page.o
  CC      fs/nls/nls_utf8.o
  CC      fs/nfs/dir.o
  CC      net/ipv4/tcp_output.o
  AR      drivers/acpi/apei/built-in.a
  LD [M]  sound/core/snd-ctl-led.o
  CC      crypto/cryptd.o
  LD [M]  sound/core/snd-hwdep.o
  AR      sound/core/built-in.a
  CC      crypto/des_generic.o
  CC      net/core/fib_rules.o
  CC      mm/interval_tree.o
  CC      fs/lockd/host.o
  CC      fs/fat/misc.o
  CC      block/badblocks.o
  CC [M]  net/sunrpc/auth_gss/gss_krb5_seqnum.o
  CC [M]  net/sunrpc/auth_gss/gss_krb5_wrap.o
  AR      fs/nls/built-in.a
  AR      fs/unicode/built-in.a
  CC      fs/ntfs/aops.o
  CC [M]  net/sunrpc/auth_gss/gss_krb5_crypto.o
  CC      fs/autofs/init.o
  CC      net/bridge/br_netlink_tunnel.o
  CC      fs/debugfs/inode.o
  CC      fs/fat/nfs.o
  CC      drivers/acpi/acpica/dsopcode.o
  CC      fs/debugfs/file.o
  CC      lib/decompress_unlzma.o
  LD [M]  sound/core/snd-pcm.o
  CC      net/ethtool/module.o
  CC      net/ethtool/pse-pd.o
  CC      fs/ext4/hash.o
  AR      sound/soc/built-in.a
  AR      sound/atmel/built-in.a
  AR      sound/hda/built-in.a
  CC      kernel/module/procfs.o
  CC [M]  sound/hda/hda_bus_type.o
  CC      fs/ext4/ialloc.o
  CC [M]  sound/hda/hdac_bus.o
  CC      mm/list_lru.o
  CC      arch/x86/kernel/traps.o
  AR      sound/x86/built-in.a
  CC [M]  drivers/video/fbdev/core/cfbcopyarea.o
  CC      net/core/net-traces.o
  CC      fs/tracefs/inode.o
  CC      fs/ext4/indirect.o
  CC      kernel/time/time.o
  CC      fs/autofs/inode.o
  AR      fs/proc/built-in.a
  CC      block/blk-rq-qos.o
  CC      kernel/time/timer.o
  CC [M]  net/sunrpc/auth_gss/gss_krb5_keys.o
  CC      fs/fat/namei_vfat.o
  CC      drivers/acpi/acpica/dspkginit.o
  CC      net/ethtool/plca.o
  CC      fs/fat/namei_msdos.o
  CC      crypto/aes_generic.o
  CC [M]  drivers/video/fbdev/core/cfbimgblt.o
  CC      net/core/selftests.o
  CC      block/disk-events.o
  CC [M]  arch/x86/kvm/i8259.o
  CC      kernel/module/sysfs.o
  CC [M]  sound/hda/hdac_device.o
  CC      drivers/acpi/acpica/dsutils.o
  CC      net/ipv6/tcp_ipv6.o
  LD [M]  net/sunrpc/auth_gss/auth_rpcgss.o
  CC      fs/lockd/svc.o
  CC      block/blk-ia-ranges.o
  CC      net/bridge/br_arp_nd_proxy.o
  CC      fs/nfs/file.o
  CC      fs/ntfs/attrib.o
  AR      fs/debugfs/built-in.a
  CC [M]  net/netfilter/nf_conntrack_standalone.o
  CC      net/ipv6/ping.o
  CC [M]  net/netfilter/nf_conntrack_expect.o
  CC [M]  net/netfilter/nf_conntrack_helper.o
  CC      kernel/time/hrtimer.o
  AR      fs/tracefs/built-in.a
  CC      drivers/pnp/pnpacpi/core.o
  AR      drivers/amba/built-in.a
  CC      drivers/pnp/core.o
  AR      drivers/clk/actions/built-in.a
  AR      drivers/clk/analogbits/built-in.a
  CC      drivers/dma/dw/core.o
  CC      fs/btrfs/super.o
  AR      drivers/clk/bcm/built-in.a
  AR      drivers/clk/imgtec/built-in.a
  CC      fs/autofs/root.o
  AR      drivers/clk/imx/built-in.a
  AR      drivers/clk/ingenic/built-in.a
  AR      drivers/clk/mediatek/built-in.a
  AR      drivers/clk/microchip/built-in.a
  AR      drivers/clk/mstar/built-in.a
  CC      mm/workingset.o
  CC      block/bsg.o
  AR      drivers/clk/mvebu/built-in.a
  AR      drivers/clk/ralink/built-in.a
  AR      drivers/clk/renesas/built-in.a
  AR      drivers/clk/socfpga/built-in.a
  CC      arch/x86/kernel/idt.o
  AR      drivers/clk/sprd/built-in.a
  AR      drivers/clk/sunxi-ng/built-in.a
  AR      drivers/clk/ti/built-in.a
  AR      drivers/clk/versatile/built-in.a
  CC      drivers/acpi/acpica/dswexec.o
  CC      drivers/clk/x86/clk-lpss-atom.o
  LD [M]  net/sunrpc/auth_gss/rpcsec_gss_krb5.o
  AR      fs/jbd2/built-in.a
  CC      drivers/clk/x86/clk-pmc-atom.o
  CC      lib/decompress_unlzo.o
  CC      lib/decompress_unxz.o
  CC      fs/btrfs/ctree.o
  CC      net/sunrpc/auth.o
  CC [M]  net/bluetooth/mgmt.o
  CC      net/sunrpc/auth_null.o
  AR      net/ethtool/built-in.a
  CC [M]  drivers/video/fbdev/core/sysfillrect.o
  AR      kernel/module/built-in.a
  AR      drivers/soc/apple/built-in.a
  CC      drivers/virtio/virtio.o
  AR      drivers/soc/aspeed/built-in.a
  AR      drivers/soc/bcm/bcm63xx/built-in.a
  AR      drivers/soc/bcm/built-in.a
  AR      drivers/soc/fsl/built-in.a
  AR      drivers/soc/fujitsu/built-in.a
  CC      net/ipv6/exthdrs.o
  AR      drivers/soc/imx/built-in.a
  AR      drivers/soc/ixp4xx/built-in.a
  AR      fs/fat/built-in.a
  AR      drivers/soc/loongson/built-in.a
  AR      drivers/soc/mediatek/built-in.a
  AR      drivers/soc/microchip/built-in.a
  CC      drivers/tty/vt/vt_ioctl.o
  AR      drivers/soc/nuvoton/built-in.a
  AR      drivers/soc/pxa/built-in.a
  AR      drivers/soc/amlogic/built-in.a
  CC [M]  sound/hda/hdac_sysfs.o
  CC      drivers/pnp/pnpacpi/rsparser.o
  AR      drivers/soc/qcom/built-in.a
  AR      drivers/soc/renesas/built-in.a
  AR      drivers/soc/rockchip/built-in.a
  AR      drivers/soc/sifive/built-in.a
  CC      drivers/tty/vt/vc_screen.o
  AR      drivers/soc/sunxi/built-in.a
  CC      drivers/tty/vt/selection.o
  CC      crypto/deflate.o
  AR      drivers/soc/ti/built-in.a
  AR      drivers/soc/xilinx/built-in.a
  AR      drivers/soc/built-in.a
  CC      drivers/virtio/virtio_ring.o
  CC      drivers/acpi/acpica/dswload.o
  CC [M]  arch/x86/kvm/irq.o
  CC      lib/decompress_unzstd.o
  CC      fs/nfs/getroot.o
  CC      block/bsg-lib.o
  AR      drivers/clk/x86/built-in.a
  CC      arch/x86/kernel/irq.o
  CC      fs/nfs/inode.o
  AR      drivers/clk/xilinx/built-in.a
  CC      fs/nfs/super.o
  CC      fs/ntfs/collate.o
  CC      drivers/clk/clk-devres.o
  CC      fs/autofs/symlink.o
  CC      fs/lockd/svclock.o
  CC      net/bridge/br_sysfs_if.o
  CC      mm/debug.o
  CC      fs/pstore/inode.o
  CC [M]  net/netfilter/nf_conntrack_proto.o
  CC      drivers/acpi/acpica/dswload2.o
  CC      fs/pstore/platform.o
  CC      fs/lockd/svcshare.o
  CC [M]  drivers/video/fbdev/core/syscopyarea.o
  CC      drivers/pci/pci-driver.o
  CC      drivers/pci/search.o
  CC      drivers/dma/dw/dw.o
  CC      mm/gup.o
  CC      crypto/crc32c_generic.o
  CC      fs/nfs/io.o
  CC      crypto/crct10dif_common.o
  CC      drivers/clk/clk-bulk.o
  CC      fs/ext4/inline.o
  CC      fs/efivarfs/inode.o
  CC      fs/autofs/waitq.o
  CC [M]  sound/hda/hdac_regmap.o
  CC [M]  fs/netfs/buffered_read.o
  CC      kernel/time/timekeeping.o
  CC      fs/ntfs/compress.o
  CC [M]  fs/netfs/io.o
  AR      drivers/pnp/pnpacpi/built-in.a
  CC      drivers/pnp/card.o
  CC [M]  arch/x86/kvm/lapic.o
  CC      fs/nfs/direct.o
  CC      block/blk-cgroup.o
  CC      fs/nfs/pagelist.o
  CC      drivers/tty/vt/keyboard.o
  CC      net/ipv6/datagram.o
  CC      drivers/acpi/acpica/dswscope.o
  CC      net/sunrpc/auth_unix.o
  CC      crypto/crct10dif_generic.o
  CC      drivers/pci/pci-sysfs.o
  CC      fs/ntfs/debug.o
  CC      fs/pstore/pmsg.o
  CC      drivers/clk/clkdev.o
  CC      drivers/dma/dw/idma32.o
  CC      net/bridge/br_sysfs_br.o
  CC      block/blk-cgroup-rwstat.o
  CC      fs/efivarfs/file.o
  CC      fs/autofs/expire.o
  CC      net/sunrpc/svc.o
  CC [M]  drivers/video/fbdev/core/sysimgblt.o
  CC      net/sunrpc/svcsock.o
  CC [M]  sound/hda/hdac_controller.o
  CC      net/sunrpc/svcauth.o
  CC      drivers/acpi/acpica/dswstate.o
  CC      fs/lockd/svcproc.o
  CC      crypto/authenc.o
  CC      net/sunrpc/svcauth_unix.o
  CC      drivers/virtio/virtio_anchor.o
  CC      drivers/pnp/driver.o
  CC      net/sunrpc/addr.o
  CC      fs/efivarfs/super.o
  CC      net/ipv4/tcp_timer.o
  CC      fs/lockd/svcsubs.o
  AR      fs/pstore/built-in.a
  CC [M]  fs/fscache/cache.o
  CC      drivers/clk/clk.o
  CC      net/ipv6/ip6_flowlabel.o
  CC      fs/ntfs/dir.o
  CC      arch/x86/kernel/irq_64.o
  CC [M]  net/netfilter/nf_conntrack_proto_generic.o
  CC      net/ipv6/inet6_connection_sock.o
  CC      drivers/dma/dw/acpi.o
  CC      fs/lockd/mon.o
  CC [M]  fs/netfs/iterator.o
  CC [M]  fs/fscache/cookie.o
  CC [M]  arch/x86/kvm/i8254.o
  CC      drivers/acpi/acpica/evevent.o
  CC      fs/autofs/dev-ioctl.o
  CC      drivers/char/hw_random/core.o
  CC      fs/ext4/inode.o
  CC      drivers/pnp/resource.o
  CC      drivers/virtio/virtio_pci_modern_dev.o
  CC      kernel/time/ntp.o
  CC [M]  sound/hda/hdac_stream.o
  CC [M]  drivers/video/fbdev/core/fb_sys_fops.o
  CC      fs/efivarfs/vars.o
  CC      drivers/pci/rom.o
  CC      arch/x86/kernel/dumpstack_64.o
  CC      net/core/ptp_classifier.o
  CC      drivers/acpi/acpica/evgpe.o
  CC      crypto/authencesn.o
  CC      net/bridge/br_nf_core.o
  CC      drivers/dma/dw/pci.o
  CC      drivers/tty/vt/consolemap.o
  CC      drivers/char/agp/backend.o
  CC      drivers/char/tpm/tpm-chip.o
  AR      drivers/acpi/pmic/built-in.a
  CC      fs/lockd/xdr.o
  CC      drivers/char/agp/generic.o
  CC      drivers/acpi/dptf/int340x_thermal.o
  CC      drivers/char/agp/isoch.o
  CC      lib/zstd/common/error_private.o
  AR      fs/autofs/built-in.a
  CC [M]  fs/netfs/main.o
  CC [M]  fs/smbfs_common/cifs_arc4.o
  CC [M]  fs/cifs/trace.o
  CC [M]  fs/netfs/objects.o
  CC      lib/zstd/common/fse_decompress.o
  CC      block/blk-throttle.o
  CC      drivers/char/hw_random/intel-rng.o
  CC [M]  net/netfilter/nf_conntrack_proto_tcp.o
  CC      mm/mmap_lock.o
  CC      drivers/virtio/virtio_pci_legacy_dev.o
  CC      kernel/time/clocksource.o
  CC [M]  fs/smbfs_common/cifs_md4.o
  CC      fs/ext4/ioctl.o
  CC      drivers/pci/setup-res.o
  CC      drivers/acpi/acpica/evgpeblk.o
  CC      fs/ntfs/file.o
  CC      drivers/acpi/acpica/evgpeinit.o
  CC      fs/ntfs/index.o
  CC      arch/x86/kernel/time.o
  CC      fs/ntfs/inode.o
  CC      fs/nfs/read.o
  LD [M]  drivers/video/fbdev/core/fb.o
  CC      drivers/acpi/tables.o
  AR      drivers/acpi/dptf/built-in.a
  AR      drivers/video/fbdev/core/built-in.a
  CC      drivers/clk/clk-divider.o
  CC      net/ipv4/tcp_ipv4.o
  AR      drivers/video/fbdev/built-in.a
  CC      drivers/char/mem.o
  AR      drivers/video/built-in.a
  AR      drivers/dma/dw/built-in.a
  AR      fs/efivarfs/built-in.a
  CC      drivers/dma/hsu/hsu.o
  CC      arch/x86/kernel/ioport.o
  CC      drivers/pnp/manager.o
  CC      fs/ext4/mballoc.o
  CC      net/ipv6/udp_offload.o
  CC [M]  sound/hda/array.o
  CC      fs/ext4/migrate.o
  CC [M]  fs/fscache/io.o
  CC      drivers/char/tpm/tpm-dev-common.o
  CC      crypto/lzo.o
  CC      crypto/lzo-rle.o
  CC      crypto/lz4.o
  CC      net/core/netprio_cgroup.o
  HOSTCC  drivers/tty/vt/conmakehash
  CC      net/sunrpc/rpcb_clnt.o
  CC      net/bridge/br_multicast.o
  CC      drivers/pnp/support.o
  AR      drivers/char/hw_random/built-in.a
  CC      drivers/char/agp/intel-agp.o
  CC      lib/dump_stack.o
  CC      drivers/acpi/acpica/evgpeutil.o
  CC      drivers/char/tpm/tpm-dev.o
  CC [M]  arch/x86/kvm/ioapic.o
  CC      drivers/char/random.o
  CC      drivers/virtio/virtio_pci_modern.o
  CC      drivers/tty/vt/vt.o
  CC      lib/zstd/common/zstd_common.o
  CC      fs/ntfs/mft.o
  CC      fs/lockd/clnt4xdr.o
  CC      drivers/pci/irq.o
  AR      lib/zstd/built-in.a
  CC      fs/lockd/xdr4.o
  CC      drivers/virtio/virtio_pci_common.o
  LD [M]  fs/netfs/netfs.o
  CC      drivers/char/agp/intel-gtt.o
  CC      drivers/pci/vpd.o
  CC      lib/earlycpio.o
  CC      drivers/pci/setup-bus.o
  CC [M]  sound/hda/hdmi_chmap.o
  COPY    drivers/tty/vt/defkeymap.c
  CC      crypto/lz4hc.o
  CC [M]  sound/hda/trace.o
  CC      kernel/time/jiffies.o
  CC      mm/highmem.o
  CC      arch/x86/kernel/dumpstack.o
  CC      drivers/char/tpm/tpm-interface.o
  CC      drivers/pnp/interface.o
  CC      net/sunrpc/timer.o
  CC      crypto/xxhash_generic.o
  CC      drivers/acpi/acpica/evglock.o
  CC      kernel/time/timer_list.o
  CC      lib/extable.o
  AR      drivers/dma/hsu/built-in.a
  AR      drivers/dma/idxd/built-in.a
  CC      crypto/rng.o
  AR      drivers/dma/mediatek/built-in.a
  AR      drivers/dma/qcom/built-in.a
  CC      drivers/char/tpm/tpm1-cmd.o
  AR      drivers/dma/ti/built-in.a
  AR      drivers/dma/xilinx/built-in.a
  CC [M]  drivers/dma/ioat/init.o
  CC [M]  fs/fscache/main.o
  CC [M]  drivers/dma/ioat/dma.o
  CC      drivers/char/tpm/tpm2-cmd.o
  CC [M]  sound/hda/hdac_component.o
  CC [M]  fs/cifs/cifsfs.o
  CC      mm/memory.o
  CC      drivers/pnp/quirks.o
  CC      crypto/drbg.o
  CC      net/ipv6/seg6.o
  CC      block/mq-deadline.o
  CC [M]  fs/fscache/volume.o
  CC      drivers/acpi/acpica/evhandler.o
  CC      lib/flex_proportions.o
  CC      net/core/dst_cache.o
  CC      fs/ntfs/mst.o
  CC      drivers/tty/hvc/hvc_console.o
  CC      drivers/tty/serial/8250/8250_core.o
  CC [M]  net/netfilter/nf_conntrack_proto_udp.o
  CC      drivers/tty/serial/serial_core.o
  CC      fs/ntfs/namei.o
  CC      arch/x86/kernel/nmi.o
  CC      drivers/virtio/virtio_pci_legacy.o
  CC [M]  arch/x86/kvm/irq_comm.o
  CC      kernel/time/timeconv.o
  CC      fs/nfs/symlink.o
  CC [M]  drivers/virtio/virtio_mem.o
  CC      fs/lockd/svc4proc.o
  CC      mm/mincore.o
  CC      arch/x86/kernel/ldt.o
  CC [M]  net/netfilter/nf_conntrack_proto_icmp.o
  AR      drivers/char/agp/built-in.a
  CC [M]  net/bluetooth/hci_sock.o
  CC [M]  net/bluetooth/hci_sysfs.o
  CC      lib/idr.o
  CC      lib/irq_regs.o
  CC      drivers/acpi/acpica/evmisc.o
  CC [M]  net/bluetooth/l2cap_core.o
  CC      fs/btrfs/extent-tree.o
  CC      arch/x86/kernel/setup.o
  CC      mm/mlock.o
  CC [M]  sound/hda/hdac_i915.o
  CC      lib/is_single_threaded.o
  CC      drivers/clk/clk-fixed-factor.o
  CC      lib/klist.o
  CC      kernel/time/timecounter.o
  CC      drivers/pnp/system.o
  CC      kernel/time/alarmtimer.o
  CC      net/sunrpc/xdr.o
  CC      drivers/char/tpm/tpmrm-dev.o
  CC      fs/ntfs/runlist.o
  AR      drivers/virtio/built-in.a
  CC      kernel/time/posix-timers.o
  CC      mm/mmap.o
  CC [M]  drivers/dma/ioat/prep.o
  CC      arch/x86/kernel/x86_init.o
  CC      drivers/acpi/acpica/evregion.o
  CC      net/ipv6/fib6_notifier.o
  CC [M]  fs/fscache/proc.o
  CC      drivers/pci/vc.o
  CC      net/core/gro_cells.o
  CC      drivers/pci/mmap.o
  AR      drivers/tty/hvc/built-in.a
  CC      lib/kobject.o
  CC      block/kyber-iosched.o
  CC [M]  fs/cifs/cifs_debug.o
  CC      block/bfq-iosched.o
  CC      lib/kobject_uevent.o
  CC      drivers/clk/clk-fixed-rate.o
  AR      drivers/pnp/built-in.a
  CC      drivers/tty/serial/8250/8250_pnp.o
  CC      drivers/clk/clk-gate.o
  CC [M]  sound/hda/intel-dsp-config.o
  CC      lib/logic_pio.o
  CC      drivers/tty/serial/8250/8250_port.o
  CC      drivers/tty/serial/8250/8250_dma.o
  CC      fs/nfs/unlink.o
  CC [M]  arch/x86/kvm/cpuid.o
  CC [M]  net/bluetooth/l2cap_sock.o
  CC      mm/mmu_gather.o
  CC      crypto/jitterentropy.o
  CC      crypto/jitterentropy-kcapi.o
  CC      drivers/char/tpm/tpm2-space.o
  CC [M]  net/bluetooth/smp.o
  CC      fs/lockd/procfs.o
  CC [M]  net/netfilter/nf_conntrack_extend.o
  CC      drivers/acpi/acpica/evrgnini.o
  CC      net/ipv4/tcp_minisocks.o
  CC      arch/x86/kernel/i8259.o
  CC      drivers/char/misc.o
  CC      drivers/pci/setup-irq.o
  CC [M]  fs/fuse/dev.o
  LD [M]  fs/fscache/fscache.o
  CC [M]  fs/overlayfs/super.o
  CC      arch/x86/kernel/irqinit.o
  CC      drivers/clk/clk-multiplier.o
  CC [M]  fs/overlayfs/namei.o
  CC      crypto/ghash-generic.o
  CC      fs/nfs/write.o
  CC      arch/x86/kernel/jump_label.o
  CONMK   drivers/tty/vt/consolemap_deftbl.c
  AR      net/core/built-in.a
  CC      drivers/tty/vt/defkeymap.o
  CC      drivers/pci/proc.o
  CC      fs/ntfs/super.o
  CC      mm/mprotect.o
  CC [M]  net/netfilter/nf_conntrack_acct.o
  CC [M]  sound/hda/intel-nhlt.o
  CC      net/ipv6/rpl.o
  CC [M]  drivers/dma/ioat/dca.o
  CC      drivers/acpi/acpica/evsci.o
  CC      drivers/tty/vt/consolemap_deftbl.o
  CC      drivers/tty/serial/8250/8250_dwlib.o
  CC [M]  net/bluetooth/lib.o
  CC      arch/x86/kernel/irq_work.o
  AR      drivers/tty/vt/built-in.a
  CC      fs/open.o
  AR      drivers/iommu/amd/built-in.a
  AR      drivers/gpu/host1x/built-in.a
  CC      drivers/iommu/intel/dmar.o
  AR      fs/lockd/built-in.a
  AR      drivers/gpu/drm/tests/built-in.a
  CC [M]  drivers/gpu/drm/tests/drm_kunit_helpers.o
  CC      fs/read_write.o
  AR      drivers/iommu/arm/arm-smmu/built-in.a
  AR      drivers/iommu/arm/arm-smmu-v3/built-in.a
  AR      drivers/iommu/arm/built-in.a
  CC      mm/mremap.o
  CC      lib/maple_tree.o
  AR      drivers/iommu/iommufd/built-in.a
  CC      mm/msync.o
  CC      drivers/iommu/intel/iommu.o
  CC      drivers/clk/clk-mux.o
  CC      drivers/iommu/iommu.o
  CC      drivers/char/tpm/tpm-sysfs.o
  CC      crypto/af_alg.o
  CC      fs/ext4/mmp.o
  CC      drivers/acpi/acpica/evxface.o
  CC      kernel/time/posix-cpu-timers.o
  CC [M]  drivers/gpu/drm/tests/drm_buddy_test.o
  CC      drivers/tty/serial/earlycon.o
  CC [M]  sound/hda/intel-sdw-acpi.o
  CC      drivers/iommu/intel/pasid.o
  CC [M]  fs/cifs/connect.o
  CC      net/ipv4/tcp_cong.o
  CC      mm/page_vma_mapped.o
  CC      drivers/pci/slot.o
  CC [M]  drivers/dma/ioat/sysfs.o
  CC      drivers/tty/serial/8250/8250_pcilib.o
  CC      drivers/clk/clk-composite.o
  CC      drivers/clk/clk-fractional-divider.o
  CC      fs/nfs/namespace.o
  CC      net/ipv6/ioam6.o
  CC      arch/x86/kernel/probe_roms.o
  CC [M]  net/netfilter/nf_conntrack_seqadj.o
  CC      fs/ntfs/sysctl.o
  CC [M]  fs/cifs/dir.o
  CC      drivers/acpi/acpica/evxfevnt.o
  CC      drivers/char/virtio_console.o
  CC      net/ipv4/tcp_metrics.o
  LD [M]  sound/hda/snd-hda-core.o
  CC      drivers/char/tpm/eventlog/common.o
  LD [M]  sound/hda/snd-intel-dspcfg.o
  LD [M]  sound/hda/snd-intel-sdw-acpi.o
  CC      fs/nfs/mount_clnt.o
  AR      sound/xen/built-in.a
  CC      net/sunrpc/sunrpc_syms.o
  AR      sound/virtio/built-in.a
  CC      sound/sound_core.o
  CC [M]  fs/overlayfs/util.o
  CC      fs/nfs/nfstrace.o
  CC      fs/btrfs/print-tree.o
  CC      fs/ntfs/unistr.o
  CC [M]  fs/cifs/file.o
  CC      drivers/clk/clk-gpio.o
  CC [M]  arch/x86/kvm/pmu.o
  LD [M]  drivers/dma/ioat/ioatdma.o
  CC      net/bridge/br_mdb.o
  CC      drivers/dma/dmaengine.o
  CC      drivers/acpi/acpica/evxfgpe.o
  CC      drivers/tty/serial/8250/8250_pci.o
  CC      net/bridge/br_multicast_eht.o
  CC      drivers/pci/pci-acpi.o
  CC [M]  fs/cifs/inode.o
  CC      drivers/tty/serial/serial_mctrl_gpio.o
  CC [M]  drivers/gpu/drm/tests/drm_cmdline_parser_test.o
  CC      fs/ntfs/upcase.o
  CC [M]  fs/fuse/dir.o
  CC      sound/last.o
  CC      arch/x86/kernel/sys_ia32.o
  CC [M]  fs/fuse/file.o
  CC [M]  fs/cifs/link.o
  CC      drivers/char/tpm/eventlog/tpm1.o
  CC      kernel/time/posix-clock.o
  CC      drivers/connector/cn_queue.o
  AR      drivers/clk/built-in.a
  CC      drivers/connector/connector.o
  CC      fs/ext4/move_extent.o
  CC      net/bridge/br_vlan.o
  CC      net/bridge/br_vlan_tunnel.o
  CC      fs/nfs/export.o
  CC      crypto/algif_hash.o
  CC      drivers/acpi/acpica/evxfregn.o
  AR      sound/built-in.a
  CC      net/sunrpc/cache.o
  CC      net/ipv4/tcp_fastopen.o
  CC      drivers/base/power/sysfs.o
  CC      net/ipv4/tcp_rate.o
  AR      fs/ntfs/built-in.a
  CC      drivers/base/power/generic_ops.o
  CC      drivers/base/power/common.o
  CC [M]  net/netfilter/nf_conntrack_proto_icmpv6.o
  CC      fs/btrfs/root-tree.o
  CC      drivers/block/loop.o
  CC [M]  fs/overlayfs/inode.o
  CC [M]  drivers/block/nbd.o
  CC      block/bfq-wf2q.o
  CC      drivers/acpi/acpica/exconcat.o
  CC      arch/x86/kernel/signal_32.o
  CC      drivers/char/tpm/eventlog/tpm2.o
  CC      drivers/char/hpet.o
  CC      fs/btrfs/dir-item.o
  CC      kernel/futex/core.o
  CC      drivers/pci/quirks.o
  CC      kernel/time/itimer.o
  CC [M]  arch/x86/kvm/mtrr.o
  CC      kernel/futex/syscalls.o
  AR      drivers/misc/eeprom/built-in.a
  AR      drivers/misc/cb710/built-in.a
  AR      drivers/misc/ti-st/built-in.a
  AR      drivers/misc/lis3lv02d/built-in.a
  CC [M]  drivers/gpu/drm/tests/drm_connector_test.o
  AR      drivers/misc/cardreader/built-in.a
  CC      drivers/base/power/qos.o
  CC [M]  drivers/misc/mei/hdcp/mei_hdcp.o
  CC      mm/pagewalk.o
  CC      kernel/futex/pi.o
  CC      drivers/tty/serial/8250/8250_exar.o
  CC [M]  drivers/misc/mei/pxp/mei_pxp.o
  CC [M]  fs/cifs/misc.o
  CC      net/ipv6/sysctl_net_ipv6.o
  CC      net/ipv4/tcp_recovery.o
  CC      drivers/connector/cn_proc.o
  CC      drivers/dma/virt-dma.o
  CC      drivers/acpi/acpica/exconfig.o
  CC      fs/ext4/namei.o
  CC      crypto/algif_skcipher.o
  CC [M]  fs/fuse/inode.o
  AR      drivers/misc/built-in.a
  CC      drivers/char/tpm/tpm_ppi.o
  CC      drivers/char/tpm/eventlog/acpi.o
  CC [M]  drivers/gpu/drm/tests/drm_damage_helper_test.o
  CC      mm/pgtable-generic.o
  CC      drivers/iommu/intel/trace.o
  AR      drivers/tty/ipwireless/built-in.a
  CC      fs/btrfs/file-item.o
  CC      net/bridge/br_vlan_options.o
  CC      arch/x86/kernel/sys_x86_64.o
  CC      drivers/base/power/runtime.o
  CC      fs/nfs/sysfs.o
  CC [M]  net/netfilter/nf_conntrack_proto_dccp.o
  CC [M]  fs/overlayfs/file.o
  CC      drivers/acpi/acpica/exconvrt.o
  CC [M]  arch/x86/kvm/hyperv.o
  AR      drivers/gpu/drm/arm/built-in.a
  AR      drivers/gpu/drm/display/built-in.a
  CC [M]  fs/cifs/netmisc.o
  CC [M]  drivers/gpu/drm/display/drm_display_helper_mod.o
  CC      block/bfq-cgroup.o
  CC      kernel/futex/requeue.o
  CC [M]  arch/x86/kvm/debugfs.o
  CC      drivers/dma/acpi-dma.o
  CC      net/ipv4/tcp_ulp.o
  CC      kernel/time/clockevents.o
  CC      block/blk-mq-pci.o
  AR      drivers/gpu/drm/rcar-du/built-in.a
  CC      drivers/tty/tty_io.o
  CC [M]  drivers/misc/mei/init.o
  CC      drivers/tty/serial/8250/8250_early.o
  CC      drivers/tty/serial/8250/8250_dw.o
  CC      net/ipv6/xfrm6_policy.o
  CC      drivers/char/nvram.o
  CC      fs/file_table.o
  CC [M]  drivers/gpu/drm/display/drm_dp_dual_mode_helper.o
  CC      drivers/char/tpm/eventlog/efi.o
  CC [M]  net/bluetooth/ecdh_helper.o
  CC      drivers/pci/ats.o
  CC      crypto/xor.o
  CC      drivers/acpi/acpica/excreate.o
  CC      drivers/acpi/acpica/exdebug.o
  AR      drivers/connector/built-in.a
  CC      fs/super.o
  CC      mm/rmap.o
  CC      arch/x86/kernel/espfix_64.o
  CC [M]  drivers/gpu/drm/display/drm_dp_helper.o
  CC      kernel/futex/waitwake.o
  CC      net/ipv6/xfrm6_state.o
  CC      drivers/iommu/intel/cap_audit.o
  CC      block/blk-mq-virtio.o
  CC      kernel/time/tick-common.o
  CC [M]  drivers/gpu/drm/tests/drm_dp_mst_helper_test.o
  CC [M]  drivers/misc/mei/hbm.o
  CC      crypto/hash_info.o
  AR      drivers/dma/built-in.a
  CC [M]  fs/overlayfs/dir.o
  CC      drivers/iommu/intel/irq_remapping.o
  CC      drivers/mfd/mfd-core.o
  CC [M]  fs/cifs/smbencrypt.o
  CC      crypto/simd.o
  CC      drivers/mfd/intel-lpss.o
  CC [M]  fs/cifs/transport.o
  CC      drivers/acpi/acpica/exdump.o
  AR      drivers/nfc/built-in.a
  CC      drivers/iommu/intel/perfmon.o
  CC      drivers/tty/serial/8250/8250_lpss.o
  CC      drivers/char/tpm/tpm_crb.o
  CC [M]  fs/cifs/cached_dir.o
  CC      drivers/base/power/wakeirq.o
  CC [M]  net/bluetooth/hci_request.o
  CC [M]  net/netfilter/nf_conntrack_proto_sctp.o
  CC      net/sunrpc/rpc_pipe.o
  CC [M]  fs/cifs/cifs_unicode.o
  CC      net/sunrpc/sysfs.o
  CC [M]  fs/fuse/control.o
  CC      fs/char_dev.o
  CC      drivers/tty/serial/8250/8250_mid.o
  CC [M]  drivers/gpu/drm/display/drm_dp_mst_topology.o
  AR      kernel/futex/built-in.a
  CC [M]  drivers/misc/mei/interrupt.o
  AR      drivers/block/built-in.a
  CC [M]  drivers/misc/mei/client.o
  CC      kernel/cgroup/cgroup.o
  CC      net/bridge/br_mst.o
  AR      drivers/gpu/vga/built-in.a
  CC      drivers/tty/serial/8250/8250_pericom.o
  CC      net/ipv4/tcp_offload.o
  CC      drivers/pci/iov.o
  CC      arch/x86/kernel/ksysfs.o
  CC      drivers/acpi/acpica/exfield.o
  CC [M]  net/netfilter/nf_conntrack_netlink.o
  CC      block/blk-mq-debugfs.o
  CC [M]  fs/fuse/xattr.o
  CC      net/ipv6/xfrm6_input.o
  CC [M]  crypto/md4.o
  CC      drivers/base/power/main.o
  CC      drivers/mfd/intel-lpss-pci.o
  CC      fs/btrfs/inode-item.o
  CC [M]  drivers/gpu/drm/tests/drm_format_helper_test.o
  CC      fs/ext4/page-io.o
  CC      drivers/mfd/intel-lpss-acpi.o
  CC      kernel/time/tick-broadcast.o
  AR      drivers/dax/hmem/built-in.a
  CC      drivers/dax/super.o
  CC      fs/stat.o
  CC      drivers/acpi/acpica/exfldio.o
  AR      drivers/char/tpm/built-in.a
  CC [M]  drivers/misc/mei/main.o
  AR      drivers/char/built-in.a
  CC [M]  drivers/misc/mei/dma-ring.o
  CC      drivers/tty/n_tty.o
  AR      drivers/tty/serial/8250/built-in.a
  CC [M]  arch/x86/kvm/mmu/mmu.o
  AR      drivers/tty/serial/built-in.a
  CC [M]  fs/overlayfs/readdir.o
  CC      arch/x86/kernel/bootflag.o
  CC      drivers/iommu/iommu-traces.o
  CC      drivers/iommu/iommu-sysfs.o
  CC      drivers/mfd/intel_soc_pmic_crc.o
  CC [M]  net/bluetooth/mgmt_util.o
  CC      fs/nfs/fs_context.o
  CC [M]  crypto/ccm.o
  CC [M]  fs/fuse/acl.o
  CC      drivers/tty/tty_ioctl.o
  CC      drivers/base/firmware_loader/builtin/main.o
  CC [M]  drivers/mfd/lpc_sch.o
  CC      drivers/base/regmap/regmap.o
  AR      drivers/base/test/built-in.a
  CC      fs/exec.o
  CC      net/ipv6/xfrm6_output.o
  AR      drivers/iommu/intel/built-in.a
  CC      drivers/base/regmap/regcache.o
  CC      drivers/tty/tty_ldisc.o
  CC      block/blk-pm.o
  CC      drivers/pci/pci-label.o
  CC      net/ipv6/xfrm6_protocol.o
  CC      drivers/acpi/acpica/exmisc.o
  CC [M]  net/bridge/br_netfilter_hooks.o
  CC      lib/memcat_p.o
  CC      drivers/iommu/dma-iommu.o
  CC      drivers/base/firmware_loader/main.o
  CC      drivers/dax/bus.o
  CC      arch/x86/kernel/e820.o
  AR      drivers/base/firmware_loader/builtin/built-in.a
  CC      kernel/time/tick-broadcast-hrtimer.o
  CC      drivers/base/component.o
  CC [M]  drivers/gpu/drm/display/drm_dsc_helper.o
  CC [M]  drivers/gpu/drm/tests/drm_format_test.o
  CC      drivers/base/power/wakeup.o
  CC [M]  drivers/mfd/lpc_ich.o
  CC      net/ipv4/tcp_plb.o
  CC      net/ipv6/netfilter.o
  CC      fs/btrfs/disk-io.o
  CC      net/sunrpc/svc_xprt.o
  CC      fs/ext4/readpage.o
  CC      net/sunrpc/xprtmultipath.o
  CC [M]  fs/fuse/readdir.o
  CC      lib/nmi_backtrace.o
  CC      drivers/acpi/acpica/exmutex.o
  CC      mm/vmalloc.o
  CC [M]  fs/fuse/ioctl.o
  CC      fs/nfs/sysctl.o
  CC      fs/ext4/resize.o
  CC      kernel/time/tick-oneshot.o
  CC      fs/btrfs/transaction.o
  CC [M]  drivers/misc/mei/bus.o
  CC      block/holder.o
  CC [M]  drivers/misc/mei/bus-fixup.o
  CC [M]  fs/overlayfs/copy_up.o
  CC      drivers/pci/pci-stub.o
  CC [M]  drivers/gpu/drm/tests/drm_framebuffer_test.o
  CC [M]  crypto/arc4.o
  AR      drivers/mfd/built-in.a
  CC [M]  net/bluetooth/mgmt_config.o
  CC      drivers/pci/vgaarb.o
  CC      fs/btrfs/inode.o
  CC      drivers/base/regmap/regcache-rbtree.o
  CC [M]  fs/cifs/nterr.o
  CC      drivers/acpi/acpica/exnames.o
  CC      fs/ext4/super.o
  CC      net/ipv4/datagram.o
  CC      kernel/time/tick-sched.o
  CC      lib/plist.o
  CC      drivers/dma-buf/dma-buf.o
  CC      drivers/dma-buf/dma-fence.o
  CC      lib/radix-tree.o
  CC [M]  drivers/gpu/drm/tests/drm_managed_test.o
  AR      drivers/base/firmware_loader/built-in.a
  CC      drivers/tty/tty_buffer.o
  CC [M]  drivers/gpu/drm/tests/drm_mm_test.o
  CC [M]  drivers/gpu/drm/display/drm_hdcp_helper.o
  CC [M]  crypto/ecc.o
  CC      drivers/dma-buf/dma-fence-array.o
  CC      arch/x86/kernel/pci-dma.o
  AR      block/built-in.a
  CC      drivers/iommu/ioasid.o
  CC      net/ipv6/fib6_rules.o
  CC      drivers/base/core.o
  CC [M]  arch/x86/kvm/mmu/page_track.o
  CC      fs/ext4/symlink.o
  LD [M]  fs/fuse/fuse.o
  AR      drivers/dax/built-in.a
  CC      drivers/dma-buf/dma-fence-chain.o
  CC [M]  fs/cifs/cifsencrypt.o
  CC      drivers/base/power/wakeup_stats.o
  CC      arch/x86/kernel/quirks.o
  CC      drivers/acpi/acpica/exoparg1.o
  CC      drivers/acpi/acpica/exoparg2.o
  CC      drivers/iommu/iova.o
  CC [M]  net/netfilter/nf_nat_core.o
  CC [M]  fs/cifs/readdir.o
  CC      drivers/acpi/acpica/exoparg3.o
  CC      net/sunrpc/stats.o
  CC      drivers/acpi/acpica/exoparg6.o
  CC      fs/nfs/nfs2super.o
  CC      drivers/dma-buf/dma-fence-unwrap.o
  CC [M]  net/netfilter/nf_nat_proto.o
  CC [M]  drivers/misc/mei/debugfs.o
  CC      drivers/dma-buf/dma-resv.o
  AR      drivers/cxl/core/built-in.a
  AR      drivers/cxl/built-in.a
  CC      fs/pipe.o
  CC [M]  fs/overlayfs/export.o
  CC [M]  net/bluetooth/hci_codec.o
  CC      drivers/tty/tty_port.o
  CC      drivers/iommu/irq_remapping.o
  CC      drivers/base/power/domain.o
  CC [M]  drivers/misc/mei/mei-trace.o
  CC [M]  drivers/gpu/drm/tests/drm_modes_test.o
  CC [M]  drivers/gpu/drm/tests/drm_plane_helper_test.o
  AR      drivers/pci/built-in.a
  CC      drivers/acpi/acpica/exprep.o
  CC      drivers/base/power/domain_governor.o
  CC [M]  net/netfilter/nf_nat_helper.o
  CC [M]  drivers/gpu/drm/display/drm_hdmi_helper.o
  CC [M]  fs/cifs/ioctl.o
  CC [M]  arch/x86/kvm/mmu/spte.o
  CC      kernel/time/vsyscall.o
  CC [M]  arch/x86/kvm/mmu/tdp_iter.o
  CC      net/ipv4/raw.o
  CC [M]  net/bridge/br_netfilter_ipv6.o
  CC      drivers/acpi/acpica/exregion.o
  CC      net/ipv4/udp.o
  CC      drivers/tty/tty_mutex.o
  CC      lib/ratelimit.o
  CC      arch/x86/kernel/topology.o
  CC [M]  drivers/misc/mei/pci-me.o
  CC      drivers/acpi/acpica/exresnte.o
  CC      drivers/dma-buf/sync_file.o
  CC      drivers/acpi/acpica/exresolv.o
  CC      lib/rbtree.o
  CC [M]  drivers/gpu/drm/display/drm_scdc_helper.o
  CC      kernel/time/timekeeping_debug.o
  CC [M]  net/netfilter/nf_nat_redirect.o
  CC      fs/nfs/proc.o
  CC      net/ipv6/proc.o
  AR      drivers/iommu/built-in.a
  CC [M]  drivers/misc/mei/hw-me.o
  LD [M]  fs/overlayfs/overlay.o
  CC [M]  net/netfilter/nf_nat_masquerade.o
  CC      drivers/acpi/acpica/exresop.o
  CC      drivers/base/power/clock_ops.o
  CC [M]  drivers/gpu/drm/tests/drm_probe_helper_test.o
  CC      drivers/tty/tty_ldsem.o
  CC      fs/nfs/nfs2xdr.o
  CC      fs/ext4/sysfs.o
  AR      drivers/macintosh/built-in.a
  CC      arch/x86/kernel/kdebugfs.o
  CC      arch/x86/kernel/alternative.o
  CC      drivers/scsi/scsi.o
  CC      drivers/base/regmap/regcache-flat.o
  CC      arch/x86/kernel/i8253.o
  CC      arch/x86/kernel/hw_breakpoint.o
  CC [M]  net/bluetooth/eir.o
  CC      drivers/scsi/hosts.o
  CC [M]  drivers/gpu/drm/display/drm_dp_aux_dev.o
  CC      drivers/dma-buf/sw_sync.o
  CC      lib/seq_buf.o
  CC      lib/show_mem.o
  CC [M]  net/netfilter/x_tables.o
  CC [M]  fs/cifs/sess.o
  CC [M]  crypto/essiv.o
  CC [M]  fs/cifs/export.o
  CC      drivers/scsi/scsi_ioctl.o
  CC      arch/x86/kernel/tsc.o
  CC      drivers/acpi/acpica/exserial.o
  CC      net/sunrpc/sysctl.o
  CC      kernel/time/namespace.o
  CC      fs/namei.o
  CC      drivers/acpi/blacklist.o
  CC [M]  net/netfilter/xt_tcpudp.o
  CC [M]  net/netfilter/xt_mark.o
  CC [M]  drivers/gpu/drm/tests/drm_rect_test.o
  CC [M]  net/netfilter/xt_nat.o
  CC [M]  net/netfilter/xt_REDIRECT.o
  CC      drivers/base/regmap/regmap-debugfs.o
  CC [M]  drivers/misc/mei/gsc-me.o
  CC      fs/btrfs/file.o
  AR      net/bridge/built-in.a
  LD [M]  net/bridge/br_netfilter.o
  CC      drivers/tty/tty_baudrate.o
  CC      fs/btrfs/defrag.o
  CC [M]  fs/cifs/unc.o
  CC      lib/siphash.o
  CC      arch/x86/kernel/tsc_msr.o
  CC [M]  crypto/ecdh.o
  AR      drivers/base/power/built-in.a
  CC      drivers/acpi/acpica/exstore.o
  CC [M]  crypto/ecdh_helper.o
  CC [M]  fs/cifs/winucase.o
  CC      drivers/acpi/acpica/exstoren.o
  CC [M]  fs/cifs/smb2ops.o
  CC      fs/ext4/xattr.o
  CC [M]  net/bluetooth/hci_sync.o
  CC      net/ipv6/syncookies.o
  CC      kernel/cgroup/rstat.o
  CC      mm/page_alloc.o
  CC      kernel/trace/trace_clock.o
  CC      drivers/tty/tty_jobctrl.o
  CC      drivers/dma-buf/sync_debug.o
  CC      kernel/trace/ftrace.o
  AR      net/sunrpc/built-in.a
  LD [M]  drivers/gpu/drm/display/drm_display_helper.o
  CC      drivers/tty/n_null.o
  CC [M]  drivers/dma-buf/selftest.o
  AR      kernel/time/built-in.a
  CC [M]  fs/cifs/smb2maperror.o
  LD [M]  drivers/misc/mei/mei.o
  CC      drivers/acpi/acpica/exstorob.o
  CC      drivers/acpi/osi.o
  AR      drivers/gpu/drm/omapdrm/built-in.a
  CC      lib/string.o
  AR      drivers/gpu/drm/tilcdc/built-in.a
  CC [M]  fs/cifs/smb2transport.o
  AR      drivers/gpu/drm/imx/built-in.a
  CC      drivers/acpi/acpica/exsystem.o
  AR      drivers/gpu/drm/i2c/built-in.a
  AR      drivers/gpu/drm/panel/built-in.a
  CC      kernel/cgroup/namespace.o
  CC      drivers/acpi/osl.o
  CC      lib/timerqueue.o
  AR      drivers/gpu/drm/bridge/analogix/built-in.a
  CC      lib/vsprintf.o
  AR      drivers/gpu/drm/bridge/cadence/built-in.a
  AR      drivers/gpu/drm/bridge/imx/built-in.a
  CC      drivers/scsi/scsicam.o
  AR      drivers/gpu/drm/bridge/synopsys/built-in.a
  CC      drivers/acpi/acpica/extrace.o
  CC      drivers/acpi/utils.o
  AR      drivers/gpu/drm/bridge/built-in.a
  CC [M]  drivers/dma-buf/st-dma-fence.o
  AR      drivers/gpu/drm/hisilicon/built-in.a
  AR      drivers/gpu/drm/mxsfb/built-in.a
  CC [M]  drivers/dma-buf/st-dma-fence-chain.o
  CC      lib/win_minmax.o
  AR      drivers/gpu/drm/tiny/built-in.a
  AR      drivers/gpu/drm/xlnx/built-in.a
  CC      arch/x86/kernel/io_delay.o
  CC      drivers/base/regmap/regmap-i2c.o
  AR      drivers/gpu/drm/gud/built-in.a
  AR      drivers/gpu/drm/solomon/built-in.a
  LD [M]  crypto/ecdh_generic.o
  CC      mm/init-mm.o
  CC [M]  drivers/gpu/drm/ttm/ttm_tt.o
  AR      crypto/built-in.a
  CC      mm/memblock.o
  CC [M]  fs/cifs/smb2misc.o
  LD [M]  drivers/misc/mei/mei-gsc.o
  LD [M]  drivers/misc/mei/mei-me.o
  CC [M]  net/netfilter/xt_MASQUERADE.o
  CC      drivers/base/regmap/regmap-irq.o
  CC      kernel/bpf/core.o
  CC      fs/nfs/nfs3super.o
  CC [M]  net/netfilter/xt_addrtype.o
  CC [M]  drivers/dma-buf/st-dma-fence-unwrap.o
  CC      kernel/events/core.o
  CC      drivers/nvme/host/core.o
  CC      drivers/nvme/host/ioctl.o
  CC      drivers/acpi/acpica/exutils.o
  CC      drivers/ata/libata-core.o
  CC      drivers/ata/libata-scsi.o
  CC      drivers/nvme/host/trace.o
  CC      arch/x86/kernel/rtc.o
  CC      drivers/nvme/host/pci.o
  CC      net/ipv6/mip6.o
  CC      drivers/tty/pty.o
  CC [M]  drivers/dma-buf/st-dma-resv.o
  CC      fs/nfs/nfs3client.o
  CC      kernel/cgroup/cgroup-v1.o
  CC      drivers/acpi/acpica/hwacpi.o
  CC      drivers/scsi/scsi_error.o
  AR      drivers/dma-buf/built-in.a
  CC      drivers/scsi/scsi_lib.o
  CC      arch/x86/kernel/resource.o
  CC      fs/fcntl.o
  CC      fs/ext4/xattr_hurd.o
  CC      lib/xarray.o
  CC      net/ipv6/addrconf_core.o
  CC [M]  drivers/gpu/drm/ttm/ttm_bo.o
  AS      arch/x86/kernel/irqflags.o
  CC      drivers/acpi/acpica/hwesleep.o
  CC [M]  drivers/gpu/drm/ttm/ttm_bo_util.o
  CC      fs/btrfs/extent_map.o
  CC      fs/btrfs/sysfs.o
  LD [M]  drivers/dma-buf/dmabuf_selftests.o
  CC      arch/x86/kernel/static_call.o
  CC [M]  fs/cifs/smb2pdu.o
  CC      kernel/trace/ring_buffer.o
  CC      net/ipv6/exthdrs_core.o
  CC      net/ipv6/ip6_checksum.o
  CC [M]  fs/cifs/smb2inode.o
  CC [M]  net/netfilter/xt_conntrack.o
  CC [M]  drivers/gpu/drm/ttm/ttm_bo_vm.o
  CC [M]  drivers/gpu/drm/ttm/ttm_module.o
  CC      drivers/tty/sysrq.o
  CC      mm/memory_hotplug.o
  CC [M]  drivers/gpu/drm/scheduler/sched_main.o
  CC      drivers/acpi/acpica/hwgpe.o
  CC [M]  drivers/gpu/drm/scheduler/sched_fence.o
  CC      net/ipv4/udplite.o
  CC      fs/nfs/nfs3proc.o
  AR      drivers/base/regmap/built-in.a
  CC      drivers/base/bus.o
  CC [M]  drivers/gpu/drm/scheduler/sched_entity.o
  CC      arch/x86/kernel/process.o
  CC      drivers/spi/spi.o
  CC      arch/x86/kernel/ptrace.o
  CC [M]  fs/cifs/smb2file.o
  CC [M]  net/netfilter/xt_ipvs.o
  CC [M]  drivers/gpu/drm/ttm/ttm_execbuf_util.o
  CC      net/ipv4/udp_offload.o
  CC      kernel/cgroup/freezer.o
  CC      mm/madvise.o
  CC      drivers/net/phy/mdio-boardinfo.o
  AR      drivers/firewire/built-in.a
  CC      fs/ioctl.o
  CC      drivers/acpi/acpica/hwregs.o
  CC [M]  drivers/gpu/drm/ttm/ttm_range_manager.o
  AR      drivers/cdrom/built-in.a
  CC      drivers/base/dd.o
  CC      fs/ext4/xattr_trusted.o
  CC      net/ipv6/ip6_icmp.o
  CC      mm/page_io.o
  CC      drivers/scsi/scsi_lib_dma.o
  CC [M]  arch/x86/kvm/mmu/tdp_mmu.o
  CC      drivers/base/syscore.o
  CC      drivers/net/phy/mdio_devres.o
  CC      kernel/cgroup/legacy_freezer.o
  AR      drivers/net/pse-pd/built-in.a
  CC      fs/ext4/xattr_user.o
  AR      drivers/tty/built-in.a
  CC      drivers/scsi/scsi_scan.o
  CC      arch/x86/kernel/tls.o
  CC      lib/lockref.o
  CC      drivers/acpi/acpica/hwsleep.o
  CC      drivers/net/phy/phy.o
  CC      fs/ext4/fast_commit.o
  CC      fs/btrfs/accessors.o
  CC      fs/btrfs/xattr.o
  CC      fs/ext4/orphan.o
  CC      mm/swap_state.o
  CC      lib/bcd.o
  AR      drivers/auxdisplay/built-in.a
  CC [M]  drivers/gpu/drm/ttm/ttm_resource.o
  CC      drivers/usb/common/common.o
  CC      lib/sort.o
  CC      drivers/usb/common/debug.o
  CC      drivers/usb/core/usb.o
  LD [M]  drivers/gpu/drm/scheduler/gpu-sched.o
  CC      drivers/base/driver.o
  CC      lib/parser.o
  CC      kernel/trace/trace.o
  AR      kernel/bpf/built-in.a
  CC      kernel/cgroup/pids.o
  CC      drivers/net/phy/phy-c45.o
  CC      kernel/fork.o
  CC      fs/nfs/nfs3xdr.o
  GEN     drivers/scsi/scsi_devinfo_tbl.c
  CC      mm/swapfile.o
  LD [M]  net/netfilter/nf_conntrack.o
  CC      fs/btrfs/ordered-data.o
  CC      arch/x86/kernel/step.o
  CC      drivers/acpi/acpica/hwvalid.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_drv.o
  LD [M]  net/netfilter/nf_nat.o
  AR      net/netfilter/built-in.a
  CC      drivers/ata/libata-eh.o
  CC      drivers/usb/core/hub.o
  CC      kernel/exec_domain.o
  CC      arch/x86/kernel/i8237.o
  CC      kernel/panic.o
  CC [M]  fs/cifs/cifsacl.o
  CC      kernel/cpu.o
  CC      net/ipv6/output_core.o
  CC      mm/swap_slots.o
  CC      net/ipv4/arp.o
  CC      drivers/net/mdio/acpi_mdio.o
  CC      lib/debug_locks.o
  CC      drivers/net/mdio/fwnode_mdio.o
  CC      fs/readdir.o
  CC      drivers/base/class.o
  CC      kernel/trace/trace_output.o
  CC [M]  net/bluetooth/sco.o
  CC      drivers/acpi/acpica/hwxface.o
  AR      drivers/usb/common/built-in.a
  CC      lib/random32.o
  CC      drivers/ata/libata-transport.o
  AR      drivers/net/pcs/built-in.a
  CC      arch/x86/kernel/stacktrace.o
  CC      kernel/cgroup/cpuset.o
  CC      kernel/trace/trace_seq.o
  CC      arch/x86/kernel/reboot.o
  CC [M]  drivers/gpu/drm/ttm/ttm_pool.o
  CC      drivers/scsi/scsi_devinfo.o
  CC      drivers/scsi/scsi_sysctl.o
  CC      drivers/scsi/scsi_debugfs.o
  CC      kernel/trace/trace_stat.o
  CC      drivers/usb/core/hcd.o
  AR      drivers/nvme/host/built-in.a
  CC      lib/bust_spinlocks.o
  AR      drivers/nvme/target/built-in.a
  AR      drivers/usb/phy/built-in.a
  AR      drivers/nvme/built-in.a
  CC      drivers/usb/storage/scsiglue.o
  CC      drivers/usb/host/pci-quirks.o
  CC      drivers/usb/host/ehci-hcd.o
  CC      drivers/input/serio/serio.o
  CC      drivers/acpi/acpica/hwxfsleep.o
  CC      drivers/input/serio/i8042.o
  CC      drivers/usb/host/ehci-pci.o
  CC      drivers/usb/storage/protocol.o
  CC      drivers/input/serio/libps2.o
  CC      drivers/net/phy/phy-core.o
  AR      drivers/net/mdio/built-in.a
  CC      drivers/usb/host/ohci-hcd.o
  CC      lib/kasprintf.o
  CC      kernel/trace/trace_printk.o
  CC      drivers/base/platform.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_device.o
  CC      net/ipv6/protocol.o
  CC      net/ipv6/ip6_offload.o
  CC [M]  drivers/gpu/drm/i915/i915_driver.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_kms.o
  CC      net/ipv6/tcpv6_offload.o
  CC      drivers/acpi/acpica/hwpci.o
  CC      lib/bitmap.o
  CC      net/ipv6/exthdrs_offload.o
  CC      drivers/base/cpu.o
  CC      drivers/scsi/scsi_trace.o
  CC [M]  arch/x86/kvm/smm.o
  CC      drivers/input/keyboard/atkbd.o
  CC [M]  drivers/gpu/drm/ttm/ttm_device.o
  CC      arch/x86/kernel/msr.o
  CC      drivers/net/phy/phy_device.o
  CC [M]  drivers/gpu/drm/vgem/vgem_drv.o
  CC      fs/btrfs/extent_io.o
  CC [M]  drivers/gpu/drm/vgem/vgem_fence.o
  CC      drivers/usb/storage/transport.o
  CC      lib/scatterlist.o
  AR      fs/nfs/built-in.a
  CC      drivers/input/input.o
  AR      drivers/input/mouse/built-in.a
  CC      mm/dmapool.o
  CC      drivers/usb/serial/usb-serial.o
  CC      drivers/input/input-compat.o
  CC      drivers/acpi/acpica/nsaccess.o
  AR      drivers/spi/built-in.a
  CC      drivers/ata/libata-trace.o
  CC      kernel/trace/pid_list.o
  CC [M]  drivers/gpu/drm/xe/tests/xe_bo_test.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/object.o
  CC      net/ipv4/icmp.o
  CC [M]  drivers/gpu/drm/xe/tests/xe_dma_buf_test.o
  CC [M]  net/bluetooth/iso.o
  CC      kernel/exit.o
  CC      drivers/acpi/reboot.o
  CC      drivers/ata/libata-sata.o
  CC      drivers/scsi/scsi_logging.o
  CC      drivers/acpi/acpica/nsalloc.o
  CC      drivers/base/firmware.o
  CC      arch/x86/kernel/cpuid.o
  AR      drivers/input/serio/built-in.a
  CC      drivers/acpi/acpica/nsarguments.o
  CC [M]  drivers/gpu/drm/ttm/ttm_sys_manager.o
  CC      drivers/input/input-mt.o
  CC      drivers/usb/host/ohci-pci.o
  CC      drivers/input/input-poller.o
  CC      drivers/acpi/nvs.o
  CC      drivers/usb/host/uhci-hcd.o
  LD [M]  drivers/gpu/drm/vgem/vgem.o
  CC [M]  drivers/gpu/drm/xe/tests/xe_migrate_test.o
  CC      fs/btrfs/volumes.o
  CC      drivers/rtc/lib.o
  CC      kernel/softirq.o
  CC [M]  drivers/gpu/drm/i915/i915_drm_client.o
  AR      drivers/i2c/algos/built-in.a
  CC      net/ipv6/inet6_hashtables.o
  CC [M]  fs/cifs/fs_context.o
  CC [M]  drivers/i2c/algos/i2c-algo-bit.o
  CC      mm/hugetlb.o
  CC      drivers/base/init.o
  AR      drivers/input/keyboard/built-in.a
  CC      fs/btrfs/async-thread.o
  CC      net/ipv6/mcast_snoop.o
  CC [M]  fs/cifs/dns_resolve.o
  CC [M]  arch/x86/kvm/vmx/vmx.o
  CC      drivers/acpi/acpica/nsconvert.o
  CC      lib/list_sort.o
  CC      lib/uuid.o
  CC [M]  net/ipv6/ip6_udp_tunnel.o
  CC      drivers/usb/storage/usb.o
  CC [M]  drivers/gpu/drm/ttm/ttm_agp_backend.o
  CC [M]  drivers/gpu/drm/i915/i915_config.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/client.o
  CC      arch/x86/kernel/early-quirks.o
  CC [M]  arch/x86/kvm/kvm-asm-offsets.s
  CC [M]  drivers/gpu/drm/xe/tests/xe_pci_test.o
  CC [M]  arch/x86/kvm/vmx/pmu_intel.o
  CC      fs/btrfs/ioctl.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/conn.o
  CC      lib/iov_iter.o
  AR      fs/ext4/built-in.a
  CC      arch/x86/kernel/smp.o
  CC      arch/x86/kernel/smpboot.o
  CC      drivers/scsi/scsi_pm.o
  CC      drivers/rtc/class.o
  CC      drivers/input/ff-core.o
  CC      drivers/acpi/wakeup.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.o
  CC      drivers/input/touchscreen.o
  CC      drivers/usb/serial/generic.o
  AR      kernel/cgroup/built-in.a
  CC      kernel/resource.o
  CC      drivers/acpi/acpica/nsdump.o
  CC      drivers/base/map.o
  CC      drivers/ata/libata-sff.o
  LD [M]  drivers/gpu/drm/ttm/ttm.o
  CC      drivers/usb/host/xhci.o
  CC      drivers/usb/core/urb.o
  CC      drivers/i2c/busses/i2c-designware-common.o
  CC [M]  drivers/gpu/drm/ast/ast_drv.o
  CC [M]  drivers/gpu/drm/i915/i915_getparam.o
  CC [M]  drivers/gpu/drm/ast/ast_i2c.o
  CC [M]  drivers/gpu/drm/xe/tests/xe_rtp_test.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/device.o
  CC      drivers/acpi/acpica/nseval.o
  CC      drivers/net/phy/linkmode.o
  CC      drivers/usb/core/message.o
  CC [M]  drivers/gpu/drm/ast/ast_main.o
  CC      drivers/scsi/scsi_bsg.o
  CC [M]  drivers/gpu/drm/xe/xe_bb.o
  AR      drivers/i3c/built-in.a
  CC      drivers/usb/core/driver.o
  CC      drivers/input/ff-memless.o
  CC      drivers/rtc/interface.o
  CC      drivers/usb/storage/initializers.o
  AR      drivers/media/i2c/built-in.a
  CC      drivers/base/devres.o
  AR      drivers/media/tuners/built-in.a
  AR      drivers/media/rc/keymaps/built-in.a
  CC      drivers/usb/storage/sierra_ms.o
  AR      drivers/media/rc/built-in.a
  CC [M]  drivers/gpu/drm/xe/tests/xe_wa_test.o
  CC      drivers/scsi/scsi_common.o
  AR      drivers/media/common/b2c2/built-in.a
  AR      drivers/media/common/saa7146/built-in.a
  AR      drivers/media/common/siano/built-in.a
  AR      drivers/media/common/v4l2-tpg/built-in.a
  AR      drivers/media/common/videobuf2/built-in.a
  CC      drivers/scsi/sd.o
  AR      drivers/media/common/built-in.a
  CC [M]  net/bluetooth/a2mp.o
  CC      drivers/acpi/sleep.o
  CC      kernel/trace/trace_sched_switch.o
  AR      drivers/media/platform/allegro-dvt/built-in.a
  AR      net/ipv6/built-in.a
  CC      kernel/trace/trace_functions.o
  AR      drivers/media/platform/amlogic/meson-ge2d/built-in.a
  AR      drivers/media/platform/amlogic/built-in.a
  CC      net/ipv4/devinet.o
  AR      drivers/media/platform/amphion/built-in.a
  CC      arch/x86/kernel/tsc_sync.o
  AR      drivers/media/platform/aspeed/built-in.a
  AR      drivers/media/platform/atmel/built-in.a
  AR      drivers/media/platform/cadence/built-in.a
  AR      drivers/media/platform/chips-media/built-in.a
  AR      drivers/media/platform/intel/built-in.a
  AR      drivers/media/platform/marvell/built-in.a
  AR      drivers/media/platform/mediatek/jpeg/built-in.a
  AR      drivers/media/platform/mediatek/mdp/built-in.a
  AR      drivers/media/platform/mediatek/vcodec/built-in.a
  AR      drivers/media/platform/mediatek/vpu/built-in.a
  CC      drivers/acpi/acpica/nsinit.o
  CC      drivers/usb/serial/bus.o
  AR      drivers/media/platform/mediatek/mdp3/built-in.a
  AR      drivers/media/platform/mediatek/built-in.a
  AR      drivers/media/platform/microchip/built-in.a
  AR      drivers/media/platform/nvidia/tegra-vde/built-in.a
  AR      drivers/media/platform/nvidia/built-in.a
  AR      drivers/media/platform/nxp/dw100/built-in.a
  AR      drivers/media/platform/nxp/imx-jpeg/built-in.a
  AR      drivers/media/platform/nxp/built-in.a
  CC      drivers/usb/serial/console.o
  AR      drivers/media/platform/qcom/camss/built-in.a
  AR      drivers/media/platform/qcom/venus/built-in.a
  AR      drivers/media/platform/qcom/built-in.a
  AR      drivers/media/platform/renesas/rcar-vin/built-in.a
  AR      drivers/media/platform/renesas/rzg2l-cru/built-in.a
  AR      drivers/media/platform/renesas/vsp1/built-in.a
  AR      drivers/media/platform/renesas/built-in.a
  CC [M]  arch/x86/kvm/vmx/vmcs12.o
  AR      drivers/media/platform/rockchip/rga/built-in.a
  AR      drivers/media/platform/samsung/exynos-gsc/built-in.a
  AR      drivers/media/platform/rockchip/rkisp1/built-in.a
  AR      drivers/media/platform/rockchip/built-in.a
  AR      drivers/media/platform/samsung/exynos4-is/built-in.a
  CC      drivers/usb/serial/ftdi_sio.o
  CC      drivers/net/phy/mdio_bus.o
  AR      drivers/media/platform/samsung/s3c-camif/built-in.a
  CC      drivers/scsi/sg.o
  AR      drivers/media/platform/samsung/s5p-g2d/built-in.a
  AR      drivers/media/platform/samsung/s5p-jpeg/built-in.a
  CC      drivers/net/phy/mdio_device.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/disp.o
  AR      drivers/media/platform/samsung/s5p-mfc/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvif/driver.o
  AR      drivers/media/platform/samsung/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvif/event.o
  CC [M]  arch/x86/kvm/vmx/hyperv.o
  AR      drivers/media/platform/st/sti/bdisp/built-in.a
  AR      drivers/media/platform/st/sti/c8sectpfe/built-in.a
  CC      drivers/i2c/busses/i2c-designware-master.o
  CC [M]  drivers/gpu/drm/xe/xe_bo.o
  AR      drivers/media/platform/st/sti/delta/built-in.a
  AR      drivers/media/platform/st/sti/hva/built-in.a
  CC [M]  drivers/gpu/drm/i915/i915_ioctl.o
  AR      drivers/media/platform/st/stm32/built-in.a
  CC      arch/x86/kernel/setup_percpu.o
  CC      drivers/rtc/nvmem.o
  CC [M]  drivers/gpu/drm/xe/xe_bo_evict.o
  AR      drivers/media/platform/st/built-in.a
  CC      drivers/usb/storage/option_ms.o
  CC      drivers/base/attribute_container.o
  AR      drivers/media/platform/sunxi/sun4i-csi/built-in.a
  AR      drivers/media/platform/sunxi/sun6i-csi/built-in.a
  AR      drivers/media/platform/sunxi/sun6i-mipi-csi2/built-in.a
  AR      drivers/media/platform/sunxi/sun8i-a83t-mipi-csi2/built-in.a
  AR      drivers/media/platform/sunxi/sun8i-di/built-in.a
  ASN.1   fs/cifs/cifs_spnego_negtokeninit.asn1.[ch]
  CC [M]  fs/cifs/smb1ops.o
  CC      kernel/events/ring_buffer.o
  CC      drivers/i2c/busses/i2c-designware-platdrv.o
  AR      drivers/media/platform/sunxi/sun8i-rotate/built-in.a
  AR      drivers/media/platform/sunxi/built-in.a
  CC      drivers/input/vivaldi-fmap.o
  CC      drivers/acpi/acpica/nsload.o
  AR      drivers/media/platform/ti/am437x/built-in.a
  AR      drivers/media/platform/ti/cal/built-in.a
  CC [M]  drivers/gpu/drm/ast/ast_mm.o
  AR      drivers/media/platform/ti/vpe/built-in.a
  AR      drivers/media/platform/ti/davinci/built-in.a
  CC      drivers/usb/storage/usual-tables.o
  CC [M]  net/bluetooth/amp.o
  AR      drivers/media/platform/ti/omap/built-in.a
  AR      drivers/media/platform/ti/omap3isp/built-in.a
  AR      drivers/media/platform/ti/built-in.a
  AR      drivers/media/platform/verisilicon/built-in.a
  AR      drivers/media/platform/via/built-in.a
  AR      drivers/media/platform/xilinx/built-in.a
  AR      drivers/media/platform/built-in.a
  CC [M]  drivers/gpu/drm/amd/amdgpu/atombios_crtc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_connectors.o
  AR      drivers/media/pci/ttpci/built-in.a
  AR      drivers/media/pci/b2c2/built-in.a
  CC      drivers/usb/host/xhci-mem.o
  CC      drivers/usb/host/xhci-ext-caps.o
  AR      drivers/media/pci/pluto2/built-in.a
  AR      drivers/media/pci/dm1105/built-in.a
  AR      drivers/media/pci/pt1/built-in.a
  AR      drivers/media/pci/pt3/built-in.a
  AR      drivers/media/pci/mantis/built-in.a
  AR      drivers/media/pci/ngene/built-in.a
  AR      drivers/media/pci/ddbridge/built-in.a
  AR      drivers/media/pci/saa7146/built-in.a
  CC      kernel/trace/trace_preemptirq.o
  AR      drivers/media/pci/smipcie/built-in.a
  AR      drivers/media/pci/netup_unidvb/built-in.a
  CC      arch/x86/kernel/ftrace.o
  AR      drivers/media/pci/intel/ipu3/built-in.a
  AR      drivers/media/pci/intel/built-in.a
  AR      drivers/media/pci/built-in.a
  CC      drivers/ata/libata-pmp.o
  CC      drivers/input/input-leds.o
  AR      drivers/media/usb/b2c2/built-in.a
  AR      drivers/media/usb/dvb-usb/built-in.a
  CC      drivers/acpi/device_sysfs.o
  AR      drivers/media/usb/dvb-usb-v2/built-in.a
  CC      drivers/acpi/acpica/nsnames.o
  AR      drivers/media/usb/s2255/built-in.a
  AR      drivers/media/usb/siano/built-in.a
  CC      drivers/acpi/device_pm.o
  AR      drivers/media/usb/ttusb-budget/built-in.a
  AR      drivers/media/usb/ttusb-dec/built-in.a
  AR      drivers/media/usb/built-in.a
  AR      drivers/media/firewire/built-in.a
  AR      drivers/media/mmc/siano/built-in.a
  AR      drivers/media/mmc/built-in.a
  CC      drivers/usb/host/xhci-ring.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/fifo.o
  CC      drivers/base/transport_class.o
  AR      drivers/media/spi/built-in.a
  AR      drivers/net/ethernet/adi/built-in.a
  AR      drivers/media/test-drivers/built-in.a
  AR      drivers/net/ethernet/alacritech/built-in.a
  AR      drivers/media/built-in.a
  AR      drivers/net/ethernet/amazon/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvif/head.o
  AR      drivers/net/ethernet/aquantia/built-in.a
  CC      drivers/base/topology.o
  AR      drivers/net/ethernet/asix/built-in.a
  CC      fs/select.o
  AR      drivers/net/ethernet/cadence/built-in.a
  AR      drivers/ptp/built-in.a
  CC      drivers/usb/core/config.o
  CC      kernel/sysctl.o
  CC [M]  drivers/ptp/ptp_clock.o
  AR      drivers/net/ethernet/broadcom/built-in.a
  CC [M]  drivers/gpu/drm/amd/amdgpu/atom.o
  CC [M]  drivers/net/ethernet/broadcom/b44.o
  AR      drivers/usb/storage/built-in.a
  CC [M]  drivers/ptp/ptp_chardev.o
  CC      drivers/acpi/proc.o
  CC      drivers/rtc/dev.o
  CC [M]  drivers/gpu/drm/i915/i915_irq.o
  CC      drivers/usb/host/xhci-hub.o
  CC      kernel/capability.o
  CC [M]  drivers/gpu/drm/ast/ast_mode.o
  CC      kernel/trace/trace_nop.o
  CC      drivers/i2c/busses/i2c-designware-baytrail.o
  AR      drivers/usb/misc/built-in.a
  CC      drivers/acpi/acpica/nsobject.o
  CC [M]  drivers/usb/misc/ftdi-elan.o
  CC      lib/clz_ctz.o
  CC      drivers/ata/libata-acpi.o
  CC [M]  drivers/i2c/busses/i2c-scmi.o
  CC [M]  arch/x86/kvm/vmx/nested.o
  CC      drivers/input/mousedev.o
  CC      lib/bsearch.o
  CC      kernel/events/callchain.o
  CC      drivers/net/phy/swphy.o
  CC      drivers/usb/serial/pl2303.o
  CC      drivers/acpi/bus.o
  CC      drivers/rtc/proc.o
  AS      arch/x86/kernel/ftrace_64.o
  CC      arch/x86/kernel/trace_clock.o
  CC      drivers/base/container.o
  CC      arch/x86/kernel/trace.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/mem.o
  CC [M]  net/bluetooth/hci_debugfs.o
  CC [M]  drivers/i2c/busses/i2c-ccgx-ucsi.o
  CC [M]  drivers/i2c/busses/i2c-i801.o
  CC [M]  drivers/i2c/busses/i2c-isch.o
  CC      drivers/base/property.o
  CC      drivers/acpi/acpica/nsparse.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/mmu.o
  CC      kernel/trace/trace_functions_graph.o
  CC      fs/btrfs/locking.o
  CC      drivers/rtc/sysfs.o
  CC [M]  drivers/ptp/ptp_sysfs.o
  CC [M]  drivers/i2c/busses/i2c-ismt.o
  CC      drivers/rtc/rtc-mc146818-lib.o
  CC      drivers/base/cacheinfo.o
  CC      drivers/base/swnode.o
  CC      lib/find_bit.o
  CC [M]  fs/cifs/cifssmb.o
  CC      kernel/trace/fgraph.o
  CC      drivers/rtc/rtc-cmos.o
  CC      drivers/scsi/scsi_sysfs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_fence.o
  CC      drivers/net/phy/fixed_phy.o
  CC      kernel/events/hw_breakpoint.o
  CC      drivers/usb/core/file.o
  CC      net/ipv4/af_inet.o
  CC      arch/x86/kernel/rethook.o
  CC [M]  drivers/gpu/drm/xe/xe_debugfs.o
  CC      drivers/input/evdev.o
  CC      drivers/acpi/acpica/nspredef.o
  CC      drivers/base/auxiliary.o
  CC      lib/llist.o
  CC      drivers/ata/libata-pata-timings.o
  CC      lib/memweight.o
  CC [M]  drivers/i2c/busses/i2c-piix4.o
  AR      drivers/usb/serial/built-in.a
  CC      drivers/usb/gadget/udc/core.o
  CC      lib/kfifo.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/outp.o
  AR      drivers/usb/gadget/function/built-in.a
  CC      drivers/gpu/drm/drm_mipi_dsi.o
  CC      drivers/usb/gadget/udc/trace.o
  CC [M]  drivers/net/ethernet/broadcom/bnx2.o
  CC [M]  drivers/gpu/drm/i915/i915_mitigations.o
  CC      kernel/ptrace.o
  CC [M]  drivers/gpu/drm/drm_aperture.o
  CC [M]  drivers/gpu/drm/ast/ast_post.o
  CC [M]  drivers/gpu/drm/ast/ast_dp501.o
  CC [M]  fs/cifs/cifs_spnego_negtokeninit.asn1.o
  CC      drivers/acpi/acpica/nsprepkg.o
  CC [M]  drivers/gpu/drm/drm_atomic.o
  CC      arch/x86/kernel/crash_core_64.o
  CC      arch/x86/kernel/module.o
  CC      kernel/events/uprobes.o
  CC [M]  drivers/net/ethernet/broadcom/cnic.o
  CC [M]  drivers/ptp/ptp_vclock.o
  CC      drivers/usb/core/buffer.o
  CC      drivers/usb/core/sysfs.o
  CC      drivers/base/devtmpfs.o
  CC      kernel/trace/blktrace.o
  CC [M]  drivers/gpu/drm/xe/xe_devcoredump.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.o
  CC      drivers/usb/core/endpoint.o
  CC      drivers/usb/core/devio.o
  CC      mm/hugetlb_vmemmap.o
  CC [M]  drivers/gpu/drm/xe/xe_device.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/timer.o
  AR      drivers/rtc/built-in.a
  CC      drivers/acpi/acpica/nsrepair.o
  CC [M]  drivers/net/phy/phylink.o
  AR      drivers/usb/gadget/legacy/built-in.a
  CC      drivers/usb/gadget/usbstring.o
  CC [M]  drivers/gpu/drm/drm_atomic_uapi.o
  CC      fs/btrfs/orphan.o
  CC      drivers/ata/ahci.o
  CC [M]  drivers/usb/class/usbtmc.o
  CC      fs/btrfs/export.o
  CC      drivers/acpi/acpica/nsrepair2.o
  CC      drivers/usb/gadget/config.o
  LD [M]  net/bluetooth/bluetooth.o
  CC      lib/percpu-refcount.o
  AR      drivers/input/built-in.a
  CC      arch/x86/kernel/early_printk.o
  AR      drivers/power/reset/built-in.a
  CC      drivers/power/supply/power_supply_core.o
  CC      drivers/power/supply/power_supply_sysfs.o
  CC [M]  drivers/gpu/drm/ast/ast_dp.o
  AR      drivers/scsi/built-in.a
  CC      drivers/usb/core/notify.o
  CC      drivers/ata/libahci.o
  CC      drivers/hwmon/hwmon.o
  CC [M]  drivers/gpu/drm/drm_auth.o
  AR      drivers/thermal/broadcom/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvif/vmm.o
  AR      drivers/thermal/samsung/built-in.a
  CC [M]  drivers/hwmon/acpi_power_meter.o
  CC      drivers/thermal/intel/intel_tcc.o
  CC [M]  drivers/i2c/busses/i2c-designware-pcidrv.o
  AR      drivers/thermal/st/built-in.a
  CC      drivers/usb/gadget/epautoconf.o
  CC [M]  drivers/ptp/ptp_kvm_x86.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/user.o
  CC      drivers/watchdog/watchdog_core.o
  CC      drivers/thermal/intel/therm_throt.o
  CC      drivers/watchdog/watchdog_dev.o
  CC [M]  drivers/md/persistent-data/dm-array.o
  CC      kernel/trace/trace_events.o
  CC      drivers/base/memory.o
  CC      drivers/acpi/acpica/nssearch.o
  CC [M]  drivers/md/persistent-data/dm-bitset.o
  CC      arch/x86/kernel/hpet.o
  CC      drivers/base/module.o
  AR      drivers/usb/gadget/udc/built-in.a
  CC      drivers/md/md.o
  CC [M]  drivers/gpu/drm/xe/xe_dma_buf.o
  CC [M]  arch/x86/kvm/vmx/posted_intr.o
  CC      mm/sparse.o
  CC      drivers/usb/gadget/composite.o
  CC      drivers/power/supply/power_supply_leds.o
  CC      arch/x86/kernel/amd_nb.o
  CC      lib/rhashtable.o
  CC [M]  drivers/gpu/drm/nouveau/nvif/userc361.o
  CC      drivers/usb/host/xhci-dbg.o
  CC      mm/sparse-vmemmap.o
  CC [M]  drivers/gpu/drm/i915/i915_module.o
  CC [M]  drivers/gpu/drm/i915/i915_params.o
  CC      drivers/acpi/acpica/nsutils.o
  CC      drivers/acpi/acpica/nswalk.o
  CC      net/ipv4/igmp.o
  LD [M]  drivers/gpu/drm/ast/ast.o
  CC      drivers/usb/gadget/functions.o
  CC      drivers/usb/host/xhci-trace.o
  CC      drivers/acpi/acpica/nsxfeval.o
  CC      net/ipv4/fib_frontend.o
  CC [M]  drivers/ptp/ptp_kvm_common.o
  CC      kernel/user.o
  LD [M]  drivers/ptp/ptp.o
  CC      drivers/watchdog/softdog.o
  LD [M]  drivers/i2c/busses/i2c-designware-pci.o
  CC      fs/btrfs/tree-log.o
  CC      drivers/usb/host/xhci-debugfs.o
  AR      drivers/i2c/busses/built-in.a
  AR      drivers/i2c/muxes/built-in.a
  CC [M]  drivers/i2c/muxes/i2c-mux-gpio.o
  CC      fs/btrfs/free-space-cache.o
  CC      fs/btrfs/zlib.o
  CC      drivers/power/supply/power_supply_hwmon.o
  CC [M]  drivers/md/persistent-data/dm-block-manager.o
  AR      kernel/events/built-in.a
  CC      drivers/base/pinctrl.o
  CC      kernel/signal.o
  CC [M]  drivers/hwmon/coretemp.o
  CC [M]  drivers/thermal/intel/x86_pkg_temp_thermal.o
  CC      drivers/usb/host/xhci-pci.o
  CC      kernel/sys.o
  CC      drivers/md/md-bitmap.o
  CC      drivers/usb/gadget/configfs.o
  CC [M]  drivers/thermal/intel/intel_menlow.o
  CC      drivers/ata/ata_piix.o
  CC      drivers/base/devcoredump.o
  CC      arch/x86/kernel/kvm.o
  CC [M]  drivers/gpu/drm/drm_blend.o
  LD [M]  arch/x86/kvm/kvm.o
  AR      drivers/watchdog/built-in.a
  UPD     arch/x86/kvm/kvm-asm-offsets.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/client.o
  CC      fs/btrfs/lzo.o
  CC [M]  drivers/gpu/drm/xe/xe_engine.o
  CC      drivers/acpi/acpica/nsxfname.o
  CC      mm/mmu_notifier.o
  CC      kernel/umh.o
  CC      drivers/acpi/acpica/nsxfobj.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/engine.o
  AR      drivers/power/supply/built-in.a
  AR      drivers/power/built-in.a
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_object.o
  CC      drivers/usb/gadget/u_f.o
  CC [M]  drivers/gpu/drm/i915/i915_pci.o
  LD [M]  drivers/ptp/ptp_kvm.o
  CC      drivers/base/platform-msi.o
  CC [M]  drivers/gpu/drm/drm_bridge.o
  CC      drivers/opp/core.o
  AS [M]  arch/x86/kvm/vmx/vmenter.o
  CC      arch/x86/kernel/kvmclock.o
  CC      drivers/usb/core/generic.o
  CC [M]  drivers/md/persistent-data/dm-space-map-common.o
  CC      drivers/opp/cpu.o
  CC      drivers/i2c/i2c-boardinfo.o
  CC      drivers/opp/debugfs.o
  CC [M]  drivers/net/phy/aquantia_main.o
  CC      drivers/md/md-autodetect.o
  CC      drivers/acpi/acpica/psargs.o
  CC      drivers/base/physical_location.o
  CC      lib/base64.o
  AR      drivers/net/ethernet/cavium/common/built-in.a
  AR      drivers/net/ethernet/cavium/thunder/built-in.a
  CC      mm/ksm.o
  AR      drivers/net/ethernet/cavium/liquidio/built-in.a
  AR      drivers/net/ethernet/cavium/octeon/built-in.a
  AR      drivers/net/ethernet/cavium/built-in.a
  CC      mm/slub.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_gart.o
  AR      drivers/thermal/intel/built-in.a
  AR      drivers/thermal/qcom/built-in.a
  AR      drivers/thermal/tegra/built-in.a
  AR      drivers/thermal/mediatek/built-in.a
  AR      drivers/hwmon/built-in.a
  CC      drivers/thermal/thermal_core.o
  CC      net/ipv4/fib_semantics.o
  CC      lib/once.o
  CC      drivers/acpi/glue.o
  CC      net/ipv4/fib_trie.o
  CC      arch/x86/kernel/paravirt.o
  CC      drivers/i2c/i2c-core-base.o
  CC      kernel/workqueue.o
  CC      drivers/cpufreq/cpufreq.o
  CC      net/ipv4/fib_notifier.o
  CC      drivers/cpuidle/governors/menu.o
  CC      drivers/usb/core/quirks.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/enum.o
  CC      drivers/cpuidle/governors/haltpoll.o
  CC      drivers/base/trace.o
  AR      drivers/ata/built-in.a
  CC      kernel/pid.o
  CC      net/ipv4/inet_fragment.o
  CC      drivers/acpi/acpica/psloop.o
  CC      drivers/cpuidle/cpuidle.o
  CC [M]  fs/cifs/asn1.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_encoders.o
  CC      arch/x86/kernel/pvclock.o
  CC      lib/refcount.o
  CC [M]  drivers/net/ethernet/broadcom/tg3.o
  CC [M]  drivers/gpu/drm/xe/xe_exec.o
  CC [M]  drivers/gpu/drm/i915/i915_scatterlist.o
  CC      kernel/trace/trace_export.o
  CC      drivers/cpufreq/freq_table.o
  AR      drivers/usb/gadget/built-in.a
  CC [M]  drivers/gpu/drm/drm_cache.o
  CC [M]  drivers/net/phy/aquantia_hwmon.o
  CC [M]  drivers/net/phy/ax88796b.o
  CC      drivers/i2c/i2c-core-smbus.o
  CC      lib/usercopy.o
  CC [M]  drivers/md/persistent-data/dm-space-map-disk.o
  CC      drivers/usb/core/devices.o
  AR      drivers/usb/host/built-in.a
  CC      drivers/md/dm-uevent.o
  CC [M]  drivers/gpu/drm/drm_client.o
  CC      drivers/md/dm.o
  CC      mm/migrate.o
  CC      drivers/acpi/acpica/psobject.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_display.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/event.o
  CC      lib/errseq.o
  AR      drivers/net/ethernet/cortina/built-in.a
  AR      drivers/base/built-in.a
  AR      drivers/net/ethernet/engleder/built-in.a
  CC      arch/x86/kernel/pcspeaker.o
  CC [M]  drivers/gpu/drm/xe/xe_execlist.o
  AR      drivers/opp/built-in.a
  AR      drivers/cpuidle/governors/built-in.a
  CC      drivers/mmc/core/core.o
  AR      drivers/net/ethernet/ezchip/built-in.a
  CC [M]  drivers/gpu/drm/i915/i915_suspend.o
  AR      drivers/net/ethernet/fungible/built-in.a
  CC      drivers/cpufreq/cpufreq_performance.o
  CC [M]  drivers/gpu/drm/xe/xe_force_wake.o
  LD [M]  arch/x86/kvm/kvm-intel.o
  CC      lib/bucket_locks.o
  CC      drivers/mmc/host/sdhci.o
  CC      drivers/acpi/acpica/psopcode.o
  CC      drivers/mmc/host/sdhci-pci-core.o
  CC      kernel/trace/trace_event_perf.o
  CC      drivers/cpufreq/cpufreq_ondemand.o
  CC      kernel/task_work.o
  CC      drivers/md/dm-table.o
  CC [M]  drivers/net/phy/bcm7xxx.o
  CC      arch/x86/kernel/check.o
  CC [M]  drivers/gpu/drm/i915/i915_switcheroo.o
  CC [M]  drivers/md/persistent-data/dm-space-map-metadata.o
  CC      drivers/thermal/thermal_sysfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/firmware.o
  CC      kernel/extable.o
  CC      drivers/md/dm-target.o
  CC [M]  drivers/net/phy/bcm87xx.o
  CC      drivers/cpuidle/driver.o
  CC      drivers/acpi/acpica/psopinfo.o
  CC      kernel/params.o
  LD [M]  fs/cifs/cifs.o
  CC [M]  drivers/gpu/drm/i915/i915_sysfs.o
  CC      drivers/usb/core/phy.o
  CC      drivers/md/dm-linear.o
  AR      drivers/net/ethernet/huawei/built-in.a
  CC      drivers/md/dm-stripe.o
  CC      lib/generic-radix-tree.o
  CC      fs/btrfs/zstd.o
  CC [M]  drivers/md/persistent-data/dm-transaction-manager.o
  CC [M]  drivers/net/phy/bcm-phy-lib.o
  AR      drivers/ufs/built-in.a
  CC [M]  drivers/net/phy/broadcom.o
  AR      drivers/leds/trigger/built-in.a
  CC [M]  drivers/leds/trigger/ledtrig-audio.o
  AR      drivers/leds/blink/built-in.a
  CC      drivers/acpi/acpica/psparse.o
  AR      drivers/leds/simple/built-in.a
  CC      drivers/leds/led-core.o
  CC [M]  drivers/gpu/drm/xe/xe_ggtt.o
  CC      kernel/kthread.o
  CC      arch/x86/kernel/uprobes.o
  CC      drivers/cpuidle/governor.o
  CC      kernel/sys_ni.o
  CC      drivers/acpi/acpica/psscope.o
  CC [M]  drivers/md/persistent-data/dm-btree.o
  CC [M]  drivers/net/phy/lxt.o
  CC      drivers/i2c/i2c-core-acpi.o
  CC      lib/string_helpers.o
  CC      kernel/trace/trace_events_filter.o
  CC      drivers/leds/led-class.o
  CC      drivers/thermal/thermal_trip.o
  CC      drivers/cpufreq/cpufreq_governor.o
  CC      arch/x86/kernel/perf_regs.o
  CC      kernel/nsproxy.o
  CC [M]  drivers/net/phy/realtek.o
  CC      drivers/mmc/core/bus.o
  CC      lib/hexdump.o
  CC      net/ipv4/ping.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/gpuobj.o
  CC      drivers/usb/core/port.o
  CC      drivers/mmc/core/host.o
  CC      kernel/trace/trace_events_trigger.o
  CC      drivers/leds/led-triggers.o
  CC [M]  drivers/net/phy/smsc.o
  CC [M]  drivers/gpu/drm/i915/i915_utils.o
  CC      drivers/cpuidle/sysfs.o
  CC      drivers/acpi/acpica/pstree.o
  CC [M]  drivers/gpu/drm/i915/intel_clock_gating.o
  CC      drivers/cpuidle/poll_state.o
  CC      drivers/cpuidle/cpuidle-haltpoll.o
  CC      drivers/acpi/acpica/psutils.o
  CC      drivers/acpi/scan.o
  CC      drivers/cpufreq/cpufreq_governor_attr_set.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_i2c.o
  CC      drivers/thermal/thermal_helpers.o
  CC      drivers/acpi/resource.o
  LD [M]  drivers/net/phy/aquantia.o
  CC      net/ipv4/ip_tunnel_core.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.o
  CC      drivers/usb/core/hcd-pci.o
  CC      drivers/mmc/host/sdhci-pci-o2micro.o
  CC      drivers/usb/core/usb-acpi.o
  CC      drivers/acpi/acpi_processor.o
  CC      fs/dcache.o
  CC      fs/btrfs/compression.o
  CC [M]  drivers/gpu/drm/xe/xe_gt.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_clock.o
  CC      drivers/mmc/host/sdhci-pci-arasan.o
  CC      arch/x86/kernel/tracepoint.o
  AR      drivers/net/phy/built-in.a
  AR      drivers/net/usb/built-in.a
  CC [M]  drivers/net/usb/pegasus.o
  CC [M]  drivers/net/ipvlan/ipvlan_core.o
  CC      drivers/acpi/acpica/pswalk.o
  CC [M]  drivers/net/usb/rtl8150.o
  CC      drivers/mmc/host/sdhci-pci-dwc-mshc.o
  CC      lib/kstrtox.o
  CC [M]  drivers/net/vxlan/vxlan_core.o
  CC [M]  drivers/net/usb/r8152.o
  CC      drivers/mmc/core/mmc.o
  CC      drivers/i2c/i2c-core-slave.o
  CC      drivers/acpi/acpica/psxface.o
  CC      drivers/cpufreq/acpi-cpufreq.o
  AR      drivers/cpuidle/built-in.a
  AR      drivers/leds/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/intr.o
  CC      drivers/i2c/i2c-dev.o
  CC      drivers/acpi/processor_core.o
  CC      drivers/mmc/core/mmc_ops.o
  CC      fs/btrfs/delayed-ref.o
  CC      drivers/thermal/thermal_hwmon.o
  CC      drivers/mmc/core/sd.o
  CC      drivers/thermal/gov_fair_share.o
  CC [M]  drivers/net/vxlan/vxlan_multicast.o
  CC [M]  drivers/gpu/drm/i915/intel_device_info.o
  CC      arch/x86/kernel/itmt.o
  CC      drivers/acpi/acpica/rsaddr.o
  CC [M]  drivers/md/persistent-data/dm-btree-remove.o
  CC      kernel/notifier.o
  CC      drivers/mmc/core/sd_ops.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/ioctl.o
  CC      lib/debug_info.o
  CC      lib/iomap.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_debugfs.o
  CC      arch/x86/kernel/umip.o
  CC      drivers/acpi/processor_pdc.o
  CC      mm/migrate_device.o
  CC      drivers/mmc/core/sdio.o
  CC      kernel/trace/trace_eprobe.o
  CC      kernel/ksysfs.o
  AR      drivers/usb/core/built-in.a
  AR      drivers/usb/built-in.a
  CC      kernel/cred.o
  CC      fs/btrfs/relocation.o
  CC      drivers/thermal/gov_step_wise.o
  CC [M]  drivers/gpu/drm/i915/intel_memory_region.o
  CC      drivers/acpi/acpica/rscalc.o
  CC      drivers/mmc/core/sdio_ops.o
  CC      mm/huge_memory.o
  CC      drivers/acpi/ec.o
  CC [M]  drivers/i2c/i2c-smbus.o
  CC      net/ipv4/gre_offload.o
  CC      arch/x86/kernel/unwind_orc.o
  CC [M]  drivers/net/vxlan/vxlan_vnifilter.o
  CC      arch/x86/kernel/callthunks.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ring.o
  CC      drivers/cpufreq/intel_pstate.o
  CC      kernel/reboot.o
  CC      net/ipv4/metrics.o
  CC      arch/x86/kernel/mmconf-fam10h_64.o
  CC      drivers/thermal/gov_user_space.o
  CC      drivers/mmc/core/sdio_bus.o
  CC      kernel/async.o
  CC      drivers/mmc/core/sdio_cis.o
  CC      drivers/acpi/acpica/rscreate.o
  CC      arch/x86/kernel/vsmp_64.o
  CC      kernel/range.o
  CC      fs/btrfs/delayed-inode.o
  CC [M]  drivers/md/persistent-data/dm-btree-spine.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_mcr.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_pagefault.o
  CC      drivers/acpi/acpica/rsdumpinfo.o
  CC [M]  drivers/i2c/i2c-mux.o
  CC      kernel/smpboot.o
  CC      kernel/ucount.o
  CC [M]  drivers/net/ipvlan/ipvlan_main.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/memory.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/mm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/object.o
  CC      drivers/mmc/host/sdhci-pci-gli.o
  CC      mm/khugepaged.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.o
  AR      drivers/thermal/built-in.a
  CC      drivers/net/loopback.o
  CC      lib/pci_iomap.o
  CC [M]  drivers/gpu/drm/drm_client_modeset.o
  CC [M]  drivers/net/ipvlan/ipvlan_l3s.o
  CC      drivers/mmc/host/sdhci-acpi.o
  CC      fs/btrfs/scrub.o
  CC      drivers/net/netconsole.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_sysfs.o
  CC      drivers/acpi/acpica/rsinfo.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_tlb_invalidation.o
  CC      mm/page_counter.o
  CC [M]  drivers/gpu/drm/drm_color_mgmt.o
  AR      drivers/i2c/built-in.a
  CC      fs/btrfs/backref.o
  CC [M]  drivers/gpu/drm/drm_connector.o
  AR      arch/x86/kernel/built-in.a
  AR      arch/x86/built-in.a
  CC [M]  drivers/gpu/drm/drm_crtc.o
  CC      kernel/regset.o
  AR      drivers/firmware/arm_ffa/built-in.a
  LD [M]  drivers/md/persistent-data/dm-persistent-data.o
  CC      drivers/md/dm-ioctl.o
  AR      drivers/firmware/arm_scmi/built-in.a
  AR      drivers/firmware/broadcom/built-in.a
  AR      drivers/firmware/cirrus/built-in.a
  CC      drivers/mmc/core/sdio_io.o
  CC      net/ipv4/netlink.o
  AR      drivers/firmware/meson/built-in.a
  CC [M]  drivers/net/dummy.o
  CC      kernel/trace/trace_kprobe.o
  AR      drivers/firmware/imx/built-in.a
  CC      drivers/firmware/efi/efi-bgrt.o
  CC      drivers/firmware/efi/libstub/efi-stub-helper.o
  CC      kernel/kmod.o
  CC      drivers/acpi/acpica/rsio.o
  CC      drivers/firmware/efi/efi.o
  CC      drivers/firmware/efi/libstub/gop.o
  CC      fs/inode.o
  CC      drivers/firmware/efi/vars.o
  CC      net/ipv4/nexthop.o
  CC      fs/btrfs/ulist.o
  CC      drivers/acpi/acpica/rsirq.o
  CC      lib/iomap_copy.o
  CC      drivers/acpi/acpica/rslist.o
  CC [M]  drivers/net/usb/asix_devices.o
  CC      drivers/firmware/efi/libstub/secureboot.o
  CC [M]  drivers/net/usb/asix_common.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/oproxy.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/option.o
  CC      lib/devres.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_bios.o
  CC      net/ipv4/udp_tunnel_stub.o
  CC [M]  drivers/gpu/drm/drm_displayid.o
  CC      drivers/acpi/acpica/rsmemory.o
  CC [M]  drivers/net/macvlan.o
  CC      lib/check_signature.o
  CC      drivers/firmware/efi/reboot.o
  CC [M]  drivers/gpu/drm/i915/intel_pcode.o
  CC [M]  drivers/net/usb/ax88172a.o
  CC [M]  drivers/gpu/drm/xe/xe_gt_topology.o
  CC      drivers/mmc/host/cqhci-core.o
  CC      mm/memcontrol.o
  CC [M]  drivers/gpu/drm/drm_drv.o
  CC      drivers/mmc/core/sdio_irq.o
  CC      drivers/mmc/core/slot-gpio.o
  CC [M]  drivers/net/ethernet/intel/e1000/e1000_main.o
  CC      net/ipv4/sysctl_net_ipv4.o
  CC      net/ipv4/proc.o
  LD [M]  drivers/net/ipvlan/ipvlan.o
  CC [M]  drivers/net/mii.o
  CC [M]  drivers/net/ethernet/intel/e1000/e1000_hw.o
  CC [M]  drivers/net/ethernet/intel/e1000/e1000_ethtool.o
  CC      kernel/trace/error_report-traces.o
  CC      fs/btrfs/qgroup.o
  CC      drivers/firmware/efi/libstub/tpm.o
  CC      drivers/firmware/efi/memattr.o
  CC [M]  drivers/gpu/drm/drm_dumb_buffers.o
  CC      drivers/acpi/acpica/rsmisc.o
  AR      drivers/cpufreq/built-in.a
  AR      drivers/crypto/stm32/built-in.a
  AR      drivers/crypto/xilinx/built-in.a
  AR      drivers/crypto/hisilicon/built-in.a
  CC      drivers/firmware/efi/tpm.o
  AR      drivers/crypto/keembay/built-in.a
  AR      drivers/crypto/built-in.a
  CC      drivers/acpi/acpica/rsserial.o
  CC      drivers/clocksource/acpi_pm.o
  CC      lib/interval_tree.o
  CC      drivers/hid/hid-core.o
  CC      drivers/hid/usbhid/hid-core.o
  CC      drivers/firmware/efi/memmap.o
  CC      drivers/hid/usbhid/hiddev.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/ramht.o
  CC      fs/btrfs/send.o
  CC [M]  drivers/gpu/drm/xe/xe_guc.o
  CC      drivers/md/dm-io.o
  CC      fs/btrfs/dev-replace.o
  CC      lib/assoc_array.o
  CC      lib/list_debug.o
  CC [M]  drivers/net/usb/ax88179_178a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.o
  CC      drivers/mmc/core/regulator.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_ads.o
  CC [M]  drivers/mmc/host/sdhci-pltfm.o
  CC      drivers/firmware/efi/esrt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/subdev.o
  CC      drivers/md/dm-kcopyd.o
  CC      drivers/acpi/acpica/rsutils.o
  CC      fs/attr.o
  CC [M]  drivers/gpu/drm/drm_edid.o
  CC [M]  drivers/gpu/drm/i915/intel_region_ttm.o
  CC      drivers/firmware/efi/libstub/file.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_ct.o
  CC      net/ipv4/syncookies.o
  CC      drivers/clocksource/i8253.o
  CC      drivers/firmware/efi/efi-pstore.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/atombios_dp.o
  CC [M]  drivers/net/usb/cdc_ether.o
  CC      kernel/trace/power-traces.o
  CC      fs/bad_inode.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/core/uevent.o
  CC      fs/btrfs/raid56.o
  CC      drivers/acpi/acpica/rsxface.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_debugfs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_afmt.o
  CC      drivers/mmc/core/debugfs.o
  CC      drivers/firmware/efi/cper.o
  CC      drivers/firmware/efi/libstub/mem.o
  AR      drivers/clocksource/built-in.a
  CC      fs/file.o
  LD [M]  drivers/net/vxlan/vxlan.o
  AR      drivers/mmc/host/built-in.a
  CC      fs/filesystems.o
  CC      fs/btrfs/uuid-tree.o
  CC      kernel/trace/rpm-traces.o
  CC      drivers/acpi/dock.o
  CC      fs/namespace.o
  CC      lib/debugobjects.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_trace_points.o
  CC      drivers/md/dm-sysfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/nvfw/fw.o
  CC      drivers/firmware/efi/cper_cxl.o
  CC      drivers/firmware/efi/runtime-wrappers.o
  CC      net/ipv4/esp4.o
  CC      kernel/trace/trace_dynevent.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/nvfw/hs.o
  CC [M]  drivers/gpu/drm/i915/intel_runtime_pm.o
  CC      net/ipv4/esp4_offload.o
  CC [M]  drivers/net/usb/cdc_eem.o
  CC      drivers/acpi/acpica/tbdata.o
  CC [M]  drivers/net/usb/smsc75xx.o
  CC      net/ipv4/netfilter.o
  AR      drivers/hid/usbhid/built-in.a
  CC      drivers/hid/hid-input.o
  CC [M]  drivers/net/ethernet/intel/e1000/e1000_param.o
  CC      lib/bitrev.o
  CC      drivers/firmware/efi/libstub/random.o
  CC      drivers/acpi/pci_root.o
  CC [M]  drivers/net/usb/smsc95xx.o
  AR      drivers/staging/media/built-in.a
  AR      drivers/staging/built-in.a
  CC [M]  drivers/net/mdio.o
  CC      drivers/firmware/efi/dev-path-parser.o
  CC      drivers/mmc/core/block.o
  AR      drivers/platform/x86/amd/built-in.a
  CC      drivers/mailbox/mailbox.o
  CC      drivers/platform/x86/intel/pmc/core.o
  CC      drivers/devfreq/devfreq.o
  CC [M]  drivers/devfreq/governor_simpleondemand.o
  CC      kernel/groups.o
  CC [M]  drivers/devfreq/governor_performance.o
  CC      drivers/powercap/powercap_sys.o
  AR      drivers/perf/built-in.a
  CC [M]  drivers/gpu/drm/xe/xe_guc_hwconfig.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/atombios_encoders.o
  AR      drivers/net/ethernet/i825xx/built-in.a
  CC      kernel/trace/trace_probe.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_log.o
  AR      drivers/firmware/psci/built-in.a
  AR      drivers/firmware/smccc/built-in.a
  CC      drivers/md/dm-stats.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/nvfw/ls.o
  CC [M]  drivers/platform/x86/intel/pmt/class.o
  CC      drivers/acpi/acpica/tbfadt.o
  CC [M]  drivers/net/usb/mcs7830.o
  AR      drivers/platform/surface/built-in.a
  CC      net/ipv4/inet_diag.o
  CC      net/ipv4/tcp_diag.o
  CC      drivers/ras/ras.o
  CC      drivers/firmware/efi/apple-properties.o
  CC      lib/crc16.o
  CC      net/ipv4/udp_diag.o
  CC      drivers/acpi/acpica/tbfind.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_sa.o
  CC      drivers/firmware/efi/libstub/randomalloc.o
  CC      kernel/trace/trace_uprobe.o
  CC      fs/btrfs/props.o
  CC      kernel/trace/rethook.o
  CC      drivers/firmware/efi/earlycon.o
  CC      drivers/mailbox/pcc.o
  CC [M]  drivers/gpu/drm/i915/intel_sbi.o
  CC      drivers/acpi/acpica/tbinstal.o
  AR      drivers/hwtracing/intel_th/built-in.a
  CC      drivers/acpi/acpica/tbprint.o
  CC      drivers/acpi/acpica/tbutils.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/atombios_i2c.o
  CC      lib/crc-t10dif.o
  CC      drivers/platform/x86/p2sb.o
  CC      net/ipv4/tcp_cubic.o
  CC [M]  drivers/net/ethernet/intel/e1000e/82571.o
  CC [M]  drivers/net/ethernet/intel/igb/igb_main.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_pc.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_main.o
  CC [M]  drivers/platform/x86/intel/pmt/telemetry.o
  CC [M]  drivers/net/ethernet/intel/e1000e/ich8lan.o
  CC      drivers/powercap/intel_rapl_common.o
  CC      drivers/platform/x86/pmc_atom.o
  CC      drivers/firmware/efi/cper-x86.o
  CC      drivers/firmware/efi/libstub/pci.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/nvfw/acr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/nvfw/flcn.o
  CC [M]  drivers/net/ethernet/intel/igb/igb_ethtool.o
  LD [M]  drivers/net/ethernet/intel/e1000/e1000.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_mac.o
  CC      net/ipv4/xfrm4_policy.o
  CC      drivers/acpi/acpica/tbxface.o
  CC      fs/btrfs/free-space-tree.o
  CC      drivers/platform/x86/intel/pmc/spt.o
  CC [M]  drivers/net/ethernet/intel/igb/e1000_82575.o
  HOSTCC  lib/gen_crc32table
  AR      drivers/mailbox/built-in.a
  CC [M]  drivers/net/ethernet/intel/igc/igc_i225.o
  CC      drivers/powercap/intel_rapl_msr.o
  CC      drivers/ras/debugfs.o
  CC      fs/btrfs/tree-checker.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_base.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_nvm.o
  CC      lib/libcrc32c.o
  CC      net/ipv4/xfrm4_state.o
  CC [M]  drivers/gpu/drm/xe/xe_guc_submit.o
  CC      drivers/acpi/pci_link.o
  CC [M]  drivers/net/usb/usbnet.o
  CC [M]  drivers/platform/x86/intel/pmt/crashlog.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_phy.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_diag.o
  AR      drivers/devfreq/built-in.a
  CC [M]  drivers/gpu/drm/i915/intel_step.o
  CC      drivers/md/dm-rq.o
  CC      drivers/android/binderfs.o
  CC [M]  drivers/gpu/drm/drm_encoder.o
  CC      drivers/acpi/acpica/tbxfload.o
  CC      mm/vmpressure.o
  CC      drivers/platform/x86/intel/pmc/cnp.o
  CC [M]  drivers/platform/x86/wmi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.o
  CC      drivers/firmware/efi/libstub/skip_spaces.o
  CC      drivers/android/binder.o
  CC      drivers/firmware/efi/libstub/lib-cmdline.o
  CC      drivers/mmc/core/queue.o
  CC      drivers/acpi/acpica/tbxfroot.o
  AR      drivers/ras/built-in.a
  CC      lib/xxhash.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.o
  CC      drivers/firmware/efi/libstub/lib-ctype.o
  CC [M]  drivers/gpu/drm/xe/xe_hw_engine.o
  CC      drivers/firmware/efi/libstub/alignedmem.o
  CC      drivers/platform/x86/intel/pmc/icl.o
  CC [M]  drivers/platform/x86/wmi-bmof.o
  CC      drivers/md/dm-io-rewind.o
  CC      drivers/nvmem/core.o
  AR      drivers/powercap/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/base.o
  CC [M]  drivers/mtd/chips/chipreg.o
  LD [M]  drivers/platform/x86/intel/pmt/pmt_class.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/cmdq.o
  LD [M]  drivers/platform/x86/intel/pmt/pmt_telemetry.o
  LD [M]  drivers/platform/x86/intel/pmt/pmt_crashlog.o
  CC      drivers/hid/hid-quirks.o
  CC [M]  drivers/uio/uio.o
  CC      drivers/md/dm-builtin.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/fw.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/msgq.o
  CC      drivers/platform/x86/intel/pmc/tgl.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_ethtool.o
  CC      drivers/acpi/acpica/utaddress.o
  CC      drivers/platform/x86/intel/turbo_max_3.o
  CC [M]  drivers/platform/x86/mxm-wmi.o
  CC      net/ipv4/xfrm4_input.o
  CC      drivers/platform/x86/intel/pmc/adl.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_ptp.o
  CC      lib/genalloc.o
  CC [M]  drivers/net/tun.o
  CC [M]  drivers/net/veth.o
  AR      kernel/trace/built-in.a
  CC      mm/swap_cgroup.o
  CC      kernel/kcmp.o
  CC [M]  drivers/gpu/drm/i915/intel_uncore.o
  CC [M]  drivers/platform/x86/intel_ips.o
  CC      net/ipv4/xfrm4_output.o
  CC      drivers/hid/hid-debug.o
  CC      drivers/android/binder_alloc.o
  CC      drivers/firmware/efi/libstub/relocate.o
  CC      mm/hugetlb_cgroup.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_dump.o
  CC [M]  drivers/platform/x86/intel/vsec.o
  AR      drivers/mmc/core/built-in.a
  AR      drivers/mmc/built-in.a
  CC      kernel/freezer.o
  CC      drivers/acpi/acpica/utalloc.o
  CC      mm/kmemleak.o
  CC [M]  drivers/md/dm-bufio.o
  CC [M]  drivers/mtd/mtdcore.o
  CC      drivers/platform/x86/intel/pmc/mtl.o
  CC      drivers/acpi/acpica/utascii.o
  CC      drivers/acpi/acpica/utbuffer.o
  CC [M]  drivers/vfio/pci/vfio_pci_core.o
  CC [M]  drivers/vfio/pci/vfio_pci_intrs.o
  CC [M]  drivers/vfio/vfio_main.o
  CC [M]  drivers/gpu/drm/xe/xe_hw_fence.o
  CC [M]  drivers/vfio/pci/vfio_pci_rdwr.o
  CC      drivers/acpi/acpica/utcksum.o
  CC [M]  drivers/platform/x86/intel/rst.o
  CC      drivers/platform/x86/intel/pmc/pltdrv.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/qmgr.o
  AR      drivers/net/ethernet/microsoft/built-in.a
  AR      drivers/net/ethernet/litex/built-in.a
  AR      drivers/net/ethernet/microchip/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/v1.o
  AR      drivers/net/ethernet/mscc/built-in.a
  CC      drivers/acpi/acpica/utcopy.o
  AR      drivers/net/ethernet/neterion/built-in.a
  CC      kernel/stacktrace.o
  CC      net/ipv4/xfrm4_protocol.o
  CC      drivers/acpi/pci_irq.o
  AR      drivers/nvmem/built-in.a
  CC      fs/btrfs/space-info.o
  AR      drivers/net/ethernet/netronome/built-in.a
  AR      drivers/net/ethernet/ni/built-in.a
  CC      fs/btrfs/block-rsv.o
  CC      fs/btrfs/delalloc-space.o
  CC      lib/percpu_counter.o
  CC      drivers/firmware/efi/libstub/printk.o
  CC [M]  drivers/pps/pps.o
  CC [M]  drivers/bluetooth/btusb.o
  CC      fs/btrfs/block-group.o
  CC      drivers/acpi/acpica/utexcep.o
  CC [M]  drivers/vfio/group.o
  CC [M]  drivers/net/usb/cdc_ncm.o
  CC [M]  drivers/pps/kapi.o
  CC      kernel/dma.o
  CC      lib/fault-inject.o
  CC [M]  drivers/net/ethernet/intel/igb/e1000_mac.o
  CC      drivers/hid/hidraw.o
  CC      lib/syscall.o
  AR      drivers/platform/x86/intel/pmc/built-in.a
  LD [M]  drivers/platform/x86/intel/intel_vsec.o
  CC      drivers/acpi/acpica/utdebug.o
  LD [M]  drivers/platform/x86/intel/intel-rst.o
  CC [M]  drivers/net/usb/r8153_ecm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_pt.o
  AR      drivers/platform/x86/intel/built-in.a
  AR      drivers/platform/x86/built-in.a
  AR      drivers/platform/built-in.a
  CC      drivers/acpi/acpica/utdecode.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_tsn.o
  CC      drivers/acpi/acpi_lpss.o
  CC [M]  drivers/net/ethernet/intel/e1000e/80003es2lan.o
  CC      drivers/acpi/acpica/utdelete.o
  CC [M]  drivers/gpu/drm/drm_file.o
  CC [M]  drivers/vfio/pci/vfio_pci_config.o
  CC      drivers/acpi/acpica/uterror.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.o
  CC [M]  drivers/vfio/iova_bitmap.o
  CC [M]  drivers/vfio/container.o
  CC      drivers/firmware/efi/libstub/vsprintf.o
  AR      drivers/net/ethernet/packetengines/built-in.a
  AR      drivers/net/ethernet/realtek/built-in.a
  CC [M]  drivers/net/ethernet/realtek/8139cp.o
  CC [M]  drivers/dca/dca-core.o
  CC      drivers/acpi/acpica/uteval.o
  CC [M]  drivers/gpu/drm/xe/xe_huc.o
  CC      kernel/smp.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/gm200.o
  CC      drivers/acpi/acpi_apd.o
  CC [M]  drivers/gpu/drm/xe/xe_huc_debugfs.o
  CC [M]  drivers/pps/sysfs.o
  CC [M]  drivers/dca/dca-sysfs.o
  CC [M]  drivers/gpu/drm/i915/intel_wakeref.o
  CC [M]  drivers/ssb/main.o
  CC      lib/dynamic_debug.o
  CC      mm/page_isolation.o
  CC [M]  drivers/bluetooth/btintel.o
  CC      drivers/firmware/efi/libstub/x86-stub.o
  CC [M]  drivers/mtd/mtdsuper.o
  CC [M]  drivers/md/dm-bio-prison-v1.o
  CC [M]  drivers/vfio/virqfd.o
  CC [M]  drivers/net/ethernet/realtek/8139too.o
  CC [M]  net/ipv4/ip_tunnel.o
  CC      fs/btrfs/discard.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_pll.o
  CC      drivers/acpi/acpica/utglobal.o
  LD [M]  drivers/pps/pps_core.o
  CC [M]  drivers/bluetooth/btbcm.o
  CC [M]  drivers/vhost/net.o
  CC [M]  drivers/vfio/pci/vfio_pci.o
  CC [M]  drivers/vfio/vfio_iommu_type1.o
  CC      drivers/hid/hid-generic.o
  CC [M]  drivers/mtd/mtdconcat.o
  CC      fs/btrfs/reflink.o
  CC      fs/btrfs/subpage.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.o
  CC      drivers/acpi/acpica/uthex.o
  CC      drivers/acpi/acpica/utids.o
  CC [M]  drivers/net/ethernet/intel/igc/igc_xdp.o
  AR      drivers/firmware/tegra/built-in.a
  CC [M]  drivers/mtd/mtdpart.o
  LD [M]  drivers/dca/dca.o
  CC      drivers/hid/hid-a4tech.o
  CC [M]  drivers/gpu/drm/xe/xe_irq.o
  CC [M]  drivers/net/ethernet/realtek/r8169_main.o
  AR      drivers/firmware/xilinx/built-in.a
  CC [M]  drivers/bluetooth/btrtl.o
  CC      kernel/uid16.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.o
  CC [M]  drivers/md/dm-bio-prison-v2.o
  CC [M]  drivers/vhost/vhost.o
  AR      drivers/firmware/efi/built-in.a
  CC      fs/btrfs/tree-mod-log.o
  CC      fs/btrfs/extent-io-tree.o
  CC [M]  drivers/mtd/mtdchar.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ctx.o
  LD [M]  drivers/vfio/vfio.o
  STUBCPY drivers/firmware/efi/libstub/alignedmem.stub.o
  STUBCPY drivers/firmware/efi/libstub/efi-stub-helper.stub.o
  CC      mm/early_ioremap.o
  STUBCPY drivers/firmware/efi/libstub/file.stub.o
  CC      lib/errname.o
  STUBCPY drivers/firmware/efi/libstub/gop.stub.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/gp102.o
  STUBCPY drivers/firmware/efi/libstub/lib-cmdline.stub.o
  STUBCPY drivers/firmware/efi/libstub/lib-ctype.stub.o
  CC      fs/btrfs/fs.o
  STUBCPY drivers/firmware/efi/libstub/mem.stub.o
  STUBCPY drivers/firmware/efi/libstub/pci.stub.o
  LD [M]  drivers/vfio/pci/vfio-pci-core.o
  CC [M]  drivers/net/ethernet/intel/e1000e/mac.o
  CC [M]  drivers/md/dm-crypt.o
  STUBCPY drivers/firmware/efi/libstub/printk.stub.o
  STUBCPY drivers/firmware/efi/libstub/random.stub.o
  STUBCPY drivers/firmware/efi/libstub/randomalloc.stub.o
  CC [M]  drivers/net/ethernet/intel/igb/e1000_nvm.o
  STUBCPY drivers/firmware/efi/libstub/relocate.stub.o
  LD [M]  drivers/net/usb/asix.o
  STUBCPY drivers/firmware/efi/libstub/secureboot.stub.o
  CC [M]  drivers/md/dm-thin.o
  CC      drivers/acpi/acpica/utinit.o
  STUBCPY drivers/firmware/efi/libstub/skip_spaces.stub.o
  CC [M]  drivers/net/ethernet/intel/e1000e/manage.o
  STUBCPY drivers/firmware/efi/libstub/tpm.stub.o
  CC [M]  drivers/ssb/scan.o
  STUBCPY drivers/firmware/efi/libstub/vsprintf.stub.o
  LD [M]  drivers/vfio/pci/vfio-pci.o
  STUBCPY drivers/firmware/efi/libstub/x86-stub.stub.o
  CC [M]  drivers/net/ethernet/intel/igb/e1000_phy.o
  CC [M]  drivers/md/dm-thin-metadata.o
  AR      drivers/firmware/efi/libstub/lib.a
  CC      drivers/firmware/dmi_scan.o
  CC      lib/nlattr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_sync.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.o
  CC      drivers/hid/hid-apple.o
  CC [M]  drivers/gpu/drm/i915/vlv_sideband.o
  CC [M]  drivers/net/ethernet/intel/igbvf/vf.o
  CC [M]  drivers/net/ethernet/intel/igbvf/mbx.o
  CC [M]  drivers/net/ethernet/intel/igbvf/ethtool.o
  CC      drivers/acpi/acpica/utlock.o
  LD [M]  drivers/net/ethernet/intel/igc/igc.o
  CC [M]  drivers/net/ethernet/intel/igbvf/netdev.o
  CC      fs/seq_file.o
  CC      kernel/kallsyms.o
  CC      kernel/acct.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_main.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_common.o
  CC [M]  drivers/gpu/drm/xe/xe_lrc.o
  CC      mm/cma.o
  CC      drivers/acpi/acpica/utmath.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/ga100.o
  CC [M]  drivers/ssb/sprom.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/falcon/ga102.o
  CC      fs/btrfs/messages.o
  CC [M]  drivers/net/ethernet/realtek/r8169_firmware.o
  CC      lib/checksum.o
  CC      fs/xattr.o
  CC      drivers/acpi/acpica/utmisc.o
  CC      fs/btrfs/bio.o
  LD [M]  drivers/mtd/mtd.o
  CC      fs/btrfs/lru_cache.o
  CC      drivers/firmware/dmi-sysfs.o
  CC      fs/btrfs/acl.o
  CC [M]  drivers/net/ethernet/intel/e1000e/nvm.o
  CC      drivers/acpi/acpica/utmutex.o
  CC [M]  drivers/net/ethernet/realtek/r8169_phy_config.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.o
  CC [M]  drivers/net/ethernet/intel/igb/e1000_mbx.o
  CC [M]  drivers/net/ethernet/intel/igb/e1000_i210.o
  CC      lib/cpu_rmap.o
  CC      fs/libfs.o
  CC      drivers/hid/hid-belkin.o
  CC      drivers/hid/hid-cherry.o
  CC      drivers/hid/hid-chicony.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_preempt_mgr.o
  CC      drivers/acpi/acpica/utnonansi.o
  CC      fs/fs-writeback.o
  CC [M]  net/ipv4/udp_tunnel_core.o
  CC      lib/dynamic_queue_limits.o
  CC      drivers/firmware/dmi-id.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/base.o
  CC [M]  drivers/vhost/iotlb.o
  CC      fs/pnode.o
  CC      lib/glob.o
  CC      drivers/hid/hid-cypress.o
  CC [M]  drivers/ssb/pci.o
  AR      drivers/android/built-in.a
  CC      drivers/hid/hid-ezkey.o
  CC [M]  drivers/gpu/drm/i915/vlv_suspend.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.o
  CC      fs/splice.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/lsfw.o
  CC      lib/strncpy_from_user.o
  CC [M]  net/ipv4/udp_tunnel_nic.o
  CC [M]  drivers/gpu/drm/xe/xe_migrate.o
  CC [M]  drivers/gpu/drm/xe/xe_mmio.o
  CC [M]  drivers/ssb/pcihost_wrapper.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_82599.o
  CC      drivers/acpi/acpica/utobject.o
  CC [M]  drivers/gpu/drm/i915/soc/intel_dram.o
  LD [M]  drivers/vhost/vhost_net.o
  CC [M]  drivers/gpu/drm/i915/soc/intel_gmch.o
  CC      drivers/acpi/acpica/utosi.o
  CC [M]  drivers/gpu/drm/i915/soc/intel_pch.o
  CC [M]  drivers/gpu/drm/i915/i915_memcpy.o
  CC      mm/secretmem.o
  CC      kernel/crash_core.o
  CC      drivers/firmware/memmap.o
  CC      lib/strnlen_user.o
  CC [M]  drivers/ssb/driver_chipcommon.o
  CC      drivers/hid/hid-kensington.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_82598.o
  CC      fs/sync.o
  CC      fs/utimes.o
  LD [M]  drivers/vhost/vhost_iotlb.o
  CC      fs/d_path.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm200.o
  CC [M]  drivers/net/ethernet/intel/igb/igb_ptp.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_virt.o
  CC      mm/userfaultfd.o
  CC      drivers/acpi/acpi_platform.o
  CC      drivers/acpi/acpica/utownerid.o
  CC      fs/stack.o
  CC      drivers/hid/hid-lg.o
  CC      drivers/acpi/acpi_pnp.o
  CC      mm/memremap.o
  CC      fs/fs_struct.o
  CC      mm/hmm.o
  CC [M]  drivers/net/ethernet/intel/e1000e/phy.o
  LD [M]  drivers/md/dm-bio-prison.o
  AR      drivers/md/built-in.a
  CC [M]  drivers/ssb/driver_chipcommon_pmu.o
  AR      fs/btrfs/built-in.a
  CC [M]  drivers/gpu/drm/drm_fourcc.o
  CC      lib/net_utils.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_phy.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.o
  CC      mm/memfd.o
  CC      kernel/compat.o
  CC      mm/bootmem_info.o
  LD [M]  drivers/net/ethernet/realtek/r8169.o
  AR      drivers/firmware/built-in.a
  CC [M]  drivers/ssb/driver_pcicore.o
  AR      drivers/net/ethernet/renesas/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/gm20b.o
  LD [M]  drivers/md/dm-thin-pool.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_sriov.o
  CC [M]  drivers/gpu/drm/drm_framebuffer.o
  CC [M]  drivers/net/ethernet/intel/igb/igb_hwmon.o
  CC      drivers/acpi/acpica/utpredef.o
  CC      drivers/acpi/acpica/utresdecode.o
  CC      drivers/hid/hid-lg-g15.o
  LD [M]  drivers/net/ethernet/intel/igbvf/igbvf.o
  AR      drivers/net/ethernet/sfc/built-in.a
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_mbx.o
  CC [M]  drivers/gpu/drm/xe/xe_mocs.o
  CC      fs/statfs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vf_error.o
  CC      drivers/acpi/acpica/utresrc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp102.o
  CC      fs/fs_pin.o
  CC [M]  drivers/gpu/drm/drm_gem.o
  CC [M]  drivers/net/ethernet/intel/e1000e/param.o
  CC [M]  drivers/gpu/drm/drm_ioctl.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp108.o
  CC [M]  drivers/gpu/drm/i915/i915_mm.o
  CC      lib/sg_pool.o
  CC [M]  drivers/gpu/drm/xe/xe_module.o
  CC [M]  drivers/gpu/drm/xe/xe_pat.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_x540.o
  CC      fs/nsfs.o
  AR      drivers/net/ethernet/smsc/built-in.a
  CC      lib/stackdepot.o
  CC [M]  drivers/net/ethernet/smsc/smsc9420.o
  CC      drivers/acpi/acpica/utstate.o
  CC [M]  drivers/gpu/drm/i915/i915_sw_fence.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_x550.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/gv100.o
  AR      drivers/net/ethernet/socionext/built-in.a
  AR      drivers/net/ethernet/vertexcom/built-in.a
  AR      drivers/net/ethernet/wangxun/built-in.a
  AR      drivers/net/ethernet/xilinx/built-in.a
  CC      drivers/acpi/acpica/utstring.o
  AR      drivers/net/ethernet/synopsys/built-in.a
  LD [M]  net/ipv4/udp_tunnel.o
  AR      drivers/net/ethernet/pensando/built-in.a
  CC      drivers/acpi/acpica/utstrsuppt.o
  AR      net/ipv4/built-in.a
  AR      net/built-in.a
  CC [M]  drivers/gpu/drm/drm_lease.o
  CC      drivers/acpi/power.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/gp10b.o
  CC      drivers/acpi/event.o
  CC      fs/fs_types.o
  LD [M]  drivers/ssb/ssb.o
  CC      kernel/utsname.o
  CC [M]  drivers/gpu/drm/xe/xe_pci.o
  CC [M]  drivers/net/ethernet/intel/ixgbevf/vf.o
  CC [M]  drivers/gpu/drm/xe/xe_pcode.o
  CC      drivers/hid/hid-microsoft.o
  CC [M]  drivers/gpu/drm/xe/xe_pm.o
  CC      kernel/user_namespace.o
  CC      kernel/pid_namespace.o
  CC      kernel/stop_machine.o
  UPD     kernel/config_data
  CC      kernel/kprobes.o
  AR      mm/built-in.a
  CC [M]  drivers/gpu/drm/drm_managed.o
  CC      fs/fs_context.o
  CC      drivers/acpi/acpica/utstrtoul64.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/tu102.o
  CC [M]  drivers/net/ethernet/intel/e1000e/ethtool.o
  CC      fs/fs_parser.o
  CC      kernel/hung_task.o
  LD [M]  drivers/net/ethernet/intel/igb/igb.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_sched.o
  CC [M]  drivers/net/ethernet/intel/ixgb/ixgb_main.o
  CC [M]  drivers/net/ethernet/intel/ixgb/ixgb_hw.o
  CC      drivers/acpi/acpica/utxface.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.o
  CC [M]  drivers/net/ethernet/intel/ixgb/ixgb_ee.o
  CC [M]  drivers/net/ethernet/intel/ixgb/ixgb_ethtool.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ids.o
  CC [M]  drivers/net/ethernet/intel/ixgb/ixgb_param.o
  CC      lib/ucs2_string.o
  CC      lib/sbitmap.o
  CC      drivers/acpi/acpica/utxfinit.o
  CC [M]  drivers/net/ethernet/intel/e1000e/netdev.o
  CC [M]  drivers/gpu/drm/i915/i915_sw_fence_work.o
  AR      drivers/net/ethernet/intel/built-in.a
  CC      drivers/acpi/acpica/utxferror.o
  CC      drivers/acpi/evged.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga100.o
  CC [M]  drivers/gpu/drm/i915/i915_syncmap.o
  CC      drivers/acpi/sysfs.o
  CC [M]  drivers/net/ethernet/intel/e1000e/ptp.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_lib.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.o
  CC      drivers/acpi/property.o
  CC [M]  drivers/net/ethernet/intel/e100.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.o
  CC      kernel/watchdog.o
  CC [M]  drivers/gpu/drm/i915/i915_user_extensions.o
  CC      drivers/acpi/acpica/utxfmutex.o
  CC [M]  drivers/gpu/drm/i915/i915_ioc32.o
  CC [M]  drivers/gpu/drm/xe/xe_preempt_fence.o
  CC      drivers/hid/hid-monterey.o
  CC [M]  drivers/gpu/drm/i915/i915_debugfs.o
  CC      lib/group_cpus.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_dcb.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_dcb_82598.o
  CC      kernel/watchdog_hld.o
  CC [M]  drivers/gpu/drm/drm_mm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/acr/ga102.o
  CC [M]  drivers/gpu/drm/drm_mode_config.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/base.o
  CC [M]  drivers/gpu/drm/drm_mode_object.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_dcb_82599.o
  CC [M]  drivers/net/ethernet/intel/ixgbevf/mbx.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/nv50.o
  CC      kernel/seccomp.o
  CC      kernel/relay.o
  CC      fs/fsopen.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_dcb_nl.o
  CC      fs/init.o
  CC      fs/kernel_read_file.o
  CC      kernel/utsname_sysctl.o
  AR      drivers/acpi/acpica/built-in.a
  CC      drivers/acpi/acpi_cmos_rtc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/g84.o
  CC [M]  lib/asn1_decoder.o
  GEN     lib/oid_registry_data.c
  CC [M]  lib/oid_registry.o
  CC [M]  drivers/gpu/drm/i915/i915_debugfs_params.o
  CC [M]  drivers/gpu/drm/i915/display/intel_display_debugfs.o
  CC [M]  drivers/gpu/drm/xe/xe_pt.o
  CC      drivers/acpi/x86/apple.o
  CC [M]  drivers/gpu/drm/i915/display/intel_pipe_crc.o
  CC      drivers/acpi/x86/utils.o
  AR      drivers/hid/built-in.a
  CC [M]  drivers/net/ethernet/intel/ixgbevf/ethtool.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_sysfs.o
  CC      kernel/delayacct.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/gf100.o
  CC      kernel/taskstats.o
  CC      kernel/tsacct.o
  AR      lib/lib.a
  GEN     lib/crc32table.h
  CC      lib/crc32.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_debugfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/gk20a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/gm107.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_hdp.o
  CC      fs/mnt_idmapping.o
  CC [M]  drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.o
  CC [M]  drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.o
  CC [M]  drivers/net/ethernet/intel/ixgbevf/ipsec.o
  CC      drivers/acpi/x86/s2idle.o
  CC [M]  drivers/gpu/drm/xe/xe_query.o
  CC      kernel/tracepoint.o
  CC      kernel/latencytop.o
  CC [M]  drivers/gpu/drm/drm_modes.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_csa.o
  CC [M]  drivers/gpu/drm/drm_modeset_lock.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/gm20b.o
  CC      kernel/irq_work.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bar/tu102.o
  CC      fs/remap_range.o
  CC      drivers/acpi/debugfs.o
  CC      drivers/acpi/acpi_lpat.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/base.o
  CC      drivers/acpi/acpi_lpit.o
  CC [M]  drivers/gpu/drm/drm_plane.o
  CC [M]  drivers/gpu/drm/drm_prime.o
  CC [M]  drivers/gpu/drm/xe/xe_reg_sr.o
  LD [M]  drivers/net/ethernet/intel/ixgb/ixgb.o
  CC      kernel/static_call.o
  CC      kernel/static_call_inline.o
  CC      kernel/user-return-notifier.o
  CC      kernel/padata.o
  AR      lib/built-in.a
  CC      kernel/jump_label.o
  CC      fs/buffer.o
  CC [M]  drivers/gpu/drm/i915/i915_pmu.o
  CC      drivers/acpi/prmt.o
  CC      fs/mpage.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/bit.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/boost.o
  CC [M]  drivers/gpu/drm/drm_print.o
  CC [M]  drivers/gpu/drm/drm_property.o
  CC      kernel/context_tracking.o
  CC      kernel/iomem.o
  CC [M]  drivers/gpu/drm/drm_pt_walk.o
  CC [M]  drivers/gpu/drm/i915/gt/gen2_engine_cs.o
  CC [M]  drivers/gpu/drm/drm_syncobj.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/conn.o
  CC [M]  drivers/gpu/drm/drm_sysfs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ras.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.o
  CC      drivers/acpi/acpi_pcc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.o
  CC [M]  drivers/gpu/drm/drm_trace_points.o
  CC [M]  drivers/gpu/drm/i915/gt/gen6_engine_cs.o
  CC [M]  drivers/gpu/drm/i915/gt/gen6_ppgtt.o
  CC [M]  drivers/gpu/drm/xe/xe_reg_whitelist.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/cstep.o
  CC      drivers/acpi/ac.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/dcb.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/disp.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.o
  CC      kernel/rseq.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.o
  CC      fs/proc_namespace.o
  CC [M]  drivers/gpu/drm/xe/xe_rtp.o
  CC      fs/direct-io.o
  CC [M]  drivers/gpu/drm/drm_vblank.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/dp.o
  GZIP    kernel/config_data.gz
  CC [M]  drivers/gpu/drm/drm_vblank_work.o
  CC      kernel/configs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_umc.o
  CC [M]  drivers/gpu/drm/drm_vma_manager.o
  CC [M]  drivers/gpu/drm/xe/xe_ring_ops.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/smu_v11_0_i2c.o
  CC      drivers/acpi/button.o
  CC      fs/eventpoll.o
  CC      drivers/acpi/fan_core.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/extdev.o
  CC      fs/anon_inodes.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/fan.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/gpio.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/i2c.o
  CC [M]  drivers/gpu/drm/drm_writeback.o
  CC [M]  drivers/gpu/drm/xe/xe_sa.o
  CC      drivers/acpi/fan_attr.o
  CC      drivers/acpi/processor_driver.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_fru_eeprom.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/iccsense.o
  CC [M]  drivers/gpu/drm/lib/drm_random.o
  CC      fs/signalfd.o
  CC [M]  drivers/gpu/drm/drm_ioc32.o
  CC [M]  drivers/gpu/drm/drm_panel.o
  CC [M]  drivers/gpu/drm/drm_pci.o
  CC      fs/timerfd.o
  CC [M]  drivers/gpu/drm/xe/xe_sched_job.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_rap.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_fw_attestation.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/image.o
  CC [M]  drivers/gpu/drm/i915/gt/gen7_renderclear.o
  CC [M]  drivers/gpu/drm/i915/gt/gen8_engine_cs.o
  LD [M]  drivers/net/ethernet/intel/ixgbe/ixgbe.o
  CC [M]  drivers/gpu/drm/i915/gt/gen8_ppgtt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.o
  CC [M]  drivers/gpu/drm/drm_debugfs.o
  CC      fs/eventfd.o
  CC      fs/userfaultfd.o
  AR      kernel/built-in.a
  CC      fs/aio.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_breadcrumbs.o
  CC      drivers/acpi/processor_thermal.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_context.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_context_sseu.o
  CC      drivers/acpi/processor_idle.o
  CC      drivers/acpi/processor_throttling.o
  CC      drivers/acpi/processor_perflib.o
  CC      drivers/acpi/container.o
  CC      drivers/acpi/thermal.o
  CC      drivers/acpi/acpi_memhotplug.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_engine_cs.o
  CC      drivers/acpi/ioapic.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/init.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/mxm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/npde.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_eeprom.o
  CC      fs/locks.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/pcir.o
  CC [M]  drivers/gpu/drm/xe/xe_step.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_mca.o
  CC [M]  drivers/gpu/drm/drm_debugfs_crc.o
  CC [M]  drivers/gpu/drm/drm_edid_load.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_engine_heartbeat.o
  CC      fs/binfmt_script.o
  CC [M]  drivers/gpu/drm/drm_panel_orientation_quirks.o
  CC      fs/binfmt_elf.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_psp_ta.o
  CC      fs/compat_binfmt_elf.o
  CC      drivers/acpi/battery.o
  CC      drivers/acpi/hed.o
  CC      drivers/acpi/bgrt.o
  CC      drivers/acpi/cppc_acpi.o
  CC [M]  drivers/gpu/drm/drm_buddy.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_lsdma.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ring_mux.o
  CC [M]  drivers/gpu/drm/xe/xe_sync.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_engine_pm.o
  CC      fs/mbcache.o
  CC [M]  drivers/gpu/drm/xe/xe_trace.o
  CC [M]  drivers/gpu/drm/drm_gem_shmem_helper.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/perf.o
  CC      drivers/acpi/spcr.o
  CC      drivers/acpi/acpi_pad.o
  CC [M]  drivers/acpi/acpi_video.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.o
  CC [M]  drivers/gpu/drm/xe/xe_ttm_sys_mgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/pll.o
  CC [M]  drivers/acpi/video_detect.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_engine_user.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/pmu.o
  CC [M]  drivers/gpu/drm/drm_suballoc.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_execlists_submission.o
  CC      fs/posix_acl.o
  CC      fs/coredump.o
  CC [M]  drivers/gpu/drm/drm_gem_ttm_helper.o
  CC      fs/drop_caches.o
  CC      fs/fhandle.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.o
  CC [M]  drivers/gpu/drm/drm_atomic_helper.o
  CC [M]  drivers/gpu/drm/xe/xe_ttm_stolen_mgr.o
  CC [M]  drivers/gpu/drm/xe/xe_ttm_vram_mgr.o
  LD [M]  drivers/net/ethernet/intel/e1000e/e1000e.o
  LD [M]  drivers/net/ethernet/intel/ixgbevf/ixgbevf.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/cik.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/cik_ih.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_ggtt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/power_budget.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_ggtt_fencing.o
  AR      drivers/net/ethernet/built-in.a
  AR      drivers/net/built-in.a
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/dce_v8_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v7_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/cik_sdma.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/uvd_v4_2.o
  CC [M]  drivers/gpu/drm/drm_atomic_state_helper.o
  CC [M]  drivers/gpu/drm/xe/xe_tuning.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/ramcfg.o
  CC [M]  drivers/gpu/drm/xe/xe_uc.o
  CC [M]  drivers/gpu/drm/xe/xe_uc_debugfs.o
  CC [M]  drivers/gpu/drm/drm_bridge_connector.o
  CC [M]  drivers/gpu/drm/xe/xe_uc_fw.o
  CC [M]  drivers/gpu/drm/xe/xe_vm.o
  CC [M]  drivers/gpu/drm/drm_crtc_helper.o
  CC [M]  drivers/gpu/drm/drm_damage_helper.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/rammap.o
  CC [M]  drivers/gpu/drm/drm_encoder_slave.o
  CC [M]  drivers/gpu/drm/drm_flip_work.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_clock_utils.o
  AR      drivers/acpi/built-in.a
  CC [M]  drivers/gpu/drm/amd/amdgpu/vce_v2_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/si.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gmc_v6_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v6_0.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_debugfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadow.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_engines_debugfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowacpi.o
  CC [M]  drivers/gpu/drm/xe/xe_vm_madvise.o
  CC [M]  drivers/gpu/drm/xe/xe_wait_user_fence.o
  CC [M]  drivers/gpu/drm/xe/xe_wa.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_irq.o
  CC [M]  drivers/gpu/drm/xe/xe_wopcm.o
  CC [M]  drivers/gpu/drm/xe/xe_display.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_mcr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/si_ih.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_pm.o
  CC [M]  drivers/gpu/drm/xe/display/xe_fb_pin.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.o
  CC [M]  drivers/gpu/drm/xe/display/xe_hdcp_gsc.o
  LD [M]  drivers/acpi/video.o
  CC [M]  drivers/gpu/drm/drm_format_helper.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowof.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_pm_irq.o
  CC [M]  drivers/gpu/drm/drm_gem_atomic_helper.o
  CC [M]  drivers/gpu/drm/drm_gem_framebuffer_helper.o
  CC [M]  drivers/gpu/drm/xe/display/xe_plane_initial.o
  CC [M]  drivers/gpu/drm/xe/display/xe_display_rps.o
  CC [M]  drivers/gpu/drm/drm_kms_helper_common.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/si_dma.o
  CC [M]  drivers/gpu/drm/xe/display/ext/i915_irq.o
  CC [M]  drivers/gpu/drm/drm_modeset_helper.o
  AR      fs/built-in.a
  CC [M]  drivers/gpu/drm/xe/display/ext/intel_clock_gating.o
  CC [M]  drivers/gpu/drm/xe/display/ext/intel_device_info.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_requests.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowpci.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_sysfs.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gtt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowramin.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_llc.o
  CC [M]  drivers/gpu/drm/drm_plane_helper.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/shadowrom.o
  CC [M]  drivers/gpu/drm/drm_probe_helper.o
  CC [M]  drivers/gpu/drm/drm_rect.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/dce_v6_0.o
  CC [M]  drivers/gpu/drm/xe/display/ext/intel_dram.o
  CC [M]  drivers/gpu/drm/xe/display/ext/intel_pch.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/uvd_v3_1.o
  CC [M]  drivers/gpu/drm/xe/i915-display/icl_dsi.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_atomic.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_atomic_plane.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_audio.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mxgpu_vi.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/timing.o
  CC [M]  drivers/gpu/drm/drm_self_refresh_helper.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nbio_v6_1.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_backlight.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_lrc.o
  CC [M]  drivers/gpu/drm/drm_simple_kms_helper.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_bios.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_migrate.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_mocs.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_ppgtt.o
  CC [M]  drivers/gpu/drm/bridge/panel.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_rc6.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/soc15.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_bw.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_region_lmem.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_cdclk.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_color.o
  CC [M]  drivers/gpu/drm/drm_fbdev_generic.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_combo_phy.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/therm.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_renderstate.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_connector.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_crtc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_crtc_state_dump.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_reset.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/emu_soc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_cursor.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mxgpu_ai.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_ddi.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_ring.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_ring_submission.o
  CC [M]  drivers/gpu/drm/drm_fb_helper.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nbio_v7_0.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_rps.o
  LD [M]  drivers/gpu/drm/drm.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_ddi_buf_trans.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/vmap.o
  LD [M]  drivers/gpu/drm/drm_shmem_helper.o
  LD [M]  drivers/gpu/drm/drm_suballoc_helper.o
  LD [M]  drivers/gpu/drm/drm_ttm_helper.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_sa_media.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/volt.o
  AR      drivers/gpu/drm/built-in.a
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/vpstate.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vega10_reg_init.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vega20_reg_init.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nbio_v7_4.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nbio_v2_3.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nv.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/xpio.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_sseu.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_sseu_debugfs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/arct_reg_init.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mxgpu_nv.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_debugfs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nbio_v7_2.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_timeline.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_power.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_wopcm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/hdp_v4_0.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_power_map.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/hdp_v5_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/aldebaran_reg_init.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_power_well.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/M0203.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_display_trace.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_workarounds.o
  CC [M]  drivers/gpu/drm/i915/gt/shmem_utils.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/M0205.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dkl_phy.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dmc.o
  CC [M]  drivers/gpu/drm/i915/gt/sysfs_engines.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/M0209.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/aldebaran.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/soc21.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/sienna_cichlid.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bios/P0260.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_ggtt_gmch.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/smu_v13_0_10.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nbio_v4_3.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_aux.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/hdp_v6_0.o
  LD [M]  drivers/gpu/drm/drm_kms_helper.o
  CC [M]  drivers/gpu/drm/i915/gt/gen6_renderstate.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_aux_backlight.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_hdcp.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_link_training.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/nbio_v7_7.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/hdp_v5_2.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/lsdma_v6_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bus/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/df_v1_7.o
  CC [M]  drivers/gpu/drm/i915/gt/gen7_renderstate.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dp_mst.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/df_v3_6.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bus/hwsq.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/df_v4_3.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gmc_v7_0.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dpll.o
  CC [M]  drivers/gpu/drm/i915/gt/gen8_renderstate.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gmc_v8_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bus/nv04.o
  CC [M]  drivers/gpu/drm/i915/gt/gen9_renderstate.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dpll_mgr.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_busy.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_clflush.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bus/nv31.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_context.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dpt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_create.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_dmabuf.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_drrs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gmc_v9_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfxhub_v1_1.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsb.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_domain.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_execbuffer.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bus/nv50.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_internal.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_object.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bus/g94.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gmc_v10_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsi.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsi_dcs_backlight.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_dsi_vbt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/bus/gf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/base.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fb.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_lmem.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_mman.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fbc.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_pages.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_phys.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v3_0.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fdi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_2.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gmc_v11_0.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fifo_underrun.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_frontbuffer.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_pm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/nv04.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_global_state.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_gmbus.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_region.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_shmem.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/nv40.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mmhub_v3_0_1.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0_3.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/umc_v6_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/nv50.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_shrinker.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_stolen.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hdcp.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hdmi.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_throttle.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hotplug.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/umc_v6_1.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_tiling.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/umc_v6_7.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_ttm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/umc_v8_7.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/g84.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/umc_v8_10.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/gt215.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_irq.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_hti.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_ttm_move.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_lspcon.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/mcp77.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ih.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_modeset_setup.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_modeset_verify.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_panel.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_userptr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/iceland_ih.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_pipe_crc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_pps.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gem_wait.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/tonga_ih.o
  CC [M]  drivers/gpu/drm/i915/gem/i915_gemfs.o
  CC [M]  drivers/gpu/drm/i915/i915_active.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/cz_ih.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vega10_ih.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_psr.o
  CC [M]  drivers/gpu/drm/i915/i915_cmd_parser.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_qp_tables.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vega20_ih.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_quirks.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_snps_phy.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/gf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk104.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/navi10_ih.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/ih_v6_0.o
  CC [M]  drivers/gpu/drm/i915/i915_deps.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_tc.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vblank.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vdsc.o
  CC [M]  drivers/gpu/drm/i915/i915_gem_evict.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_psp.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vga.o
  CC [M]  drivers/gpu/drm/i915/i915_gem_gtt.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_vrr.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_wm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/gk20a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/psp_v3_1.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/psp_v10_0.o
  CC [M]  drivers/gpu/drm/i915/i915_gem_ww.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/psp_v11_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/psp_v11_0_8.o
  CC [M]  drivers/gpu/drm/i915/i915_gem.o
  CC [M]  drivers/gpu/drm/xe/i915-display/skl_scaler.o
  CC [M]  drivers/gpu/drm/xe/i915-display/skl_universal_plane.o
  CC [M]  drivers/gpu/drm/i915/i915_query.o
  CC [M]  drivers/gpu/drm/i915/i915_request.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/psp_v12_0.o
  CC [M]  drivers/gpu/drm/i915/i915_scheduler.o
  CC [M]  drivers/gpu/drm/xe/i915-display/skl_watermark.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/psp_v13_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/psp_v13_0_4.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_acpi.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_opregion.o
  CC [M]  drivers/gpu/drm/i915/i915_trace_points.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/dce_v10_0.o
  CC [M]  drivers/gpu/drm/i915/i915_ttm_buddy_manager.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/gm20b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/pllnv04.o
  CC [M]  drivers/gpu/drm/xe/i915-display/intel_fbdev.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/clk/pllgt215.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/dce_v11_0.o
  CC [M]  drivers/gpu/drm/i915/i915_vma.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vkms.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_rlc.o
  HDRTEST drivers/gpu/drm/xe/abi/guc_klvs_abi.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v8_0.o
  HDRTEST drivers/gpu/drm/xe/abi/guc_errors_abi.h
  HDRTEST drivers/gpu/drm/xe/abi/guc_actions_slpc_abi.h
  HDRTEST drivers/gpu/drm/xe/abi/guc_communication_mmio_abi.h
  HDRTEST drivers/gpu/drm/xe/abi/guc_actions_abi.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/nv04.o
  CC [M]  drivers/gpu/drm/i915/i915_vma_resource.o
  HDRTEST drivers/gpu/drm/xe/abi/guc_communication_ctb_abi.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_gsc_fw.o
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v9_0.o
  HDRTEST drivers/gpu/drm/xe/abi/guc_messages_abi.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v9_4.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_vma_types.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/vlv_sideband_reg.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/intel_wakeref.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v10_0.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_drv.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_reg_defs.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_trace.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_heci_cmd_submit.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_reg.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_active_types.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_utils.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_config.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/imu_v11_0.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_vma.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/vlv_sideband.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/intel_mchbar_regs.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_debugfs.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_ads.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/soc/intel_gmch.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_capture.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v11_0.o
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_ct.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_vgpu.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/i915_fixed.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/nv05.o
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_fw.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/intel_runtime_pm.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/gfx_v11_0_3.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/imu_v11_0_3.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/intel_pm_types.h
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/intel_pci_config.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/nv10.o
  HDRTEST drivers/gpu/drm/xe/compat-i915-headers/intel_clock_gating.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/nv1a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v2_4.o
  HDRTEST drivers/gpu/drm/xe/display/ext/i915_irq.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_hwconfig.o
  HDRTEST drivers/gpu/drm/xe/display/ext/intel_pch.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/nv20.o
  HDRTEST drivers/gpu/drm/xe/display/ext/intel_dram.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v3_0.o
  HDRTEST drivers/gpu/drm/xe/display/ext/intel_device_info.h
  HDRTEST drivers/gpu/drm/xe/display/xe_de.h
  HDRTEST drivers/gpu/drm/xe/regs/xe_reg_defs.h
  HDRTEST drivers/gpu/drm/xe/regs/xe_gt_regs.h
  HDRTEST drivers/gpu/drm/xe/regs/xe_regs.h
  HDRTEST drivers/gpu/drm/xe/regs/xe_gpu_commands.h
  HDRTEST drivers/gpu/drm/xe/regs/xe_lrc_layout.h
  HDRTEST drivers/gpu/drm/xe/regs/xe_engine_regs.h
  HDRTEST drivers/gpu/drm/xe/tests/xe_test.h
  HDRTEST drivers/gpu/drm/xe/tests/xe_pci_test.h
  HDRTEST drivers/gpu/drm/xe/tests/xe_migrate_test.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_log.o
  HDRTEST drivers/gpu/drm/xe/tests/xe_dma_buf_test.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v4_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/nv50.o
  HDRTEST drivers/gpu/drm/xe/tests/xe_bo_test.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/g84.o
  HDRTEST drivers/gpu/drm/xe/xe_bb.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_rc.o
  HDRTEST drivers/gpu/drm/xe/xe_bb_types.h
  HDRTEST drivers/gpu/drm/xe/xe_bo.h
  HDRTEST drivers/gpu/drm/xe/xe_bo_doc.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.o
  HDRTEST drivers/gpu/drm/xe/xe_bo_evict.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v4_4.o
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_guc_submission.o
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_huc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v4_4_2.o
  HDRTEST drivers/gpu/drm/xe/xe_bo_types.h
  HDRTEST drivers/gpu/drm/xe/xe_debugfs.h
  HDRTEST drivers/gpu/drm/xe/xe_devcoredump.h
  HDRTEST drivers/gpu/drm/xe/xe_devcoredump_types.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_huc_debugfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/g98.o
  HDRTEST drivers/gpu/drm/xe/xe_device.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_huc_fw.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gt215.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/mcp89.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v5_0.o
  HDRTEST drivers/gpu/drm/xe/xe_device_types.h
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_uc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v5_2.o
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_uc_debugfs.o
  CC [M]  drivers/gpu/drm/i915/gt/uc/intel_uc_fw.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/sdma_v6_0.o
  CC [M]  drivers/gpu/drm/i915/gt/intel_gsc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_mes.o
  HDRTEST drivers/gpu/drm/xe/xe_display.h
  CC [M]  drivers/gpu/drm/i915/i915_hwmon.o
  HDRTEST drivers/gpu/drm/xe/xe_dma_buf.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/mes_v10_1.o
  HDRTEST drivers/gpu/drm/xe/xe_drv.h
  CC [M]  drivers/gpu/drm/i915/display/hsw_ips.o
  HDRTEST drivers/gpu/drm/xe/xe_engine.h
  HDRTEST drivers/gpu/drm/xe/xe_engine_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm107.o
  CC [M]  drivers/gpu/drm/i915/display/intel_atomic.o
  CC [M]  drivers/gpu/drm/i915/display/intel_atomic_plane.o
  CC [M]  drivers/gpu/drm/i915/display/intel_audio.o
  HDRTEST drivers/gpu/drm/xe/xe_exec.h
  HDRTEST drivers/gpu/drm/xe/xe_execlist.h
  CC [M]  drivers/gpu/drm/i915/display/intel_bios.o
  HDRTEST drivers/gpu/drm/xe/xe_execlist_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mes_v11_0.o
  HDRTEST drivers/gpu/drm/xe/xe_force_wake.h
  CC [M]  drivers/gpu/drm/i915/display/intel_bw.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gv100.o
  CC [M]  drivers/gpu/drm/i915/display/intel_cdclk.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.o
  HDRTEST drivers/gpu/drm/xe/xe_force_wake_types.h
  HDRTEST drivers/gpu/drm/xe/xe_ggtt.h
  CC [M]  drivers/gpu/drm/i915/display/intel_color.o
  CC [M]  drivers/gpu/drm/i915/display/intel_combo_phy.o
  HDRTEST drivers/gpu/drm/xe/xe_ggtt_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/tu102.o
  HDRTEST drivers/gpu/drm/xe/xe_gt.h
  CC [M]  drivers/gpu/drm/i915/display/intel_connector.o
  HDRTEST drivers/gpu/drm/xe/xe_gt_clock.h
  HDRTEST drivers/gpu/drm/xe/xe_gt_debugfs.h
  HDRTEST drivers/gpu/drm/xe/xe_gt_mcr.h
  HDRTEST drivers/gpu/drm/xe/xe_gt_pagefault.h
  HDRTEST drivers/gpu/drm/xe/xe_gt_sysfs.h
  CC [M]  drivers/gpu/drm/i915/display/intel_crtc.o
  HDRTEST drivers/gpu/drm/xe/xe_gt_sysfs_types.h
  HDRTEST drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h
  HDRTEST drivers/gpu/drm/xe/xe_gt_tlb_invalidation_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/devinit/ga100.o
  HDRTEST drivers/gpu/drm/xe/xe_gt_topology.h
  CC [M]  drivers/gpu/drm/i915/display/intel_crtc_state_dump.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/uvd_v5_0.o
  CC [M]  drivers/gpu/drm/i915/display/intel_cursor.o
  HDRTEST drivers/gpu/drm/xe/xe_gt_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fault/base.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fault/user.o
  HDRTEST drivers/gpu/drm/xe/xe_guc.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_ads.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_ads_types.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_ct.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_ct_types.h
  CC [M]  drivers/gpu/drm/i915/display/intel_display.o
  HDRTEST drivers/gpu/drm/xe/xe_guc_debugfs.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_engine_types.h
  CC [M]  drivers/gpu/drm/i915/display/intel_display_power.o
  HDRTEST drivers/gpu/drm/xe/xe_guc_fwif.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_hwconfig.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_log.h
  CC [M]  drivers/gpu/drm/i915/display/intel_display_power_map.o
  HDRTEST drivers/gpu/drm/xe/xe_guc_log_types.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_pc.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_pc_types.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_reg.h
  HDRTEST drivers/gpu/drm/xe/xe_guc_submit.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fault/gp100.o
  HDRTEST drivers/gpu/drm/xe/xe_guc_submit_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fault/gp10b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fault/gv100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/uvd_v6_0.o
  HDRTEST drivers/gpu/drm/xe/xe_guc_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fault/tu102.o
  HDRTEST drivers/gpu/drm/xe/xe_huc.h
  HDRTEST drivers/gpu/drm/xe/xe_huc_debugfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/uvd_v7_0.o
  HDRTEST drivers/gpu/drm/xe/xe_huc_types.h
  HDRTEST drivers/gpu/drm/xe/xe_hw_engine.h
  HDRTEST drivers/gpu/drm/xe/xe_hw_engine_types.h
  HDRTEST drivers/gpu/drm/xe/xe_hw_fence.h
  CC [M]  drivers/gpu/drm/i915/display/intel_display_power_well.o
  HDRTEST drivers/gpu/drm/xe/xe_hw_fence_types.h
  CC [M]  drivers/gpu/drm/i915/display/intel_display_rps.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dmc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vce.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vce_v3_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv04.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dpio_phy.o
  HDRTEST drivers/gpu/drm/xe/xe_irq.h
  HDRTEST drivers/gpu/drm/xe/xe_lrc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/vce_v4_0.o
  HDRTEST drivers/gpu/drm/xe/xe_lrc_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv10.o
  HDRTEST drivers/gpu/drm/xe/xe_macros.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv1a.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv20.o
  HDRTEST drivers/gpu/drm/xe/xe_map.h
  HDRTEST drivers/gpu/drm/xe/xe_migrate.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv25.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dpll.o
  HDRTEST drivers/gpu/drm/xe/xe_migrate_doc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.o
  HDRTEST drivers/gpu/drm/xe/xe_mmio.h
  CC [M]  drivers/gpu/drm/i915/display/intel_dpll_mgr.o
  HDRTEST drivers/gpu/drm/xe/xe_mocs.h
  CC [M]  drivers/gpu/drm/i915/display/intel_dpt.o
  CC [M]  drivers/gpu/drm/i915/display/intel_drrs.o
  HDRTEST drivers/gpu/drm/xe/xe_module.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/vcn_sw_ring.o
  HDRTEST drivers/gpu/drm/xe/xe_pat.h
  HDRTEST drivers/gpu/drm/xe/xe_pci.h
  HDRTEST drivers/gpu/drm/xe/xe_pci_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv30.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv35.o
  HDRTEST drivers/gpu/drm/xe/xe_pcode.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/vcn_v1_0.o
  HDRTEST drivers/gpu/drm/xe/xe_pcode_api.h
  HDRTEST drivers/gpu/drm/xe/xe_platform_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/vcn_v2_0.o
  HDRTEST drivers/gpu/drm/xe/xe_pm.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv36.o
  HDRTEST drivers/gpu/drm/xe/xe_preempt_fence.h
  CC [M]  drivers/gpu/drm/i915/display/intel_dsb.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv40.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv41.o
  HDRTEST drivers/gpu/drm/xe/xe_preempt_fence_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv44.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vcn_v2_5.o
  HDRTEST drivers/gpu/drm/xe/xe_pt.h
  HDRTEST drivers/gpu/drm/xe/xe_pt_types.h
  CC [M]  drivers/gpu/drm/i915/display/intel_fb.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vcn_v3_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv46.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv47.o
  HDRTEST drivers/gpu/drm/xe/xe_query.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv49.o
  CC [M]  drivers/gpu/drm/i915/display/intel_fb_pin.o
  CC [M]  drivers/gpu/drm/i915/display/intel_fbc.o
  HDRTEST drivers/gpu/drm/xe/xe_reg_sr.h
  HDRTEST drivers/gpu/drm/xe/xe_reg_sr_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv4e.o
  HDRTEST drivers/gpu/drm/xe/xe_reg_whitelist.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/nv50.o
  HDRTEST drivers/gpu/drm/xe/xe_res_cursor.h
  CC [M]  drivers/gpu/drm/i915/display/intel_fdi.o
  HDRTEST drivers/gpu/drm/xe/xe_ring_ops.h
  HDRTEST drivers/gpu/drm/xe/xe_ring_ops_types.h
  HDRTEST drivers/gpu/drm/xe/xe_rtp.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/g84.o
  HDRTEST drivers/gpu/drm/xe/xe_rtp_types.h
  HDRTEST drivers/gpu/drm/xe/xe_sa.h
  HDRTEST drivers/gpu/drm/xe/xe_sa_types.h
  HDRTEST drivers/gpu/drm/xe/xe_sched_job.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gt215.o
  CC [M]  drivers/gpu/drm/i915/display/intel_fifo_underrun.o
  CC [M]  drivers/gpu/drm/i915/display/intel_frontbuffer.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/vcn_v4_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/mcp77.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/mcp89.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf100.o
  CC [M]  drivers/gpu/drm/i915/display/intel_global_state.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_jpeg.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gf108.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.o
  HDRTEST drivers/gpu/drm/xe/xe_sched_job_types.h
  CC [M]  drivers/gpu/drm/i915/display/intel_hdcp.o
  HDRTEST drivers/gpu/drm/xe/xe_step.h
  HDRTEST drivers/gpu/drm/xe/xe_step_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk104.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk110.o
  HDRTEST drivers/gpu/drm/xe/xe_sync.h
  HDRTEST drivers/gpu/drm/xe/xe_sync_types.h
  CC [M]  drivers/gpu/drm/i915/display/intel_hdcp_gsc.o
  CC [M]  drivers/gpu/drm/i915/display/intel_hotplug.o
  HDRTEST drivers/gpu/drm/xe/xe_trace.h
  CC [M]  drivers/gpu/drm/i915/display/intel_hti.o
  CC [M]  drivers/gpu/drm/i915/display/intel_lpe_audio.o
  HDRTEST drivers/gpu/drm/xe/xe_ttm_stolen_mgr.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gk20a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.o
  HDRTEST drivers/gpu/drm/xe/xe_ttm_sys_mgr.h
  HDRTEST drivers/gpu/drm/xe/xe_ttm_vram_mgr.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm107.o
  CC [M]  drivers/gpu/drm/i915/display/intel_modeset_verify.o
  CC [M]  drivers/gpu/drm/i915/display/intel_modeset_setup.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/jpeg_v4_0.o
  CC [M]  drivers/gpu/drm/i915/display/intel_overlay.o
  HDRTEST drivers/gpu/drm/xe/xe_ttm_vram_mgr_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm200.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gm20b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp100.o
  CC [M]  drivers/gpu/drm/i915/display/intel_pch_display.o
  CC [M]  drivers/gpu/drm/i915/display/intel_pch_refclk.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gp10b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gv100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/athub_v1_0.o
  HDRTEST drivers/gpu/drm/xe/xe_tuning.h
  HDRTEST drivers/gpu/drm/xe/xe_uc.h
  HDRTEST drivers/gpu/drm/xe/xe_uc_debugfs.h
  HDRTEST drivers/gpu/drm/xe/xe_uc_fw.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/athub_v2_0.o
  CC [M]  drivers/gpu/drm/i915/display/intel_plane_initial.o
  CC [M]  drivers/gpu/drm/i915/display/intel_psr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/athub_v2_1.o
  HDRTEST drivers/gpu/drm/xe/xe_uc_fw_abi.h
  HDRTEST drivers/gpu/drm/xe/xe_uc_fw_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/athub_v3_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/tu102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga100.o
  HDRTEST drivers/gpu/drm/xe/xe_uc_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ga102.o
  HDRTEST drivers/gpu/drm/xe/xe_vm.h
  CC [M]  drivers/gpu/drm/i915/display/intel_quirks.o
  CC [M]  drivers/gpu/drm/i915/display/intel_sprite.o
  CC [M]  drivers/gpu/drm/i915/display/intel_sprite_uapi.o
  HDRTEST drivers/gpu/drm/xe/xe_vm_doc.h
  HDRTEST drivers/gpu/drm/xe/xe_vm_madvise.h
  HDRTEST drivers/gpu/drm/xe/xe_vm_types.h
  HDRTEST drivers/gpu/drm/xe/xe_wa.h
  HDRTEST drivers/gpu/drm/xe/xe_wait_user_fence.h
  HDRTEST drivers/gpu/drm/xe/xe_wopcm.h
  CC [M]  drivers/gpu/drm/i915/display/intel_tc.o
  HDRTEST drivers/gpu/drm/xe/xe_wopcm_types.h
  LD [M]  drivers/gpu/drm/xe/xe.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ram.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv04.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv10.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/smuio_v9_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv1a.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv20.o
  CC [M]  drivers/gpu/drm/i915/display/intel_vblank.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/smuio_v11_0.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/smuio_v11_0_6.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/smuio_v13_0.o
  CC [M]  drivers/gpu/drm/i915/display/intel_vga.o
  CC [M]  drivers/gpu/drm/i915/display/intel_wm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv40.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv41.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv44.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/smuio_v13_0_6.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_reset.o
  CC [M]  drivers/gpu/drm/i915/display/i9xx_plane.o
  CC [M]  drivers/gpu/drm/i915/display/i9xx_wm.o
  CC [M]  drivers/gpu/drm/i915/display/skl_scaler.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv49.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/mca_v3_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv4e.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramnv50.o
  CC [M]  drivers/gpu/drm/i915/display/skl_universal_plane.o
  CC [M]  drivers/gpu/drm/i915/display/skl_watermark.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgt215.o
drivers/gpu/drm/xe/xe.o: warning: objtool: intel_set_cpu_fifo_underrun_reporting+0x2a0: unreachable instruction
  CC [M]  drivers/gpu/drm/i915/display/intel_acpi.o
  CC [M]  drivers/gpu/drm/i915/display/intel_opregion.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/rammcp77.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgf108.o
  CC [M]  drivers/gpu/drm/i915/display/intel_fbdev.o
  CC [M]  drivers/gpu/drm/i915/display/dvo_ch7017.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgk104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgm107.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.o
  CC [M]  drivers/gpu/drm/i915/display/dvo_ch7xxx.o
  CC [M]  drivers/gpu/drm/i915/display/dvo_ivch.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_module.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_device.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_chardev.o
  CC [M]  drivers/gpu/drm/i915/display/dvo_ns2501.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgm200.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramgp100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_topology.o
  CC [M]  drivers/gpu/drm/i915/display/dvo_sil164.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_pasid.o
  CC [M]  drivers/gpu/drm/i915/display/dvo_tfp410.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/ramga102.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_doorbell.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/sddr2.o
  CC [M]  drivers/gpu/drm/i915/display/g4x_dp.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_flat_memory.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/sddr3.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_process.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_queue.o
  CC [M]  drivers/gpu/drm/i915/display/g4x_hdmi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_mqd_manager.o
  CC [M]  drivers/gpu/drm/i915/display/icl_dsi.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gddr3.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fb/gddr5.o
  CC [M]  drivers/gpu/drm/i915/display/intel_backlight.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_mqd_manager_cik.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fuse/base.o
  CC [M]  drivers/gpu/drm/i915/display/intel_crt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fuse/nv50.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_mqd_manager_vi.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fuse/gf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_mqd_manager_v9.o
  CC [M]  drivers/gpu/drm/i915/display/intel_ddi.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/fuse/gm107.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gpio/base.o
  CC [M]  drivers/gpu/drm/i915/display/intel_ddi_buf_trans.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_mqd_manager_v10.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gpio/nv10.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_mqd_manager_v11.o
  CC [M]  drivers/gpu/drm/i915/display/intel_display_trace.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_kernel_queue.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gpio/nv50.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dkl_phy.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dp.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dp_aux.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_packet_manager.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dp_aux_backlight.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_packet_manager_vi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_packet_manager_v9.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dp_hdcp.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gpio/g94.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dp_link_training.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_process_queue_manager.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_device_queue_manager.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_device_queue_manager_cik.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dp_mst.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_device_queue_manager_vi.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dsi.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gpio/gf119.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gpio/gk104.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gpio/ga102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gsp/base.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dsi_vbt.o
  CC [M]  drivers/gpu/drm/i915/display/intel_dvo.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gsp/gv100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/gsp/ga102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/base.o
  CC [M]  drivers/gpu/drm/i915/display/intel_gmbus.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_device_queue_manager_v9.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_device_queue_manager_v10.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_device_queue_manager_v11.o
  CC [M]  drivers/gpu/drm/i915/display/intel_hdmi.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/nv04.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_interrupt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_events.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/cik_event_interrupt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_int_process_v9.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_int_process_v11.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/nv4e.o
  CC [M]  drivers/gpu/drm/i915/display/intel_lspcon.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_smi_events.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/nv50.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/g94.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_crat.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/gf117.o
  CC [M]  drivers/gpu/drm/i915/display/intel_lvds.o
  CC [M]  drivers/gpu/drm/i915/display/intel_panel.o
  CC [M]  drivers/gpu/drm/i915/display/intel_pps.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_debugfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/gf119.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/gk104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/gk110.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/gm200.o
  CC [M]  drivers/gpu/drm/i915/display/intel_qp_tables.o
  CC [M]  drivers/gpu/drm/i915/display/intel_sdvo.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/pad.o
  CC [M]  drivers/gpu/drm/i915/display/intel_snps_phy.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/padnv04.o
  CC [M]  drivers/gpu/drm/i915/display/intel_tv.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/padnv4e.o
  CC [M]  drivers/gpu/drm/i915/display/intel_vdsc.o
  CC [M]  drivers/gpu/drm/i915/display/intel_vrr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/padnv50.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/padg94.o
  CC [M]  drivers/gpu/drm/i915/display/vlv_dsi.o
  CC [M]  drivers/gpu/drm/i915/display/vlv_dsi_pll.o
  CC [M]  drivers/gpu/drm/i915/i915_perf.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/padgf119.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp_tee.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_svm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../amdkfd/kfd_migrate.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/padgm200.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp_huc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/bus.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/busnv04.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/busnv4e.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp_cmd.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp_debugfs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/busnv50.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/busgf119.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/bit.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/aux.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/auxg94.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/auxgf119.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_aldebaran.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/auxgm200.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/i2c/anx9805.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp_irq.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v10_3.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/iccsense/gf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v11.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp_pm.o
  CC [M]  drivers/gpu/drm/i915/pxp/intel_pxp_session.o
  CC [M]  drivers/gpu/drm/i915/i915_gpu_error.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/instmem/base.o
  CC [M]  drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv04.o
  CC [M]  drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.o
  CC [M]  drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv40.o
  CC [M]  drivers/gpu/drm/i915/selftests/i915_random.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/instmem/nv50.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/instmem/gk20a.o
  CC [M]  drivers/gpu/drm/i915/selftests/i915_selftest.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/base.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gf100.o
  CC [M]  drivers/gpu/drm/i915/selftests/igt_atomic.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gk104.o
  CC [M]  drivers/gpu/drm/i915/selftests/igt_flush_test.o
  CC [M]  drivers/gpu/drm/i915/selftests/igt_live_test.o
  CC [M]  drivers/gpu/drm/i915/selftests/igt_mmap.o
  CC [M]  drivers/gpu/drm/i915/selftests/igt_reset.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm107.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gm200.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.o
  CC [M]  drivers/gpu/drm/i915/selftests/igt_spinner.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_job.o
  CC [M]  drivers/gpu/drm/i915/selftests/librapl.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/gp10b.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_acp.o
  CC [M]  drivers/gpu/drm/i915/i915_vgpu.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dkl_phy_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/ltc/ga102.o
  HDRTEST drivers/gpu/drm/i915/display/intel_crtc_state_dump.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/base.o
  HDRTEST drivers/gpu/drm/i915/display/hsw_ips.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../acp/acp_hw.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_ioc32.o
  HDRTEST drivers/gpu/drm/i915/display/g4x_hdmi.h
  HDRTEST drivers/gpu/drm/i915/display/intel_hdcp_regs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_overlay.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv04.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dmc.h
  HDRTEST drivers/gpu/drm/i915/display/intel_vga.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/amdgpu_hmm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu11/arcturus_ppt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv11.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv17.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv44.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/nv50.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu11/navi10_ppt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/g84.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/g98.o
  HDRTEST drivers/gpu/drm/i915/display/intel_audio.h
  HDRTEST drivers/gpu/drm/i915/display/intel_lvds.h
  HDRTEST drivers/gpu/drm/i915/display/intel_modeset_setup.h
  HDRTEST drivers/gpu/drm/i915/display/intel_cdclk.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu11/sienna_cichlid_ppt.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display_limits.h
  HDRTEST drivers/gpu/drm/i915/display/intel_hotplug.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/gt215.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dkl_phy.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu11/vangogh_ppt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu11/cyan_skillfish_ppt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/gf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk104.o
  HDRTEST drivers/gpu/drm/i915/display/intel_atomic.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dpll.h
  HDRTEST drivers/gpu/drm/i915/display/vlv_dsi_pll_regs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dp_mst.h
  HDRTEST drivers/gpu/drm/i915/display/g4x_dp.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu11/smu_v11_0.o
  HDRTEST drivers/gpu/drm/i915/display/intel_tc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu12/renoir_ppt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu12/smu_v12_0.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/gk20a.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/smu_v13_0.o
  HDRTEST drivers/gpu/drm/i915/display/intel_frontbuffer.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/gp10b.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dsi_vbt.h
  HDRTEST drivers/gpu/drm/i915/display/intel_psr.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/aldebaran_ppt.o
  HDRTEST drivers/gpu/drm/i915/display/intel_crt.h
  HDRTEST drivers/gpu/drm/i915/display/intel_opregion.h
  HDRTEST drivers/gpu/drm/i915/display/intel_snps_phy_regs.h
  HDRTEST drivers/gpu/drm/i915/display/i9xx_wm.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mc/ga100.o
  HDRTEST drivers/gpu/drm/i915/display/intel_global_state.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/yellow_carp_ppt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/smu_v13_0_0_ppt.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/nv04.o
  HDRTEST drivers/gpu/drm/i915/display/intel_lpe_audio.h
  HDRTEST drivers/gpu/drm/i915/display/intel_drrs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_display_rps.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/nv41.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/nv44.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/smu_v13_0_4_ppt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/smu_v13_0_5_ppt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/smu_v13_0_7_ppt.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu13/smu_v13_0_6_ppt.o
  HDRTEST drivers/gpu/drm/i915/display/intel_fbdev.h
  HDRTEST drivers/gpu/drm/i915/display/intel_hdmi.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/nv50.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/amdgpu_smu.o
  HDRTEST drivers/gpu/drm/i915/display/intel_fdi.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/swsmu/smu_cmn.o
  HDRTEST drivers/gpu/drm/i915/display/intel_fb.h
  HDRTEST drivers/gpu/drm/i915/display/intel_qp_tables.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/g84.o
  HDRTEST drivers/gpu/drm/i915/display/intel_vdsc.h
  HDRTEST drivers/gpu/drm/i915/display/intel_snps_phy.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display_core.h
  HDRTEST drivers/gpu/drm/i915/display/vlv_dsi_pll.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dvo_dev.h
  HDRTEST drivers/gpu/drm/i915/display/intel_hdcp.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/mcp77.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/smu8_smumgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/tonga_smumgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/fiji_smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_sdvo_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/polaris10_smumgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gk104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gk20a.o
  HDRTEST drivers/gpu/drm/i915/display/intel_pch_refclk.h
  HDRTEST drivers/gpu/drm/i915/display/intel_display_trace.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gm200.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gm20b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gp100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gp10b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/gv100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/tu102.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/iceland_smumgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/smu7_smumgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/mem.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/memnv04.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/memnv50.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/memgf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/vega10_smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display_power.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/smu10_smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/i9xx_plane.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dp_aux_backlight.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dpll_mgr.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.o
  HDRTEST drivers/gpu/drm/i915/display/vlv_dsi.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/ci_smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_plane_initial.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/vega12_smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_fifo_underrun.h
  HDRTEST drivers/gpu/drm/i915/display/intel_cursor.h
  HDRTEST drivers/gpu/drm/i915/display/vlv_dsi_regs.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/vegam_smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/skl_scaler.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmnv04.o
  HDRTEST drivers/gpu/drm/i915/display/intel_hti.h
  HDRTEST drivers/gpu/drm/i915/display/icl_dsi_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmnv41.o
  HDRTEST drivers/gpu/drm/i915/display/intel_atomic_plane.h
  HDRTEST drivers/gpu/drm/i915/display/skl_watermark.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmnv44.o
  HDRTEST drivers/gpu/drm/i915/display/intel_fbc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/smu9_smumgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display_reg_defs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_acpi.h
  HDRTEST drivers/gpu/drm/i915/display/intel_connector.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/smumgr/vega20_smumgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmnv50.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dpt.h
  HDRTEST drivers/gpu/drm/i915/display/intel_quirks.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dp_link_training.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/hwmgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_color.h
  HDRTEST drivers/gpu/drm/i915/display/intel_crtc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/processpptables.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display_debugfs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_modeset_verify.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/hardwaremanager.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display_power_well.h
  HDRTEST drivers/gpu/drm/i915/display/intel_wm.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmmcp77.o
  HDRTEST drivers/gpu/drm/i915/display/intel_pipe_crc.h
  HDRTEST drivers/gpu/drm/i915/display/intel_audio_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk104.o
  HDRTEST drivers/gpu/drm/i915/display/intel_panel.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu8_hwmgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_sprite.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/pppcielanes.o
  HDRTEST drivers/gpu/drm/i915/display/intel_wm_types.h
  HDRTEST drivers/gpu/drm/i915/display/intel_tv.h
  HDRTEST drivers/gpu/drm/i915/display/intel_hti_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgk20a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/process_pptables_v1_0.o
  HDRTEST drivers/gpu/drm/i915/display/intel_vrr.h
  HDRTEST drivers/gpu/drm/i915/display/skl_universal_plane.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/ppatomctrl.o
  HDRTEST drivers/gpu/drm/i915/display/intel_mg_phy_regs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_bw.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm200.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/ppatomfwctrl.o
  HDRTEST drivers/gpu/drm/i915/display/intel_de.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu7_hwmgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgm20b.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu7_powertune.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgp10b.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu7_thermal.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu7_clockpowergating.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmgv100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega10_processpptables.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega10_hwmgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega10_powertune.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega10_thermal.o
  HDRTEST drivers/gpu/drm/i915/display/intel_lvds_regs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_gmbus_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmmtu102.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dsi_dcs_backlight.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dvo.h
  HDRTEST drivers/gpu/drm/i915/display/intel_sdvo.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dp_aux.h
  HDRTEST drivers/gpu/drm/i915/display/intel_vdsc_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/umem.o
  HDRTEST drivers/gpu/drm/i915/display/intel_combo_phy.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dvo_regs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_gmbus.h
  HDRTEST drivers/gpu/drm/i915/display/intel_hdcp_gsc.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dsi.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dmc_regs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_ddi.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu10_hwmgr.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dsb.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/ummu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/pp_psm.o
  HDRTEST drivers/gpu/drm/i915/display/intel_bios.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega12_processpptables.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mmu/uvmm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega12_hwmgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mxm/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega12_thermal.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mxm/mxms.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/mxm/nv50.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/pp_overdriver.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu_helper.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega20_processpptables.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega20_hwmgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega20_powertune.o
  HDRTEST drivers/gpu/drm/i915/display/intel_pch_display.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega20_thermal.o
  HDRTEST drivers/gpu/drm/i915/display/intel_display_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/agp.o
  HDRTEST drivers/gpu/drm/i915/display/intel_backlight.h
  HDRTEST drivers/gpu/drm/i915/display/intel_vblank.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/common_baco.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dp.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega10_baco.o
  HDRTEST drivers/gpu/drm/i915/display/intel_backlight_regs.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega20_baco.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/base.o
  HDRTEST drivers/gpu/drm/i915/display/intel_combo_phy_regs.h
  HDRTEST drivers/gpu/drm/i915/display/intel_display_power_map.h
  HDRTEST drivers/gpu/drm/i915/display/intel_ddi_buf_trans.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/vega12_baco.o
  HDRTEST drivers/gpu/drm/i915/display/icl_dsi.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/pcie.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu9_baco.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/tonga_baco.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/nv04.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/polaris_baco.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/fiji_baco.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/ci_baco.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/nv40.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/hwmgr/smu7_baco.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/powerplay/amd_powerplay.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/nv46.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/legacy-dpm/legacy_dpm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/legacy-dpm/kv_dpm.o
  HDRTEST drivers/gpu/drm/i915/display/intel_lspcon.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/nv4c.o
  HDRTEST drivers/gpu/drm/i915/display/intel_dpio_phy.h
  HDRTEST drivers/gpu/drm/i915/display/intel_dp_hdcp.h
  HDRTEST drivers/gpu/drm/i915/display/intel_fb_pin.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/legacy-dpm/kv_smc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/g84.o
  HDRTEST drivers/gpu/drm/i915/display/intel_pps.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/g92.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/g94.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/legacy-dpm/si_dpm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/gf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/legacy-dpm/si_smc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/gf106.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/amdgpu_dpm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/amdgpu_pm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../pm/amdgpu_dpm_internal.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/gk104.o
  HDRTEST drivers/gpu/drm/i915/display/intel_sprite_uapi.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_ttm.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_region.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_context_types.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_lmem.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pci/gp100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_plane.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/base.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/memx.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_crtc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gt215.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_mman.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_irq.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_mst_types.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_object_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_color.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_context.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_clflush.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/dc_fpu.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gf119.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_tiling.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_stolen.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_ttm_pm.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk104.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_create.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_ttm_move.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_ioctls.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_services.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_domain.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_helpers.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk110.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_pp_smu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_psr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk208.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gk20a.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_internal.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_hdcp.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_dmabuf.h
  HDRTEST drivers/gpu/drm/i915/gem/selftests/mock_context.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm107.o
  HDRTEST drivers/gpu/drm/i915/gem/selftests/huge_gem_object.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_crc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/amdgpu_dm/amdgpu_dm_debugfs.o
  HDRTEST drivers/gpu/drm/i915/gem/selftests/mock_gem_object.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/basics/conversion.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/basics/fixpt31_32.o
  HDRTEST drivers/gpu/drm/i915/gem/selftests/mock_dmabuf.h
  HDRTEST drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm200.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_userptr.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_pm.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gm20b.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_shrinker.h
  HDRTEST drivers/gpu/drm/i915/gem/i915_gemfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp102.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/basics/vector.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/basics/dc_common.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/pmu/gp10b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/privring/gf100.o
  HDRTEST drivers/gpu/drm/i915/gem/i915_gem_object.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/privring/gf117.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/bios_parser.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/bios_parser_interface.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/bios_parser_helper.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/privring/gk104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/privring/gk20a.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_timeline_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/privring/gm200.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/command_table.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/command_table_helper.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/privring/gp10b.o
  HDRTEST drivers/gpu/drm/i915/gt/selftest_engine.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_breadcrumbs.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_engine_heartbeat.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/base.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_context_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/fan.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_execlists_submission.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/bios_parser_common.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/fannil.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/command_table2.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/command_table_helper2.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/bios_parser2.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_pm.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/dce60/command_table_helper_dce60.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/fanpwm.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/dce80/command_table_helper_dce80.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/fantog.o
  HDRTEST drivers/gpu/drm/i915/gt/selftest_rc6.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_llc_types.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/ic.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/temp.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/nv40.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_region_lmem.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/nv50.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/dce110/command_table_helper_dce110.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_requests.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/dce112/command_table_helper_dce112.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_ggtt_gmch.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/bios/dce112/command_table_helper2_dce112.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_print.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/g84.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/calcs/dce_calcs.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/calcs/custom_float.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/gt215.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/calcs/bw_fixed.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/display_mode_lib.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/display_rq_dlg_helpers.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dml1_display_rq_dlg_calc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf100.o
  HDRTEST drivers/gpu/drm/i915/gt/gen8_ppgtt.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/gf119.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn10/dcn10_fpu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/dcn20_fpu.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/gk104.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_mcr.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_timeline.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/gm107.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/gm200.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/display_mode_vba.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_rq_dlg_calc_20.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20.o
  HDRTEST drivers/gpu/drm/i915/gt/gen6_engine_cs.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_pm_debugfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/therm/gp100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/timer/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_rq_dlg_calc_20v2.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn20/display_mode_vba_20v2.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/timer/nv04.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/timer/nv40.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/timer/nv41.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn21/display_rq_dlg_calc_21.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/timer/gk20a.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_workarounds_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/top/base.o
  HDRTEST drivers/gpu/drm/i915/gt/selftest_rps.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_sa_media.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_debugfs.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_clock_utils.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_rps_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/top/gk104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/top/ga100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn21/display_mode_vba_21.o
  HDRTEST drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn30/dcn30_fpu.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/vfn/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn30/display_mode_vba_30.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn30/display_rq_dlg_calc_30.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn31/display_mode_vba_31.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn31/display_rq_dlg_calc_31.o
  HDRTEST drivers/gpu/drm/i915/gt/sysfs_engines.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/vfn/uvfn.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/vfn/gv100.o
  HDRTEST drivers/gpu/drm/i915/gt/gen7_renderclear.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn314/display_mode_vba_314.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn314/display_rq_dlg_calc_314.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_context.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_wopcm.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn32/display_mode_vba_32.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_mocs.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_engine_pm.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_sysfs.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn32/display_rq_dlg_calc_32.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_rc6.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn32/display_mode_vba_util_32.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/vfn/tu102.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_ring_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn31/dcn31_fpu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn32/dcn32_fpu.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/vfn/ga100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn321/dcn321_fpu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn301/dcn301_fpu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn302/dcn302_fpu.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_workarounds.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn303/dcn303_fpu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dcn314/dcn314_fpu.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_engine_regs.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_pm_irq.h
  HDRTEST drivers/gpu/drm/i915/gt/shmem_utils.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_engine.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/base.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/gpio.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_reset_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/nv40.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/dsc/rc_calc_fpu.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_regs.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_reset.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_gsc_fw.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/guc_capture_fwif.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_uc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/calcs/dcn_calcs.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/gf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/gf117.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_uc_fw_abi.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/gk104.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/calcs/dcn_calc_math.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_print.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_fw.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_debugfs.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/abi/guc_klvs_abi.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/abi/guc_errors_abi.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/gk20a.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/abi/guc_actions_slpc_abi.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dml/calcs/dcn_calc_auto.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/abi/guc_communication_mmio_abi.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/abi/guc_actions_abi.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/abi/guc_communication_ctb_abi.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/abi/guc_messages_abi.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_gsc_uc_heci_cmd_submit.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/subdev/volt/gm20b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/falcon.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_reg.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dce60/dce60_clk_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dce100/dce_clk_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dce110/dce110_clk_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dce112/dce112_clk_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dce120/dce120_clk_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn10/rv1_clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_gsc_uc.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/xtensa.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn10/rv1_clk_mgr_vbios_smu.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/bsp/g84.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gt215.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_huc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn10/rv2_clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn20/dcn20_clk_mgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gk104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gm107.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gm200.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn201/dcn201_clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_huc_fw.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_fwif.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_capture.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_log_debugfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gp100.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_submission.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_slpc_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn21/rn_clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_log.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_ct.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gp102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/gv100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn21/rn_clk_mgr_vbios_smu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn30/dcn30_clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_slpc.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/tu102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/ga100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn30/dcn30_clk_mgr_smu_msg.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/ce/ga102.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_uc_fw.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_ads.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn301/vg_clk_mgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/cipher/g84.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn301/dcn301_smu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn31/dcn31_smu.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/device/acpi.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_uc_debugfs.h
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_guc_rc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn31/dcn31_clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/uc/intel_huc_debugfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/device/base.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_hwconfig.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn314/dcn314_smu.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_llc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn314/dcn314_clk_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn315/dcn315_smu.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn315/dcn315_clk_mgr.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/device/ctrl.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn316/dcn316_smu.o
  HDRTEST drivers/gpu/drm/i915/gt/gen8_engine_cs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/device/pci.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_sseu_debugfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/device/tegra.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_rc6_types.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_context_param.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gpu_commands.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_engine_user.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_irq.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/device/user.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gsc.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn316/dcn316_clk_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn32/dcn32_clk_mgr.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_rps.h
  HDRTEST drivers/gpu/drm/i915/gt/selftest_llc.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/clk_mgr/dcn32/dcn32_clk_mgr_smu_msg.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_audio.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/chan.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_stream_encoder.o
  HDRTEST drivers/gpu/drm/i915/gt/gen6_ppgtt.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_link_encoder.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_hwseq.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_ggtt_fencing.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_mem_input.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_migrate_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_clock_source.o
  HDRTEST drivers/gpu/drm/i915/gt/selftests/mock_timeline.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/conn.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_lrc.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_lrc_reg.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_migrate.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_sysfs_pm.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/hdmi.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_breadcrumbs_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/head.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.o
  HDRTEST drivers/gpu/drm/i915/gt/mock_engine.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/vga.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_scl_filters.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_engine_stats.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_transform.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_opp.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/nv04.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_dmcu.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gtt.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_abm.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_buffer_pool_types.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_ring.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_ipp.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/g84.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_aux.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/g94.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gt200.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_renderstate.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_i2c.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_i2c_hw.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_sseu.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_i2c_sw.o
  HDRTEST drivers/gpu/drm/i915/gt/intel_engine_types.h
  HDRTEST drivers/gpu/drm/i915/gt/intel_gt_engines_debugfs.h
  HDRTEST drivers/gpu/drm/i915/gt/gen2_engine_cs.h
  HDRTEST drivers/gpu/drm/i915/gvt/gvt.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp77.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gt215.o
  HDRTEST drivers/gpu/drm/i915/gvt/trace.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/mcp89.o
  HDRTEST drivers/gpu/drm/i915/gvt/debug.h
  HDRTEST drivers/gpu/drm/i915/gvt/edid.h
  HDRTEST drivers/gpu/drm/i915/gvt/page_track.h
  HDRTEST drivers/gpu/drm/i915/gvt/mmio.h
  HDRTEST drivers/gpu/drm/i915/gvt/sched_policy.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gf119.o
  HDRTEST drivers/gpu/drm/i915/gvt/fb_decoder.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gk104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gk110.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dmub_psr.o
  HDRTEST drivers/gpu/drm/i915/gvt/cmd_parser.h
  HDRTEST drivers/gpu/drm/i915/gvt/dmabuf.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gm107.o
  HDRTEST drivers/gpu/drm/i915/gvt/mmio_context.h
  HDRTEST drivers/gpu/drm/i915/gvt/display.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gm200.o
  HDRTEST drivers/gpu/drm/i915/gvt/gtt.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gp100.o
  HDRTEST drivers/gpu/drm/i915/gvt/scheduler.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dmub_abm.o
  HDRTEST drivers/gpu/drm/i915/gvt/reg.h
  HDRTEST drivers/gpu/drm/i915/gvt/execlist.h
  HDRTEST drivers/gpu/drm/i915/gvt/interrupt.h
  HDRTEST drivers/gpu/drm/i915/i915_active.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gp102.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dce_panel_cntl.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/gv100.o
  HDRTEST drivers/gpu/drm/i915/i915_active_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/tu102.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dmub_hw_lock_mgr.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dce/dmub_outbox.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/ga102.o
  HDRTEST drivers/gpu/drm/i915/i915_cmd_parser.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/gpio_base.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/udisp.o
  HDRTEST drivers/gpu/drm/i915/i915_config.h
  HDRTEST drivers/gpu/drm/i915/i915_debugfs.h
  HDRTEST drivers/gpu/drm/i915/i915_debugfs_params.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/gpio_service.o
  HDRTEST drivers/gpu/drm/i915/i915_deps.h
  HDRTEST drivers/gpu/drm/i915/i915_driver.h
  HDRTEST drivers/gpu/drm/i915/i915_drm_client.h
  HDRTEST drivers/gpu/drm/i915/i915_drv.h
  HDRTEST drivers/gpu/drm/i915/i915_file_private.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/uconn.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/hw_factory.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/uoutp.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/hw_gpio.o
  HDRTEST drivers/gpu/drm/i915/i915_fixed.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/disp/uhead.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/hw_hpd.o
  HDRTEST drivers/gpu/drm/i915/i915_gem.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/hw_ddc.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/hw_generic.o
  HDRTEST drivers/gpu/drm/i915/i915_gem_evict.h
  HDRTEST drivers/gpu/drm/i915/i915_gem_gtt.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/hw_translate.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/nv04.o
  HDRTEST drivers/gpu/drm/i915/i915_gem_ww.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/nv50.o
  HDRTEST drivers/gpu/drm/i915/i915_getparam.h
  HDRTEST drivers/gpu/drm/i915/i915_gpu_error.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce60/hw_translate_dce60.o
  HDRTEST drivers/gpu/drm/i915/i915_hwmon.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/gf100.o
  HDRTEST drivers/gpu/drm/i915/i915_ioc32.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce60/hw_factory_dce60.o
  HDRTEST drivers/gpu/drm/i915/i915_ioctl.h
  HDRTEST drivers/gpu/drm/i915/i915_iosf_mbi.h
  HDRTEST drivers/gpu/drm/i915/i915_irq.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/gf119.o
  HDRTEST drivers/gpu/drm/i915/i915_memcpy.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/gv100.o
  HDRTEST drivers/gpu/drm/i915/i915_mitigations.h
  HDRTEST drivers/gpu/drm/i915/i915_mm.h
  HDRTEST drivers/gpu/drm/i915/i915_params.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/user.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce80/hw_translate_dce80.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce80/hw_factory_dce80.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/usernv04.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/usernv50.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/usergf100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce110/hw_translate_dce110.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/usergf119.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/dma/usergv100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce110/hw_factory_dce110.o
  HDRTEST drivers/gpu/drm/i915/i915_pci.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/cgrp.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/chan.o
  HDRTEST drivers/gpu/drm/i915/i915_perf.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/chid.o
  HDRTEST drivers/gpu/drm/i915/i915_perf_oa_regs.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce120/hw_translate_dce120.o
  HDRTEST drivers/gpu/drm/i915/i915_perf_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dce120/hw_factory_dce120.o
  HDRTEST drivers/gpu/drm/i915/i915_pmu.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/runl.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn10/hw_translate_dcn10.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn10/hw_factory_dcn10.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/runq.o
  HDRTEST drivers/gpu/drm/i915/i915_priolist_types.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn20/hw_translate_dcn20.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv04.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv10.o
  HDRTEST drivers/gpu/drm/i915/i915_pvinfo.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn20/hw_factory_dcn20.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv17.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv40.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn21/hw_translate_dcn21.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn21/hw_factory_dcn21.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/nv50.o
  HDRTEST drivers/gpu/drm/i915/i915_query.h
  HDRTEST drivers/gpu/drm/i915/i915_reg.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/g84.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn30/hw_translate_dcn30.o
  HDRTEST drivers/gpu/drm/i915/i915_reg_defs.h
  HDRTEST drivers/gpu/drm/i915/i915_request.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/g98.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn30/hw_factory_dcn30.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn315/hw_translate_dcn315.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn315/hw_factory_dcn315.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk104.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn32/hw_translate_dcn32.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/gpio/dcn32/hw_factory_dcn32.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/irq_service.o
  HDRTEST drivers/gpu/drm/i915/i915_scatterlist.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dce60/irq_service_dce60.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk110.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk208.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gk20a.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm107.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dce80/irq_service_dce80.o
  HDRTEST drivers/gpu/drm/i915/i915_scheduler.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gm200.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dce110/irq_service_dce110.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gp100.o
  HDRTEST drivers/gpu/drm/i915/i915_scheduler_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/gv100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/tu102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/ga102.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dce120/irq_service_dce120.o
  HDRTEST drivers/gpu/drm/i915/i915_selftest.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/ucgrp.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn10/irq_service_dcn10.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/fifo/uchan.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/base.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn20/irq_service_dcn20.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn21/irq_service_dcn21.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv04.o
  HDRTEST drivers/gpu/drm/i915/i915_suspend.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn201/irq_service_dcn201.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv10.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv15.o
  HDRTEST drivers/gpu/drm/i915/i915_sw_fence.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn30/irq_service_dcn30.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv17.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv20.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv25.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn302/irq_service_dcn302.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv2a.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv30.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv34.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn303/irq_service_dcn303.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn31/irq_service_dcn31.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv35.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn314/irq_service_dcn314.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn315/irq_service_dcn315.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/irq/dcn32/irq_service_dcn32.o
  HDRTEST drivers/gpu/drm/i915/i915_sw_fence_work.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv40.o
  HDRTEST drivers/gpu/drm/i915/i915_switcheroo.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/link_detection.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv44.o
  HDRTEST drivers/gpu/drm/i915/i915_syncmap.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/nv50.o
  HDRTEST drivers/gpu/drm/i915/i915_sysfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/g84.o
  HDRTEST drivers/gpu/drm/i915/i915_tasklet.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gt200.o
  HDRTEST drivers/gpu/drm/i915/i915_trace.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/mcp79.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gt215.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/mcp89.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/link_dpms.o
  HDRTEST drivers/gpu/drm/i915/i915_ttm_buddy_manager.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gf100.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gf104.o
  HDRTEST drivers/gpu/drm/i915/i915_user_extensions.h
  HDRTEST drivers/gpu/drm/i915/i915_utils.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gf108.o
  HDRTEST drivers/gpu/drm/i915/i915_vgpu.h
  HDRTEST drivers/gpu/drm/i915/i915_vma.h
  HDRTEST drivers/gpu/drm/i915/i915_vma_resource.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/link_factory.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/link_resource.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/link_validation.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gf110.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/accessories/link_dp_trace.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gf117.o
  HDRTEST drivers/gpu/drm/i915/i915_vma_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gf119.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gk104.o
  HDRTEST drivers/gpu/drm/i915/intel_clock_gating.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110.o
  HDRTEST drivers/gpu/drm/i915/intel_device_info.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/accessories/link_dp_cts.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/accessories/link_fpga.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gk110b.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/hwss/link_hwss_dio.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gk208.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gk20a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/hwss/link_hwss_dpia.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gm107.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/hwss/link_hwss_hpo_dp.o
  HDRTEST drivers/gpu/drm/i915/intel_gvt.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_hpd.o
  HDRTEST drivers/gpu/drm/i915/intel_mchbar_regs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gm200.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gm20b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gp100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_ddc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gp102.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dpcd.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gp104.o
  HDRTEST drivers/gpu/drm/i915/intel_memory_region.h
  HDRTEST drivers/gpu/drm/i915/intel_pci_config.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gp107.o
  HDRTEST drivers/gpu/drm/i915/intel_pcode.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_dpia.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gp108.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_training.o
  HDRTEST drivers/gpu/drm/i915/intel_region_ttm.h
  HDRTEST drivers/gpu/drm/i915/intel_runtime_pm.h
  HDRTEST drivers/gpu/drm/i915/intel_sbi.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gp10b.o
  HDRTEST drivers/gpu/drm/i915/intel_step.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/gv100.o
  HDRTEST drivers/gpu/drm/i915/intel_uncore.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_training_8b_10b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/tu102.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ga102.o
  HDRTEST drivers/gpu/drm/i915/intel_wakeref.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxnv40.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_training_128b_132b.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxnv50.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_training_dpia.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_tee.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf100.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_irq.h
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_session.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_training_auxless.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_43.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_training_fixed_vs_pe_retimer.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf108.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf110.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_phy.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_cmd.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_capability.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf117.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_edp_panel_control.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_types.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgf119.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk104.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_debugfs.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_cmn.h
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk110b.o
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_huc.h
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_pm.h
  HDRTEST drivers/gpu/drm/i915/pxp/intel_pxp_cmd_interface_42.h
  HDRTEST drivers/gpu/drm/i915/selftests/igt_live_test.h
  HDRTEST drivers/gpu/drm/i915/selftests/igt_atomic.h
  HDRTEST drivers/gpu/drm/i915/selftests/mock_gem_device.h
  HDRTEST drivers/gpu/drm/i915/selftests/mock_drm.h
  HDRTEST drivers/gpu/drm/i915/selftests/igt_reset.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_irq_handler.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk208.o
  HDRTEST drivers/gpu/drm/i915/selftests/intel_scheduler_helpers.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/link/protocols/link_dp_dpia_bw.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgk20a.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/virtual/virtual_link_encoder.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/virtual/virtual_stream_encoder.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/virtual/virtual_link_hwss.o
  HDRTEST drivers/gpu/drm/i915/selftests/lib_sw_fence.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dsc/dc_dsc.o
  HDRTEST drivers/gpu/drm/i915/selftests/i915_perf_selftests.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dsc/rc_calc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm107.o
  HDRTEST drivers/gpu/drm/i915/selftests/mock_uncore.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dsc/rc_calc_dpi.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn20/dcn20_resource.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn20/dcn20_init.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm200.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgm20b.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn20/dcn20_hwseq.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn20/dcn20_dpp.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn20/dcn20_dpp_cm.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp100.o
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn20/dcn20_hubp.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp102.o
  HDRTEST drivers/gpu/drm/i915/selftests/mock_gtt.h
  HDRTEST drivers/gpu/drm/i915/selftests/i915_mock_selftests.h
  CC [M]  drivers/gpu/drm/amd/amdgpu/../display/dc/dcn20/dcn20_mpc.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp104.o
  CC [M]  drivers/gpu/drm/nouveau/nvkm/engine/gr/ctxgp107.o
  C

[Intel-xe] ○ CI.BAT: info for Introduce xe_devcoredump.

[Patchwork]

[-- Attachment #1: Type: text/plain, Size: 343 bytes --]

== Series Details ==

Series: Introduce xe_devcoredump.
URL   : https://patchwork.freedesktop.org/series/117017/
State : info

== Summary ==

Participating hosts:
bat-atsm-2
bat-dg2-oem2
bat-adlp-7
Missing hosts results[2]:
bat-atsm-2
bat-adlp-7
Results: [xe-pw-117017v1](https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-117017v1/index.html)



[-- Attachment #2: Type: text/html, Size: 865 bytes --]

Re: [Intel-xe] [PATCH 00/14] Introduce xe_devcoredump.

[Matthew Brost]

On Wed, Apr 26, 2023 at 04:56:59PM -0400, Rodrigo Vivi wrote:
> Xe needs to align with other drivers on the way that the error states are
> dumped, avoiding a Xe only error_state solution. The goal is to use devcoredump
> infrastructure to report error states, since it produces a standardized way
> by exposing a virtual and temporary /sys/class/devcoredump device.
> 
> The initial goal is to have the simple_error_state in the devcoredump
> so we start using the infrastructure.
> 
> But this is just a start point to start building a useful and
> organized crash dump, using standard infrastructure. Later this
> will be changed to have output that can be parsed by tools and
> used for error replay.

We are certainly missing the GuC log, it would also be really nice to
get the ftrace included too. Not sure if the later is easy, I know I
looked into this on the i915 and couldn't figure it out but this was a
while ago and admittedly didn't try all that hard.

Matt 

> 
> Later, when we are in-tree, the goal is to collaborate with devcoredump
> infrastructure with overall possible improvements, like multiple file support
> for better organization of the dumps, snapshot support, dmesg extra print,
> and whatever may make sense and help the overall infrastructure.
> 
> Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
> Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
> 
> Rodrigo Vivi (14):
>   drm/xe: Fix print of RING_EXECLIST_SQ_CONTENTS_HI
>   drm/xe: Introduce the dev_coredump infrastructure.
>   drm/xe: Do not take any action if our device was removed.
>   drm/xe: Extract non mapped regions out of GuC CTB into its own struct.
>   drm/xe: Convert GuC CT print to snapshot capture and print.
>   drm/xe: Add GuC CT snapshot to xe_devcoredump.
>   drm/xe: Introduce guc_submit_types.h with relevant structs.
>   drm/xe: Convert GuC Engine print to snapshot capture and print.
>   drm/xe: Add GuC Submit Engine snapshot to xe_devcoredump.
>   drm/xe: Convert Xe HW Engine print to snapshot capture and print.
>   drm/xe: Add HW Engine snapshot to xe_devcoredump.
>   drm/xe: Limit CONFIG_DRM_XE_SIMPLE_ERROR_CAPTURE to itself.
>   drm/xe: Convert VM print to snapshot capture and print.
>   drm/xe: Add VM snapshot to xe_devcoredump.
> 
>  drivers/gpu/drm/xe/Kconfig                |   1 +
>  drivers/gpu/drm/xe/Makefile               |   1 +
>  drivers/gpu/drm/xe/regs/xe_engine_regs.h  |   3 +-
>  drivers/gpu/drm/xe/xe_devcoredump.c       | 227 ++++++++++++++++++
>  drivers/gpu/drm/xe/xe_devcoredump.h       |  22 ++
>  drivers/gpu/drm/xe/xe_devcoredump_types.h |  60 +++++
>  drivers/gpu/drm/xe/xe_device_types.h      |   4 +
>  drivers/gpu/drm/xe/xe_execlist.c          |   4 +-
>  drivers/gpu/drm/xe/xe_gt_debugfs.c        |   2 +-
>  drivers/gpu/drm/xe/xe_guc_ct.c            | 275 +++++++++++++++-------
>  drivers/gpu/drm/xe/xe_guc_ct.h            |   7 +-
>  drivers/gpu/drm/xe/xe_guc_ct_types.h      |  46 +++-
>  drivers/gpu/drm/xe/xe_guc_fwif.h          |  29 ---
>  drivers/gpu/drm/xe/xe_guc_submit.c        | 258 ++++++++++++++------
>  drivers/gpu/drm/xe/xe_guc_submit.h        |  10 +-
>  drivers/gpu/drm/xe/xe_guc_submit_types.h  | 155 ++++++++++++
>  drivers/gpu/drm/xe/xe_hw_engine.c         | 210 ++++++++++++-----
>  drivers/gpu/drm/xe/xe_hw_engine.h         |   8 +-
>  drivers/gpu/drm/xe/xe_hw_engine_types.h   |  78 ++++++
>  drivers/gpu/drm/xe/xe_pci.c               |   2 +
>  drivers/gpu/drm/xe/xe_vm.c                | 140 +++++++++--
>  drivers/gpu/drm/xe/xe_vm.h                |   6 +-
>  drivers/gpu/drm/xe/xe_vm_types.h          |  18 ++
>  23 files changed, 1288 insertions(+), 278 deletions(-)
>  create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.c
>  create mode 100644 drivers/gpu/drm/xe/xe_devcoredump.h
>  create mode 100644 drivers/gpu/drm/xe/xe_devcoredump_types.h
>  create mode 100644 drivers/gpu/drm/xe/xe_guc_submit_types.h
> 
> --
> 2.39.2