[PATCH] perf/x86/intel: Add a quirk for the calculation of the number of counters on Alder Lake

[PATCH] perf/x86/intel: Add a quirk for the calculation of the number of counters on Alder Lake

[kan.liang]

From: Kan Liang <kan.liang@linux.intel.com>

For some Alder Lake machine with all E-cores disabled in a BIOS, the
below warning may be triggered.

[ 2.010766] hw perf events fixed 5 > max(4), clipping!

Current perf code relies on the CPUID leaf 0xA and leaf 7.EDX[15] to
calculate the number of the counters and follow the below assumption.

For a hybrid configuration, the leaf 7.EDX[15] (X86_FEATURE_HYBRID_CPU)
is set. The leaf 0xA only enumerate the common counters. Linux perf has
to manually add the extra GP counters and fixed counters for P-cores.
For a non-hybrid configuration, the X86_FEATURE_HYBRID_CPU should not
be set. The leaf 0xA enumerates all counters.

However, that's not the case when all E-cores are disabled in a BIOS.
Although there are only P-cores in the system, the leaf 7.EDX[15]
(X86_FEATURE_HYBRID_CPU) is still set. But the leaf 0xA is updated
to enumerate all counters of P-cores. The inconsistency triggers the
warning.

Several software ways were considered to handle the inconsistency.
- Drop the leaf 0xA and leaf 7.EDX[15] CPUID enumeration support.
  Hardcode the number of counters. This solution may be a problem for
  virtualization. A hypervisor cannot control the number of counters
  in a Linux guest via changing the guest CPUID enumeration anymore.
- Find another CPUID bit that is also updated with E-cores disabled.
  There may be a problem in the virtualization environment too. Because
  a hypervisor may disable the feature/CPUID bit.
- The P-cores have a maximum of 8 GP counters and 4 fixed counters on
  ADL. The maximum number can be used to detect the case.
  This solution is implemented in this patch.

Fixes: ee72a94ea4a6 ("perf/x86/intel: Fix fixed counter check warning for some Alder Lake")
Reported-by: Damjan Marion (damarion) <damarion@cisco.com>
Tested-by: Damjan Marion (damarion) <damarion@cisco.com>
Reported-by: Chan Edison <edison_chan_gz@hotmail.com>
Signed-off-by: Kan Liang <kan.liang@linux.intel.com>
Cc: stable@vger.kernel.org
---
 arch/x86/events/intel/core.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index 187906e..f1201e8 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -6239,6 +6239,18 @@ __init int intel_pmu_init(void)
 			pmu->num_counters = x86_pmu.num_counters;
 			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
 		}
+
+		/* Quirk: For some Alder Lake machine, when all E-cores are disabled in
+		 * a BIOS, the leaf 0xA will enumerate all counters of P-cores. However,
+		 * the X86_FEATURE_HYBRID_CPU is still set. The above codes will
+		 * mistakenly add extra counters for P-cores. Correct the number of
+		 * counters here.
+		 */
+		if ((pmu->num_counters > 8) || (pmu->num_counters_fixed > 4)) {
+			pmu->num_counters = x86_pmu.num_counters;
+			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
+		}
+
 		pmu->max_pebs_events = min_t(unsigned, MAX_PEBS_EVENTS, pmu->num_counters);
 		pmu->unconstrained = (struct event_constraint)
 					__EVENT_CONSTRAINT(0, (1ULL << pmu->num_counters) - 1,
-- 
2.7.4

Re: [PATCH] perf/x86/intel: Add a quirk for the calculation of the number of counters on Alder Lake

[Peter Zijlstra]

On Tue, Jan 11, 2022 at 10:20:38AM -0800, kan.liang@linux.intel.com wrote:
> From: Kan Liang <kan.liang@linux.intel.com>
> 
> For some Alder Lake machine with all E-cores disabled in a BIOS, the
> below warning may be triggered.
> 
> [ 2.010766] hw perf events fixed 5 > max(4), clipping!
> 
> Current perf code relies on the CPUID leaf 0xA and leaf 7.EDX[15] to
> calculate the number of the counters and follow the below assumption.
> 
> For a hybrid configuration, the leaf 7.EDX[15] (X86_FEATURE_HYBRID_CPU)
> is set. The leaf 0xA only enumerate the common counters. Linux perf has
> to manually add the extra GP counters and fixed counters for P-cores.
> For a non-hybrid configuration, the X86_FEATURE_HYBRID_CPU should not
> be set. The leaf 0xA enumerates all counters.
> 
> However, that's not the case when all E-cores are disabled in a BIOS.
> Although there are only P-cores in the system, the leaf 7.EDX[15]
> (X86_FEATURE_HYBRID_CPU) is still set. But the leaf 0xA is updated
> to enumerate all counters of P-cores. The inconsistency triggers the
> warning.
> 
> Several software ways were considered to handle the inconsistency.
> - Drop the leaf 0xA and leaf 7.EDX[15] CPUID enumeration support.
>   Hardcode the number of counters. This solution may be a problem for
>   virtualization. A hypervisor cannot control the number of counters
>   in a Linux guest via changing the guest CPUID enumeration anymore.
> - Find another CPUID bit that is also updated with E-cores disabled.
>   There may be a problem in the virtualization environment too. Because
>   a hypervisor may disable the feature/CPUID bit.
> - The P-cores have a maximum of 8 GP counters and 4 fixed counters on
>   ADL. The maximum number can be used to detect the case.
>   This solution is implemented in this patch.

ARGH!! This is horrific :-(

This is also the N-th problem with hybrid enumeration; is there a plan
to fix all that for the next generation or are we going to keep muddling
things?

> Fixes: ee72a94ea4a6 ("perf/x86/intel: Fix fixed counter check warning for some Alder Lake")
> Reported-by: Damjan Marion (damarion) <damarion@cisco.com>
> Tested-by: Damjan Marion (damarion) <damarion@cisco.com>
> Reported-by: Chan Edison <edison_chan_gz@hotmail.com>
> Signed-off-by: Kan Liang <kan.liang@linux.intel.com>
> Cc: stable@vger.kernel.org
> ---
>  arch/x86/events/intel/core.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
> index 187906e..f1201e8 100644
> --- a/arch/x86/events/intel/core.c
> +++ b/arch/x86/events/intel/core.c
> @@ -6239,6 +6239,18 @@ __init int intel_pmu_init(void)
>  			pmu->num_counters = x86_pmu.num_counters;
>  			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
>  		}
> +
> +		/* Quirk: For some Alder Lake machine, when all E-cores are disabled in
> +		 * a BIOS, the leaf 0xA will enumerate all counters of P-cores. However,
> +		 * the X86_FEATURE_HYBRID_CPU is still set. The above codes will
> +		 * mistakenly add extra counters for P-cores. Correct the number of
> +		 * counters here.
> +		 */

I fixed that comment style for you.

> +		if ((pmu->num_counters > 8) || (pmu->num_counters_fixed > 4)) {
> +			pmu->num_counters = x86_pmu.num_counters;
> +			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
> +		}
> +
>  		pmu->max_pebs_events = min_t(unsigned, MAX_PEBS_EVENTS, pmu->num_counters);
>  		pmu->unconstrained = (struct event_constraint)
>  					__EVENT_CONSTRAINT(0, (1ULL << pmu->num_counters) - 1,
> -- 
> 2.7.4
> 

Re: [PATCH] perf/x86/intel: Add a quirk for the calculation of the number of counters on Alder Lake

[Liang, Kan]

On 1/12/2022 4:54 AM, Peter Zijlstra wrote:
> On Tue, Jan 11, 2022 at 10:20:38AM -0800, kan.liang@linux.intel.com wrote:
>> From: Kan Liang <kan.liang@linux.intel.com>
>>
>> For some Alder Lake machine with all E-cores disabled in a BIOS, the
>> below warning may be triggered.
>>
>> [ 2.010766] hw perf events fixed 5 > max(4), clipping!
>>
>> Current perf code relies on the CPUID leaf 0xA and leaf 7.EDX[15] to
>> calculate the number of the counters and follow the below assumption.
>>
>> For a hybrid configuration, the leaf 7.EDX[15] (X86_FEATURE_HYBRID_CPU)
>> is set. The leaf 0xA only enumerate the common counters. Linux perf has
>> to manually add the extra GP counters and fixed counters for P-cores.
>> For a non-hybrid configuration, the X86_FEATURE_HYBRID_CPU should not
>> be set. The leaf 0xA enumerates all counters.
>>
>> However, that's not the case when all E-cores are disabled in a BIOS.
>> Although there are only P-cores in the system, the leaf 7.EDX[15]
>> (X86_FEATURE_HYBRID_CPU) is still set. But the leaf 0xA is updated
>> to enumerate all counters of P-cores. The inconsistency triggers the
>> warning.
>>
>> Several software ways were considered to handle the inconsistency.
>> - Drop the leaf 0xA and leaf 7.EDX[15] CPUID enumeration support.
>>    Hardcode the number of counters. This solution may be a problem for
>>    virtualization. A hypervisor cannot control the number of counters
>>    in a Linux guest via changing the guest CPUID enumeration anymore.
>> - Find another CPUID bit that is also updated with E-cores disabled.
>>    There may be a problem in the virtualization environment too. Because
>>    a hypervisor may disable the feature/CPUID bit.
>> - The P-cores have a maximum of 8 GP counters and 4 fixed counters on
>>    ADL. The maximum number can be used to detect the case.
>>    This solution is implemented in this patch.
> 
> ARGH!! This is horrific :-(
> 
> This is also the N-th problem with hybrid enumeration; is there a plan
> to fix all that for the next generation or are we going to keep muddling
> things?

Yes, that's annoying. We are working on it for the future generation.
The internal validation team is also enhancing the test case to test 
different configurations.

> 
>> Fixes: ee72a94ea4a6 ("perf/x86/intel: Fix fixed counter check warning for some Alder Lake")
>> Reported-by: Damjan Marion (damarion) <damarion@cisco.com>
>> Tested-by: Damjan Marion (damarion) <damarion@cisco.com>
>> Reported-by: Chan Edison <edison_chan_gz@hotmail.com>
>> Signed-off-by: Kan Liang <kan.liang@linux.intel.com>
>> Cc: stable@vger.kernel.org
>> ---
>>   arch/x86/events/intel/core.c | 12 ++++++++++++
>>   1 file changed, 12 insertions(+)
>>
>> diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
>> index 187906e..f1201e8 100644
>> --- a/arch/x86/events/intel/core.c
>> +++ b/arch/x86/events/intel/core.c
>> @@ -6239,6 +6239,18 @@ __init int intel_pmu_init(void)
>>   			pmu->num_counters = x86_pmu.num_counters;
>>   			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
>>   		}
>> +
>> +		/* Quirk: For some Alder Lake machine, when all E-cores are disabled in
>> +		 * a BIOS, the leaf 0xA will enumerate all counters of P-cores. However,
>> +		 * the X86_FEATURE_HYBRID_CPU is still set. The above codes will
>> +		 * mistakenly add extra counters for P-cores. Correct the number of
>> +		 * counters here.
>> +		 */
> 
> I fixed that comment style for you.

Ah, sorry for that. Thanks!

Kan

> 
>> +		if ((pmu->num_counters > 8) || (pmu->num_counters_fixed > 4)) {
>> +			pmu->num_counters = x86_pmu.num_counters;
>> +			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
>> +		}
>> +
>>   		pmu->max_pebs_events = min_t(unsigned, MAX_PEBS_EVENTS, pmu->num_counters);
>>   		pmu->unconstrained = (struct event_constraint)
>>   					__EVENT_CONSTRAINT(0, (1ULL << pmu->num_counters) - 1,
>> -- 
>> 2.7.4
>>

[tip: perf/urgent] perf/x86/intel: Add a quirk for the calculation of the number of counters on Alder Lake

[tip-bot2 for Kan Liang]

The following commit has been merged into the perf/urgent branch of tip:

Commit-ID:     7fa981cad216e9f64f49e22112f610c0bfed91bc
Gitweb:        https://git.kernel.org/tip/7fa981cad216e9f64f49e22112f610c0bfed91bc
Author:        Kan Liang <kan.liang@linux.intel.com>
AuthorDate:    Tue, 11 Jan 2022 10:20:38 -08:00
Committer:     Peter Zijlstra <peterz@infradead.org>
CommitterDate: Tue, 18 Jan 2022 12:09:47 +01:00

perf/x86/intel: Add a quirk for the calculation of the number of counters on Alder Lake

For some Alder Lake machine with all E-cores disabled in a BIOS, the
below warning may be triggered.

[ 2.010766] hw perf events fixed 5 > max(4), clipping!

Current perf code relies on the CPUID leaf 0xA and leaf 7.EDX[15] to
calculate the number of the counters and follow the below assumption.

For a hybrid configuration, the leaf 7.EDX[15] (X86_FEATURE_HYBRID_CPU)
is set. The leaf 0xA only enumerate the common counters. Linux perf has
to manually add the extra GP counters and fixed counters for P-cores.
For a non-hybrid configuration, the X86_FEATURE_HYBRID_CPU should not
be set. The leaf 0xA enumerates all counters.

However, that's not the case when all E-cores are disabled in a BIOS.
Although there are only P-cores in the system, the leaf 7.EDX[15]
(X86_FEATURE_HYBRID_CPU) is still set. But the leaf 0xA is updated
to enumerate all counters of P-cores. The inconsistency triggers the
warning.

Several software ways were considered to handle the inconsistency.
- Drop the leaf 0xA and leaf 7.EDX[15] CPUID enumeration support.
  Hardcode the number of counters. This solution may be a problem for
  virtualization. A hypervisor cannot control the number of counters
  in a Linux guest via changing the guest CPUID enumeration anymore.
- Find another CPUID bit that is also updated with E-cores disabled.
  There may be a problem in the virtualization environment too. Because
  a hypervisor may disable the feature/CPUID bit.
- The P-cores have a maximum of 8 GP counters and 4 fixed counters on
  ADL. The maximum number can be used to detect the case.
  This solution is implemented in this patch.

Fixes: ee72a94ea4a6 ("perf/x86/intel: Fix fixed counter check warning for some Alder Lake")
Reported-by: Damjan Marion (damarion) <damarion@cisco.com>
Reported-by: Chan Edison <edison_chan_gz@hotmail.com>
Signed-off-by: Kan Liang <kan.liang@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Damjan Marion (damarion) <damarion@cisco.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/1641925238-149288-1-git-send-email-kan.liang@linux.intel.com
---
 arch/x86/events/intel/core.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c
index fd9f908..d5f940c 100644
--- a/arch/x86/events/intel/core.c
+++ b/arch/x86/events/intel/core.c
@@ -6236,6 +6236,19 @@ __init int intel_pmu_init(void)
 			pmu->num_counters = x86_pmu.num_counters;
 			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
 		}
+
+		/*
+		 * Quirk: For some Alder Lake machine, when all E-cores are disabled in
+		 * a BIOS, the leaf 0xA will enumerate all counters of P-cores. However,
+		 * the X86_FEATURE_HYBRID_CPU is still set. The above codes will
+		 * mistakenly add extra counters for P-cores. Correct the number of
+		 * counters here.
+		 */
+		if ((pmu->num_counters > 8) || (pmu->num_counters_fixed > 4)) {
+			pmu->num_counters = x86_pmu.num_counters;
+			pmu->num_counters_fixed = x86_pmu.num_counters_fixed;
+		}
+
 		pmu->max_pebs_events = min_t(unsigned, MAX_PEBS_EVENTS, pmu->num_counters);
 		pmu->unconstrained = (struct event_constraint)
 					__EVENT_CONSTRAINT(0, (1ULL << pmu->num_counters) - 1,