linux-pm.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v4 0/4] Sapphire Rapids C0.x idle states support
@ 2023-07-10  9:30 Artem Bityutskiy
  2023-07-10  9:30 ` [PATCH v4 1/4] x86/umwait: use 'IS_ENABLED()' Artem Bityutskiy
                   ` (5 more replies)
  0 siblings, 6 replies; 17+ messages in thread
From: Artem Bityutskiy @ 2023-07-10  9:30 UTC (permalink / raw)
  To: x86, Rafael J. Wysocki
  Cc: Linux PM Mailing List, Arjan van de Ven, Artem Bityutskiy,
	Thomas Gleixner

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

Background
----------

Idle states reduce power consumption when a CPU has no work to do. The most
shallow CPU idle state is "POLL". It has lowest wake up latency, but saves
little power. The next idle state on Intel platforms is "C1". It has has higher
latency, but saves more power than "POLL".

Sapphire Rapids Xeons add new C0.1 and C0.2 (later C0.x) idle states which
conceptually sit between "POLL" and "C1". These provide a very attractive
midpoint: near-POLL wake-up latency and power consumption halfway between
"POLL" and "C1".

In other words, the expectation is that most latency-sensitive users will
prefer C0.x over POLL.

Enable C0.2 idle state support on Sapphire Rapids Xeon (later - SPR) by adding
it between POLL and C1.

Base commit
-----------

Based on the "linux-next" branch of "linux-pm" git tree.

base-commit: bd9bb08847da3b1eba2ea8cebf514d9287e7f4fb

Changelog
---------

* v4:
  - Address issues pointed out by Thomas Gleixner.
    . mwait.h: use 'IS_ENABLED()' instead of '#ifdef'.
    . mwait.h: use '__always_inline'.
    . mwait.h: use inline stub instead for macro for "!CONFIG_X86_64" case.
    . mwait.h: use proper commentaries for '#endif' and '#else'.
    . mwait.h: tested with llvm/clang.
    . Use imperative form (removed "this patch").
  - intel_idle: rename 'intel_idle_hlt_irq_on()' for consistency.
* v3
  - Dropped patch 'x86/umwait: Increase tpause and umwait quanta' after, as
    suggested by Andy Lutomirski.
  - Followed Peter Zijlstra's suggestion and removed explicit 'umwait'
    deadline. Rely on the global implicit deadline instead.
  - Rebased on top of Arjan's patches.
  - C0.2 was tested in a VM by Arjan van de Ven.
  - Re-measured on 2S and 4S Sapphire Rapids Xeon.
* v2
  - Do not mix 'raw_local_irq_enable()' and 'local_irq_disable()'. I failed to
    directly verify it, but I believe it'll address the '.noinstr.text' warning.
  - Minor kerneldoc commentary fix.

C0.2 vs POLL latency and power
------------------------------

I compared POLL to C0.2 using 'wult' tool (https://github.com/intel/wult),
which measures idle states latency.

* In "POLL" experiments, all C-states except for POLL were disabled.
* In "C0.2" experiments, all C-states except for POLL and C0.2 were disabled.

Here are the measurement results. The numbers are percent change from POLL to
C0.2.

-----------|-----------|----------|-----------
 Median IR | 99th % IR | AC Power | RAPL power
-----------|-----------|----------|-----------
 24%       | 12%       | -13%     | -18%
-----------------------|----------|-----------

* IR stands for interrupt latency. The table provides the median and 99th
  percentile. Wult measures it as the delay between the moment a timer
  interrupt fires to the moment the CPU reaches the interrupt handler.
* AC Power is the wall socket AC power.
* RAPL power is the CPU package power, measured using the 'turbostat' tool.

Hackbench measurements
----------------------

I ran the 'hackbench' benchmark using the following commands:

# 4 groups, 200 threads
hackbench -s 128 -l 100000000 -g4 -f 25 -P
# 8 groups, 400 threads.
hackbench -s 128 -l 100000000 -g8 -f 25 -P

My SPR system has 224 CPUs, so the first command did not use all CPUs, the
second command used all of them. However, in both cases CPU power reached TDP.

I ran hackbench 5 times for every configuration and compared hackbench "score"
averages.

In case of 4 groups, C0.2 improved the score by about 4%, and in case of 8
groups by about 0.6%.

Q&A
---

1. Can C0.2 be disabled?

C0.2 can be disabled via sysfs and with the following kernel boot option:

  intel_idle.states_off=2

2. Why C0.2, not C0.1?

I measured both C0.1 and C0.2, but did not notice a clear C0.1 advantage in
terms of latency, but did notice that C0.2 saves more power.

But if users want to try using C0.1 instead of C0.2, they can do this:

echo 0 > /sys/devices/system/cpu/umwait_control/enable_c02

This will make sure that C0.2 requests from 'intel_idle' are automatically
converted to C0.1 requests.

3. How did you verify that system enters C0.2?

I used 'perf' to read the corresponding PMU counters:

perf stat -e CPU_CLK_UNHALTED.C01,CPU_CLK_UNHALTED.C02,cycles -a sleep 1

4. Ho to change the global explicit 'umwait' deadline?

Via '/sys/devices/system/cpu/umwait_control/max_time'

Artem Bityutskiy (4):
  x86/umwait: use 'IS_ENABLED()'
  x86/mwait: Add support for idle via umwait
  intel_idle: rename 'intel_idle_hlt_irq_on()'
  intel_idle: add C0.2 state for Sapphire Rapids Xeon

 arch/x86/include/asm/mwait.h | 85 ++++++++++++++++++++++++++++++++----
 drivers/idle/intel_idle.c    | 52 +++++++++++++++++++---
 2 files changed, 123 insertions(+), 14 deletions(-)

-- 
2.40.1


^ permalink raw reply	[flat|nested] 17+ messages in thread

* [PATCH v4 1/4] x86/umwait: use 'IS_ENABLED()'
  2023-07-10  9:30 [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Artem Bityutskiy
@ 2023-07-10  9:30 ` Artem Bityutskiy
  2023-07-10  9:30 ` [PATCH v4 2/4] x86/mwait: Add support for idle via umwait Artem Bityutskiy
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 17+ messages in thread
From: Artem Bityutskiy @ 2023-07-10  9:30 UTC (permalink / raw)
  To: x86, Rafael J. Wysocki
  Cc: Linux PM Mailing List, Arjan van de Ven, Artem Bityutskiy,
	Thomas Gleixner

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

Both kernel coding style and x86 maintainers prefer using 'IS_ENABLED()'
instead of '#ifdef' whenever possible.

Switch '__tpause()' to 'IS_ENABLED()'.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 arch/x86/include/asm/mwait.h | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.h
index 778df05f8539..03bef2bc28d4 100644
--- a/arch/x86/include/asm/mwait.h
+++ b/arch/x86/include/asm/mwait.h
@@ -130,15 +130,15 @@ static __always_inline void mwait_idle_with_hints(unsigned long eax, unsigned lo
 static inline void __tpause(u32 ecx, u32 edx, u32 eax)
 {
 	/* "tpause %ecx, %edx, %eax;" */
-	#ifdef CONFIG_AS_TPAUSE
-	asm volatile("tpause %%ecx\n"
-		     :
-		     : "c"(ecx), "d"(edx), "a"(eax));
-	#else
-	asm volatile(".byte 0x66, 0x0f, 0xae, 0xf1\t\n"
-		     :
-		     : "c"(ecx), "d"(edx), "a"(eax));
-	#endif
+	if (IS_ENABLED(CONFIG_AS_TPAUSE)) {
+		asm volatile("tpause %%ecx\n"
+			     :
+			     : "c"(ecx), "d"(edx), "a"(eax));
+	} else {
+		asm volatile(".byte 0x66, 0x0f, 0xae, 0xf1\t\n"
+			     :
+			     : "c"(ecx), "d"(edx), "a"(eax));
+	}
 }
 
 #endif /* _ASM_X86_MWAIT_H */
-- 
2.40.1


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v4 2/4] x86/mwait: Add support for idle via umwait
  2023-07-10  9:30 [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Artem Bityutskiy
  2023-07-10  9:30 ` [PATCH v4 1/4] x86/umwait: use 'IS_ENABLED()' Artem Bityutskiy
@ 2023-07-10  9:30 ` Artem Bityutskiy
  2023-07-10  9:30 ` [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()' Artem Bityutskiy
                   ` (3 subsequent siblings)
  5 siblings, 0 replies; 17+ messages in thread
From: Artem Bityutskiy @ 2023-07-10  9:30 UTC (permalink / raw)
  To: x86, Rafael J. Wysocki
  Cc: Linux PM Mailing List, Arjan van de Ven, Artem Bityutskiy,
	Thomas Gleixner

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

On Intel platforms, C-states are requested using the 'monitor/mwait'
instructions pair, as implemented in 'mwait_idle_with_hints()'. This
mechanism allows for entering C1 and deeper C-states.

Sapphire Rapids Xeon supports new idle states - C0.1 and C0.2 (later
C0.x). These idle states have lower latency comparing to C1, and can be
requested with either 'tpause' or 'umwait' instructions.

Linux kernel already supports the 'tpause' instruction and uses it in
delay functions like 'udelay()'. Add 'umwait' support by implementing
the 'umwait_idle()' function. This function is analogous to
'mwait_idle_with_hints()', but instead of requesting a C-state with
'monitor/mwait', it requests C0.x with 'umonitor/umwait'.

Tested with both gcc/binutils and clang/llvm.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 arch/x86/include/asm/mwait.h | 67 ++++++++++++++++++++++++++++++++++++
 1 file changed, 67 insertions(+)

diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.h
index 03bef2bc28d4..48210f4d7c77 100644
--- a/arch/x86/include/asm/mwait.h
+++ b/arch/x86/include/asm/mwait.h
@@ -141,4 +141,71 @@ static inline void __tpause(u32 ecx, u32 edx, u32 eax)
 	}
 }
 
+#ifdef CONFIG_X86_64
+/*
+ * Monitor a memory address at 'rcx' using the 'umonitor' instruction.
+ */
+static __always_inline void __umonitor(const void *rcx)
+{
+	/* "umonitor %rcx" */
+	if (IS_ENABLED(CONFIG_AS_TPAUSE)) {
+		asm volatile("umonitor %%rcx\n"
+			     :
+			     : "c"(rcx));
+	} else {
+		asm volatile(".byte 0xf3, 0x0f, 0xae, 0xf1\t\n"
+			     :
+			     : "c"(rcx));
+	}
+}
+
+/*
+ * Same as '__tpause()', but uses the 'umwait' instruction. It is very
+ * similar to 'tpause', but also breaks out if the data at the address
+ * monitored with 'umonitor' is modified.
+ */
+static __always_inline void __umwait(u32 ecx, u32 edx, u32 eax)
+{
+	/* "umwait %ecx, %edx, %eax;" */
+	if (IS_ENABLED(CONFIG_AS_TPAUSE)) {
+		asm volatile("umwait %%ecx\n"
+			     :
+			     : "c"(ecx), "d"(edx), "a"(eax));
+	} else {
+		asm volatile(".byte 0xf2, 0x0f, 0xae, 0xf1\t\n"
+			     :
+			     : "c"(ecx), "d"(edx), "a"(eax));
+	}
+}
+
+/*
+ * Enter C0.1 or C0.2 state and stay there until an event happens (an interrupt
+ * or 'need_resched()'), the explicit deadline is reached, or the implicit
+ * global limit is reached.
+ *
+ * The deadline is the absolute TSC value to exit the idle state at. If it
+ * exceeds the global limit in the 'IA32_UMWAIT_CONTROL' register, the global
+ * limit prevails, and the idle state is exited earlier than the deadline.
+ */
+static __always_inline void umwait_idle(u64 deadline, u32 state)
+{
+	if (!current_set_polling_and_test()) {
+		u32 eax, edx;
+
+		eax = lower_32_bits(deadline);
+		edx = upper_32_bits(deadline);
+
+		__umonitor(&current_thread_info()->flags);
+		if (!need_resched())
+			__umwait(state, edx, eax);
+	}
+	current_clr_polling();
+}
+#else /* CONFIG_X86_64 */
+static __always_inline void umwait_idle(u64 deadline, u32 state)
+{
+	WARN_ONCE(1, "umwait CPU instruction is not supported");
+}
+#endif /* !CONFIG_X86_64 */
+
 #endif /* _ASM_X86_MWAIT_H */
-- 
2.40.1


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()'
  2023-07-10  9:30 [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Artem Bityutskiy
  2023-07-10  9:30 ` [PATCH v4 1/4] x86/umwait: use 'IS_ENABLED()' Artem Bityutskiy
  2023-07-10  9:30 ` [PATCH v4 2/4] x86/mwait: Add support for idle via umwait Artem Bityutskiy
@ 2023-07-10  9:30 ` Artem Bityutskiy
  2023-07-14 15:34   ` Rafael J. Wysocki
  2023-07-10  9:31 ` [PATCH v4 4/4] intel_idle: add C0.2 state for Sapphire Rapids Xeon Artem Bityutskiy
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 17+ messages in thread
From: Artem Bityutskiy @ 2023-07-10  9:30 UTC (permalink / raw)
  To: x86, Rafael J. Wysocki
  Cc: Linux PM Mailing List, Arjan van de Ven, Artem Bityutskiy,
	Thomas Gleixner

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

Rename 'intel_idle_hlt_irq_on()' to 'intel_idle_hlt_irq()' for consistency with
'intel_idle_irq()'.

While on it, fix indentation in 'intel_idle_hlt_irq()' declaration to use tabs
instead of white-spaces.

Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 drivers/idle/intel_idle.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c
index b930036edbbe..0a835f97de72 100644
--- a/drivers/idle/intel_idle.c
+++ b/drivers/idle/intel_idle.c
@@ -224,8 +224,8 @@ static __cpuidle int intel_idle_hlt(struct cpuidle_device *dev,
 	return __intel_idle_hlt(dev, drv, index);
 }
 
-static __cpuidle int intel_idle_hlt_irq_on(struct cpuidle_device *dev,
-                                   struct cpuidle_driver *drv, int index)
+static __cpuidle int intel_idle_hlt_irq(struct cpuidle_device *dev,
+					struct cpuidle_driver *drv, int index)
 {
        int ret;
 
@@ -1900,11 +1900,11 @@ static void state_update_enter_method(struct cpuidle_state *state, int cstate)
 	if (state->enter == intel_idle_hlt) {
 		if (force_irq_on) {
 			pr_info("forced intel_idle_irq for state %d\n", cstate);
-			state->enter = intel_idle_hlt_irq_on;
+			state->enter = intel_idle_hlt_irq;
 		}
 		return;
 	}
-	if (state->enter == intel_idle_hlt_irq_on)
+	if (state->enter == intel_idle_hlt_irq)
 		return; /* no update scenarios */
 
 	if (state->flags & CPUIDLE_FLAG_INIT_XSTATE) {
@@ -1949,7 +1949,7 @@ static bool should_verify_mwait(struct cpuidle_state *state)
 {
 	if (state->enter == intel_idle_hlt)
 		return false;
-	if (state->enter == intel_idle_hlt_irq_on)
+	if (state->enter == intel_idle_hlt_irq)
 		return false;
 
 	return true;
-- 
2.40.1


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* [PATCH v4 4/4] intel_idle: add C0.2 state for Sapphire Rapids Xeon
  2023-07-10  9:30 [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Artem Bityutskiy
                   ` (2 preceding siblings ...)
  2023-07-10  9:30 ` [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()' Artem Bityutskiy
@ 2023-07-10  9:31 ` Artem Bityutskiy
  2023-07-20 18:35 ` [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Rafael J. Wysocki
  2023-08-28 16:43 ` Artem Bityutskiy
  5 siblings, 0 replies; 17+ messages in thread
From: Artem Bityutskiy @ 2023-07-10  9:31 UTC (permalink / raw)
  To: x86, Rafael J. Wysocki
  Cc: Linux PM Mailing List, Arjan van de Ven, Artem Bityutskiy,
	Thomas Gleixner

From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

Add Sapphire Rapids Xeon C0.2 state support. This state has a lower exit
latency comparing to C1, and saves energy comparing to POLL.

C0.2 may also improve performance (e.g., as measured by 'hackbench'), because
idle CPU power savings in C0.2 increase busy CPU power budget and therefore,
improve turbo boost of the busy CPU.

Suggested-by: Len Brown <len.brown@intel.com>
Suggested-by: Arjan Van De Ven <arjan.van.de.ven@intel.com>
Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
---
 drivers/idle/intel_idle.c | 44 ++++++++++++++++++++++++++++++++++++++-
 1 file changed, 43 insertions(+), 1 deletion(-)

diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c
index 0a835f97de72..eb2bcc7f8ea0 100644
--- a/drivers/idle/intel_idle.c
+++ b/drivers/idle/intel_idle.c
@@ -130,6 +130,11 @@ static unsigned int mwait_substates __initdata;
 #define flg2MWAIT(flags) (((flags) >> 24) & 0xFF)
 #define MWAIT2flg(eax) ((eax & 0xFF) << 24)
 
+/*
+ * The maximum possible 'umwait' deadline value.
+ */
+#define UMWAIT_MAX_DEADLINE (~((u64)0))
+
 static __always_inline int __intel_idle(struct cpuidle_device *dev,
 					struct cpuidle_driver *drv, int index)
 {
@@ -263,6 +268,32 @@ static __cpuidle int intel_idle_s2idle(struct cpuidle_device *dev,
 	return 0;
 }
 
+/**
+ * intel_idle_umwait_irq - Request C0.x using the 'umwait' instruction.
+ * @dev: cpuidle device of the target CPU.
+ * @drv: cpuidle driver (assumed to point to intel_idle_driver).
+ * @index: Target idle state index.
+ *
+ * Request C0.1 or C0.2 using 'umwait' instruction with interrupts enabled.
+ */
+static __cpuidle int intel_idle_umwait_irq(struct cpuidle_device *dev,
+					   struct cpuidle_driver *drv,
+					   int index)
+{
+	u32 state = flg2MWAIT(drv->states[index].flags);
+
+	raw_local_irq_enable();
+	/*
+	 * Use the maximum possible deadline value. This means that 'C0.x'
+	 * residency will be limited by the global limit in
+	 * 'IA32_UMWAIT_CONTROL'.
+	 */
+	umwait_idle(UMWAIT_MAX_DEADLINE, state);
+	raw_local_irq_disable();
+
+	return index;
+}
+
 /*
  * States are indexed by the cstate number,
  * which is also the index into the MWAIT hint array.
@@ -1006,6 +1037,13 @@ static struct cpuidle_state adl_n_cstates[] __initdata = {
 };
 
 static struct cpuidle_state spr_cstates[] __initdata = {
+	{
+		.name = "C0.2",
+		.desc = "UMWAIT C0.2",
+		.flags = MWAIT2flg(TPAUSE_C02_STATE) | CPUIDLE_FLAG_IRQ_ENABLE,
+		.exit_latency_ns = 200,
+		.target_residency_ns = 200,
+		.enter = &intel_idle_umwait_irq, },
 	{
 		.name = "C1",
 		.desc = "MWAIT 0x00",
@@ -1904,7 +1942,9 @@ static void state_update_enter_method(struct cpuidle_state *state, int cstate)
 		}
 		return;
 	}
-	if (state->enter == intel_idle_hlt_irq)
+
+	if (state->enter == intel_idle_hlt_irq ||
+	    state->enter == intel_idle_umwait_irq)
 		return; /* no update scenarios */
 
 	if (state->flags & CPUIDLE_FLAG_INIT_XSTATE) {
@@ -1951,6 +1991,8 @@ static bool should_verify_mwait(struct cpuidle_state *state)
 		return false;
 	if (state->enter == intel_idle_hlt_irq)
 		return false;
+	if (state->enter == intel_idle_umwait_irq)
+		return false;
 
 	return true;
 }
-- 
2.40.1


^ permalink raw reply related	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()'
  2023-07-10  9:30 ` [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()' Artem Bityutskiy
@ 2023-07-14 15:34   ` Rafael J. Wysocki
  2023-07-14 15:39     ` Arjan van de Ven
                       ` (2 more replies)
  0 siblings, 3 replies; 17+ messages in thread
From: Rafael J. Wysocki @ 2023-07-14 15:34 UTC (permalink / raw)
  To: Artem Bityutskiy
  Cc: x86, Rafael J. Wysocki, Linux PM Mailing List, Arjan van de Ven,
	Thomas Gleixner

On Mon, Jul 10, 2023 at 11:31 AM Artem Bityutskiy <dedekind1@gmail.com> wrote:
>
> From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
>
> Rename 'intel_idle_hlt_irq_on()' to 'intel_idle_hlt_irq()' for consistency with
> 'intel_idle_irq()'.
>
> While on it, fix indentation in 'intel_idle_hlt_irq()' declaration to use tabs
> instead of white-spaces.
>
> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>

I think I can apply this right away without the rest of the series as
it doesn't seem to depend on the other patches.

Any concerns regarding this?

> ---
>  drivers/idle/intel_idle.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/idle/intel_idle.c b/drivers/idle/intel_idle.c
> index b930036edbbe..0a835f97de72 100644
> --- a/drivers/idle/intel_idle.c
> +++ b/drivers/idle/intel_idle.c
> @@ -224,8 +224,8 @@ static __cpuidle int intel_idle_hlt(struct cpuidle_device *dev,
>         return __intel_idle_hlt(dev, drv, index);
>  }
>
> -static __cpuidle int intel_idle_hlt_irq_on(struct cpuidle_device *dev,
> -                                   struct cpuidle_driver *drv, int index)
> +static __cpuidle int intel_idle_hlt_irq(struct cpuidle_device *dev,
> +                                       struct cpuidle_driver *drv, int index)
>  {
>         int ret;
>
> @@ -1900,11 +1900,11 @@ static void state_update_enter_method(struct cpuidle_state *state, int cstate)
>         if (state->enter == intel_idle_hlt) {
>                 if (force_irq_on) {
>                         pr_info("forced intel_idle_irq for state %d\n", cstate);
> -                       state->enter = intel_idle_hlt_irq_on;
> +                       state->enter = intel_idle_hlt_irq;
>                 }
>                 return;
>         }
> -       if (state->enter == intel_idle_hlt_irq_on)
> +       if (state->enter == intel_idle_hlt_irq)
>                 return; /* no update scenarios */
>
>         if (state->flags & CPUIDLE_FLAG_INIT_XSTATE) {
> @@ -1949,7 +1949,7 @@ static bool should_verify_mwait(struct cpuidle_state *state)
>  {
>         if (state->enter == intel_idle_hlt)
>                 return false;
> -       if (state->enter == intel_idle_hlt_irq_on)
> +       if (state->enter == intel_idle_hlt_irq)
>                 return false;
>
>         return true;
> --
> 2.40.1
>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()'
  2023-07-14 15:34   ` Rafael J. Wysocki
@ 2023-07-14 15:39     ` Arjan van de Ven
  2023-07-14 18:11     ` Artem Bityutskiy
  2023-07-14 21:01     ` Peter Zijlstra
  2 siblings, 0 replies; 17+ messages in thread
From: Arjan van de Ven @ 2023-07-14 15:39 UTC (permalink / raw)
  To: Rafael J. Wysocki, Artem Bityutskiy
  Cc: x86, Linux PM Mailing List, Thomas Gleixner

On 7/14/2023 8:34 AM, Rafael J. Wysocki wrote:
> On Mon, Jul 10, 2023 at 11:31 AM Artem Bityutskiy <dedekind1@gmail.com> wrote:
>>
>> From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
>>
>> Rename 'intel_idle_hlt_irq_on()' to 'intel_idle_hlt_irq()' for consistency with
>> 'intel_idle_irq()'.
>>
>> While on it, fix indentation in 'intel_idle_hlt_irq()' declaration to use tabs
>> instead of white-spaces.
>>
>> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> 
> I think I can apply this right away without the rest of the series as
> it doesn't seem to depend on the other patches.
> 
> Any concerns regarding this?
> 

no other than that I will likely delete some of the code in a next patch but that's ok



^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()'
  2023-07-14 15:34   ` Rafael J. Wysocki
  2023-07-14 15:39     ` Arjan van de Ven
@ 2023-07-14 18:11     ` Artem Bityutskiy
  2023-07-14 21:01     ` Peter Zijlstra
  2 siblings, 0 replies; 17+ messages in thread
From: Artem Bityutskiy @ 2023-07-14 18:11 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: x86, Linux PM Mailing List, Arjan van de Ven, Thomas Gleixner

On Fri, 2023-07-14 at 17:34 +0200, Rafael J. Wysocki wrote:
> Any concerns regarding this?

No concerns, thanks!

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()'
  2023-07-14 15:34   ` Rafael J. Wysocki
  2023-07-14 15:39     ` Arjan van de Ven
  2023-07-14 18:11     ` Artem Bityutskiy
@ 2023-07-14 21:01     ` Peter Zijlstra
  2023-07-14 21:02       ` Arjan van de Ven
  2 siblings, 1 reply; 17+ messages in thread
From: Peter Zijlstra @ 2023-07-14 21:01 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: Artem Bityutskiy, x86, Linux PM Mailing List, Arjan van de Ven,
	Thomas Gleixner

On Fri, Jul 14, 2023 at 05:34:43PM +0200, Rafael J. Wysocki wrote:
> On Mon, Jul 10, 2023 at 11:31 AM Artem Bityutskiy <dedekind1@gmail.com> wrote:
> >
> > From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> >
> > Rename 'intel_idle_hlt_irq_on()' to 'intel_idle_hlt_irq()' for consistency with
> > 'intel_idle_irq()'.
> >
> > While on it, fix indentation in 'intel_idle_hlt_irq()' declaration to use tabs
> > instead of white-spaces.
> >
> > Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> 
> I think I can apply this right away without the rest of the series as
> it doesn't seem to depend on the other patches.
> 
> Any concerns regarding this?

Yeah, as per:

  https://lkml.kernel.org/r/20230711132553.GN3062772%40hirez.programming.kicks-ass.net

that function is complete drivel, renaming it isn't going to help.

In fact, I don't buy the whole premise of that intel_idle guest mode
stuff, can we please just delete all of that and start over?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()'
  2023-07-14 21:01     ` Peter Zijlstra
@ 2023-07-14 21:02       ` Arjan van de Ven
  2023-07-14 21:12         ` Peter Zijlstra
  0 siblings, 1 reply; 17+ messages in thread
From: Arjan van de Ven @ 2023-07-14 21:02 UTC (permalink / raw)
  To: Peter Zijlstra, Rafael J. Wysocki
  Cc: Artem Bityutskiy, x86, Linux PM Mailing List, Thomas Gleixner

On 7/14/2023 2:01 PM, Peter Zijlstra wrote:
> On Fri, Jul 14, 2023 at 05:34:43PM +0200, Rafael J. Wysocki wrote:
>> On Mon, Jul 10, 2023 at 11:31 AM Artem Bityutskiy <dedekind1@gmail.com> wrote:
>>>
>>> From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
>>>
>>> Rename 'intel_idle_hlt_irq_on()' to 'intel_idle_hlt_irq()' for consistency with
>>> 'intel_idle_irq()'.
>>>
>>> While on it, fix indentation in 'intel_idle_hlt_irq()' declaration to use tabs
>>> instead of white-spaces.
>>>
>>> Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
>>
>> I think I can apply this right away without the rest of the series as
>> it doesn't seem to depend on the other patches.
>>
>> Any concerns regarding this?
> 
> Yeah, as per:
> 
>    https://lkml.kernel.org/r/20230711132553.GN3062772%40hirez.programming.kicks-ass.net

and it's going to get deleted - renaming is harmless for that


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()'
  2023-07-14 21:02       ` Arjan van de Ven
@ 2023-07-14 21:12         ` Peter Zijlstra
  0 siblings, 0 replies; 17+ messages in thread
From: Peter Zijlstra @ 2023-07-14 21:12 UTC (permalink / raw)
  To: Arjan van de Ven
  Cc: Rafael J. Wysocki, Artem Bityutskiy, x86, Linux PM Mailing List,
	Thomas Gleixner

On Fri, Jul 14, 2023 at 02:02:51PM -0700, Arjan van de Ven wrote:
> On 7/14/2023 2:01 PM, Peter Zijlstra wrote:
> > On Fri, Jul 14, 2023 at 05:34:43PM +0200, Rafael J. Wysocki wrote:
> > > On Mon, Jul 10, 2023 at 11:31 AM Artem Bityutskiy <dedekind1@gmail.com> wrote:
> > > > 
> > > > From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> > > > 
> > > > Rename 'intel_idle_hlt_irq_on()' to 'intel_idle_hlt_irq()' for consistency with
> > > > 'intel_idle_irq()'.
> > > > 
> > > > While on it, fix indentation in 'intel_idle_hlt_irq()' declaration to use tabs
> > > > instead of white-spaces.
> > > > 
> > > > Signed-off-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
> > > 
> > > I think I can apply this right away without the rest of the series as
> > > it doesn't seem to depend on the other patches.
> > > 
> > > Any concerns regarding this?
> > 
> > Yeah, as per:
> > 
> >    https://lkml.kernel.org/r/20230711132553.GN3062772%40hirez.programming.kicks-ass.net
> 
> and it's going to get deleted - renaming is harmless for that

Yeah, but what's the point, why waste more commits on this?

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 0/4] Sapphire Rapids C0.x idle states support
  2023-07-10  9:30 [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Artem Bityutskiy
                   ` (3 preceding siblings ...)
  2023-07-10  9:31 ` [PATCH v4 4/4] intel_idle: add C0.2 state for Sapphire Rapids Xeon Artem Bityutskiy
@ 2023-07-20 18:35 ` Rafael J. Wysocki
  2023-08-28 16:43 ` Artem Bityutskiy
  5 siblings, 0 replies; 17+ messages in thread
From: Rafael J. Wysocki @ 2023-07-20 18:35 UTC (permalink / raw)
  To: Artem Bityutskiy, x86, Thomas Gleixner
  Cc: Rafael J. Wysocki, Linux PM Mailing List, Arjan van de Ven

On Mon, Jul 10, 2023 at 11:31 AM Artem Bityutskiy <dedekind1@gmail.com> wrote:
>
> From: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
>
> Background
> ----------
>
> Idle states reduce power consumption when a CPU has no work to do. The most
> shallow CPU idle state is "POLL". It has lowest wake up latency, but saves
> little power. The next idle state on Intel platforms is "C1". It has has higher
> latency, but saves more power than "POLL".
>
> Sapphire Rapids Xeons add new C0.1 and C0.2 (later C0.x) idle states which
> conceptually sit between "POLL" and "C1". These provide a very attractive
> midpoint: near-POLL wake-up latency and power consumption halfway between
> "POLL" and "C1".
>
> In other words, the expectation is that most latency-sensitive users will
> prefer C0.x over POLL.
>
> Enable C0.2 idle state support on Sapphire Rapids Xeon (later - SPR) by adding
> it between POLL and C1.
>
> Base commit
> -----------
>
> Based on the "linux-next" branch of "linux-pm" git tree.
>
> base-commit: bd9bb08847da3b1eba2ea8cebf514d9287e7f4fb
>
> Changelog
> ---------
>
> * v4:
>   - Address issues pointed out by Thomas Gleixner.
>     . mwait.h: use 'IS_ENABLED()' instead of '#ifdef'.
>     . mwait.h: use '__always_inline'.
>     . mwait.h: use inline stub instead for macro for "!CONFIG_X86_64" case.
>     . mwait.h: use proper commentaries for '#endif' and '#else'.
>     . mwait.h: tested with llvm/clang.
>     . Use imperative form (removed "this patch").
>   - intel_idle: rename 'intel_idle_hlt_irq_on()' for consistency.

I'm wondering if the v4 looks better than the previous versions to the
x86 folks (and Thomas in particular).

Note that patch [3/4] will not be needed any more as it refines a
commit that is going to be reverted.

> * v3
>   - Dropped patch 'x86/umwait: Increase tpause and umwait quanta' after, as
>     suggested by Andy Lutomirski.
>   - Followed Peter Zijlstra's suggestion and removed explicit 'umwait'
>     deadline. Rely on the global implicit deadline instead.
>   - Rebased on top of Arjan's patches.
>   - C0.2 was tested in a VM by Arjan van de Ven.
>   - Re-measured on 2S and 4S Sapphire Rapids Xeon.
> * v2
>   - Do not mix 'raw_local_irq_enable()' and 'local_irq_disable()'. I failed to
>     directly verify it, but I believe it'll address the '.noinstr.text' warning.
>   - Minor kerneldoc commentary fix.
>
> C0.2 vs POLL latency and power
> ------------------------------
>
> I compared POLL to C0.2 using 'wult' tool (https://github.com/intel/wult),
> which measures idle states latency.
>
> * In "POLL" experiments, all C-states except for POLL were disabled.
> * In "C0.2" experiments, all C-states except for POLL and C0.2 were disabled.
>
> Here are the measurement results. The numbers are percent change from POLL to
> C0.2.
>
> -----------|-----------|----------|-----------
>  Median IR | 99th % IR | AC Power | RAPL power
> -----------|-----------|----------|-----------
>  24%       | 12%       | -13%     | -18%
> -----------------------|----------|-----------
>
> * IR stands for interrupt latency. The table provides the median and 99th
>   percentile. Wult measures it as the delay between the moment a timer
>   interrupt fires to the moment the CPU reaches the interrupt handler.
> * AC Power is the wall socket AC power.
> * RAPL power is the CPU package power, measured using the 'turbostat' tool.
>
> Hackbench measurements
> ----------------------
>
> I ran the 'hackbench' benchmark using the following commands:
>
> # 4 groups, 200 threads
> hackbench -s 128 -l 100000000 -g4 -f 25 -P
> # 8 groups, 400 threads.
> hackbench -s 128 -l 100000000 -g8 -f 25 -P
>
> My SPR system has 224 CPUs, so the first command did not use all CPUs, the
> second command used all of them. However, in both cases CPU power reached TDP.
>
> I ran hackbench 5 times for every configuration and compared hackbench "score"
> averages.
>
> In case of 4 groups, C0.2 improved the score by about 4%, and in case of 8
> groups by about 0.6%.
>
> Q&A
> ---
>
> 1. Can C0.2 be disabled?
>
> C0.2 can be disabled via sysfs and with the following kernel boot option:
>
>   intel_idle.states_off=2
>
> 2. Why C0.2, not C0.1?
>
> I measured both C0.1 and C0.2, but did not notice a clear C0.1 advantage in
> terms of latency, but did notice that C0.2 saves more power.
>
> But if users want to try using C0.1 instead of C0.2, they can do this:
>
> echo 0 > /sys/devices/system/cpu/umwait_control/enable_c02
>
> This will make sure that C0.2 requests from 'intel_idle' are automatically
> converted to C0.1 requests.
>
> 3. How did you verify that system enters C0.2?
>
> I used 'perf' to read the corresponding PMU counters:
>
> perf stat -e CPU_CLK_UNHALTED.C01,CPU_CLK_UNHALTED.C02,cycles -a sleep 1
>
> 4. Ho to change the global explicit 'umwait' deadline?
>
> Via '/sys/devices/system/cpu/umwait_control/max_time'
>
> Artem Bityutskiy (4):
>   x86/umwait: use 'IS_ENABLED()'
>   x86/mwait: Add support for idle via umwait
>   intel_idle: rename 'intel_idle_hlt_irq_on()'
>   intel_idle: add C0.2 state for Sapphire Rapids Xeon
>
>  arch/x86/include/asm/mwait.h | 85 ++++++++++++++++++++++++++++++++----
>  drivers/idle/intel_idle.c    | 52 +++++++++++++++++++---
>  2 files changed, 123 insertions(+), 14 deletions(-)
>
> --
> 2.40.1
>

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 0/4] Sapphire Rapids C0.x idle states support
  2023-07-10  9:30 [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Artem Bityutskiy
                   ` (4 preceding siblings ...)
  2023-07-20 18:35 ` [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Rafael J. Wysocki
@ 2023-08-28 16:43 ` Artem Bityutskiy
  2023-09-13 11:37   ` Artem Bityutskiy
  5 siblings, 1 reply; 17+ messages in thread
From: Artem Bityutskiy @ 2023-08-28 16:43 UTC (permalink / raw)
  To: x86, Thomas Gleixner
  Cc: Linux PM Mailing List, Arjan van de Ven, Rafael J. Wysocki

Hi,

On Mon, 2023-07-10 at 12:30 +0300, Artem Bityutskiy wrote:

> Artem Bityutskiy (4):
>   x86/umwait: use 'IS_ENABLED()'
>   x86/mwait: Add support for idle via umwait

If these 2 patches are OK now, is there a chance to get them merged?

>   intel_idle: rename 'intel_idle_hlt_irq_on()'
>   intel_idle: add C0.2 state for Sapphire Rapids Xeon

The intel_idle part requires a refresh, but I thought I'd do that if/when the
x86 bits are merged.

Thanks!


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 0/4] Sapphire Rapids C0.x idle states support
  2023-08-28 16:43 ` Artem Bityutskiy
@ 2023-09-13 11:37   ` Artem Bityutskiy
  2023-09-13 12:34     ` Rafael J. Wysocki
  0 siblings, 1 reply; 17+ messages in thread
From: Artem Bityutskiy @ 2023-09-13 11:37 UTC (permalink / raw)
  To: x86, Thomas Gleixner
  Cc: Linux PM Mailing List, Arjan van de Ven, Rafael J. Wysocki

On Mon, 2023-08-28 at 19:43 +0300, Artem Bityutskiy wrote:
> Hi,
> 
> On Mon, 2023-07-10 at 12:30 +0300, Artem Bityutskiy wrote:
> 
> > Artem Bityutskiy (4):
> >   x86/umwait: use 'IS_ENABLED()'
> >   x86/mwait: Add support for idle via umwait
> 
> If these 2 patches are OK now, is there a chance to get them merged?

Hi, I wonder if these 2 patches have any chances to get merged?

May be there is something I am expected to do have not done? If so, please,
accept my apologies and give me a direction.

Thanks!

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 0/4] Sapphire Rapids C0.x idle states support
  2023-09-13 11:37   ` Artem Bityutskiy
@ 2023-09-13 12:34     ` Rafael J. Wysocki
  2023-09-13 12:49       ` Artem Bityutskiy
  0 siblings, 1 reply; 17+ messages in thread
From: Rafael J. Wysocki @ 2023-09-13 12:34 UTC (permalink / raw)
  To: Artem Bityutskiy
  Cc: x86, Thomas Gleixner, Linux PM Mailing List, Arjan van de Ven,
	Rafael J. Wysocki

On Wed, Sep 13, 2023 at 1:37 PM Artem Bityutskiy <dedekind1@gmail.com> wrote:
>
> On Mon, 2023-08-28 at 19:43 +0300, Artem Bityutskiy wrote:
> > Hi,
> >
> > On Mon, 2023-07-10 at 12:30 +0300, Artem Bityutskiy wrote:
> >
> > > Artem Bityutskiy (4):
> > >   x86/umwait: use 'IS_ENABLED()'
> > >   x86/mwait: Add support for idle via umwait
> >
> > If these 2 patches are OK now, is there a chance to get them merged?
>
> Hi, I wonder if these 2 patches have any chances to get merged?
>
> May be there is something I am expected to do have not done? If so, please,
> accept my apologies and give me a direction.

I think that they were based on the Arjan\s changes that got dropped.
If so, they need to be rebased and resent.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 0/4] Sapphire Rapids C0.x idle states support
  2023-09-13 12:34     ` Rafael J. Wysocki
@ 2023-09-13 12:49       ` Artem Bityutskiy
  2023-09-13 12:55         ` Rafael J. Wysocki
  0 siblings, 1 reply; 17+ messages in thread
From: Artem Bityutskiy @ 2023-09-13 12:49 UTC (permalink / raw)
  To: Rafael J. Wysocki
  Cc: x86, Thomas Gleixner, Linux PM Mailing List, Arjan van de Ven

On Wed, 2023-09-13 at 14:34 +0200, Rafael J. Wysocki wrote:
> On Wed, Sep 13, 2023 at 1:37 PM Artem Bityutskiy <dedekind1@gmail.com> wrote:
> > 
> > On Mon, 2023-08-28 at 19:43 +0300, Artem Bityutskiy wrote:
> > > Hi,
> > > 
> > > On Mon, 2023-07-10 at 12:30 +0300, Artem Bityutskiy wrote:
> > > 
> > > > Artem Bityutskiy (4):
> > > >   x86/umwait: use 'IS_ENABLED()'
> > > >   x86/mwait: Add support for idle via umwait
> > > 
> > > If these 2 patches are OK now, is there a chance to get them merged?
> > 
> > Hi, I wonder if these 2 patches have any chances to get merged?
> > 
> > May be there is something I am expected to do have not done? If so, please,
> > accept my apologies and give me a direction.
> 
> I think that they were based on the Arjan\s changes that got dropped.
> If so, they need to be rebased and resent.

The above quoted 2 patches are x86 bits, they introduce the basic umwait
primitives. They are not based on Arjan's patches. They still apply cleanly, and
do not need a refresh, as far as I can see.

I was hoping to get arch/x86 bits merged. Then I would send the refreshed
version of the intel_idle patches.

Thanks,
Artem.

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [PATCH v4 0/4] Sapphire Rapids C0.x idle states support
  2023-09-13 12:49       ` Artem Bityutskiy
@ 2023-09-13 12:55         ` Rafael J. Wysocki
  0 siblings, 0 replies; 17+ messages in thread
From: Rafael J. Wysocki @ 2023-09-13 12:55 UTC (permalink / raw)
  To: Artem Bityutskiy
  Cc: Rafael J. Wysocki, x86, Thomas Gleixner, Linux PM Mailing List,
	Arjan van de Ven

On Wed, Sep 13, 2023 at 2:49 PM Artem Bityutskiy <dedekind1@gmail.com> wrote:
>
> On Wed, 2023-09-13 at 14:34 +0200, Rafael J. Wysocki wrote:
> > On Wed, Sep 13, 2023 at 1:37 PM Artem Bityutskiy <dedekind1@gmail.com> wrote:
> > >
> > > On Mon, 2023-08-28 at 19:43 +0300, Artem Bityutskiy wrote:
> > > > Hi,
> > > >
> > > > On Mon, 2023-07-10 at 12:30 +0300, Artem Bityutskiy wrote:
> > > >
> > > > > Artem Bityutskiy (4):
> > > > >   x86/umwait: use 'IS_ENABLED()'
> > > > >   x86/mwait: Add support for idle via umwait
> > > >
> > > > If these 2 patches are OK now, is there a chance to get them merged?
> > >
> > > Hi, I wonder if these 2 patches have any chances to get merged?
> > >
> > > May be there is something I am expected to do have not done? If so, please,
> > > accept my apologies and give me a direction.
> >
> > I think that they were based on the Arjan\s changes that got dropped.
> > If so, they need to be rebased and resent.
>
> The above quoted 2 patches are x86 bits, they introduce the basic umwait
> primitives. They are not based on Arjan's patches. They still apply cleanly, and
> do not need a refresh, as far as I can see.
>
> I was hoping to get arch/x86 bits merged.

This is up to the x86 maintainers, then.

> Then I would send the refreshed version of the intel_idle patches.

Well, if the x86 changes are only needed because of the subsequent
intel_idle changes, it may be still better to send a new version of
the whole patchset.

^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2023-09-13 12:55 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-07-10  9:30 [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Artem Bityutskiy
2023-07-10  9:30 ` [PATCH v4 1/4] x86/umwait: use 'IS_ENABLED()' Artem Bityutskiy
2023-07-10  9:30 ` [PATCH v4 2/4] x86/mwait: Add support for idle via umwait Artem Bityutskiy
2023-07-10  9:30 ` [PATCH v4 3/4] intel_idle: rename 'intel_idle_hlt_irq_on()' Artem Bityutskiy
2023-07-14 15:34   ` Rafael J. Wysocki
2023-07-14 15:39     ` Arjan van de Ven
2023-07-14 18:11     ` Artem Bityutskiy
2023-07-14 21:01     ` Peter Zijlstra
2023-07-14 21:02       ` Arjan van de Ven
2023-07-14 21:12         ` Peter Zijlstra
2023-07-10  9:31 ` [PATCH v4 4/4] intel_idle: add C0.2 state for Sapphire Rapids Xeon Artem Bityutskiy
2023-07-20 18:35 ` [PATCH v4 0/4] Sapphire Rapids C0.x idle states support Rafael J. Wysocki
2023-08-28 16:43 ` Artem Bityutskiy
2023-09-13 11:37   ` Artem Bityutskiy
2023-09-13 12:34     ` Rafael J. Wysocki
2023-09-13 12:49       ` Artem Bityutskiy
2023-09-13 12:55         ` Rafael J. Wysocki

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).