[PATCH 1/2] x86/hw_breakpoint: Prevent data breakpoints on __per_cpu_offset

[PATCH 1/2] x86/hw_breakpoint: Prevent data breakpoints on __per_cpu_offset

[Lai Jiangshan]

From: Lai Jiangshan <laijs@linux.alibaba.com>

When FSGSBASE is enabled, paranoid_entry() fetches the per-CPU
GSBASE value via __per_cpu_offset or pcpu_unit_offsets.

When data breakpoint is set on __per_cpu_offset[cpu] (read-write
operation), the specific cpu will be stuck in the infinite #DB loop.
RCU will try to send NMI to the specific cpu, but it is not working
either since NMI also relies on paranoid_entry().

Fixes: eaad981291ee3("x86/entry/64: Introduce the FIND_PERCPU_BASE macro")
Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>
---
 arch/x86/kernel/hw_breakpoint.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index 03aa33b58165..bc7493a0736f 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -269,6 +269,20 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end)
 			CPU_ENTRY_AREA_TOTAL_SIZE))
 		return true;
 
+	/*
+	 * When FSGSBASE is enabled, paranoid_entry() fetches the per-CPU
+	 * GSBASE value via __per_cpu_offset or pcpu_unit_offsets.
+	 */
+#ifdef CONFIG_SMP
+	if (within_area(addr, end, (unsigned long)__per_cpu_offset,
+			sizeof(unsigned long) * nr_cpu_ids))
+		return true;
+#else
+	if (within_area(addr, end, (unsigned long)&pcpu_unit_offsets,
+			sizeof(pcpu_unit_offsets)))
+		return true;
+#endif
+
 	for_each_possible_cpu(cpu) {
 		/* The original rw GDT is being used after load_direct_gdt() */
 		if (within_area(addr, end, (unsigned long)get_cpu_gdt_rw(cpu),
-- 
2.19.1.6.gb485710b

[PATCH 2/2] x86/hw_breakpoint: Prevent data breakpoints on cpu_dr7

[Lai Jiangshan]

From: Lai Jiangshan <laijs@linux.alibaba.com>

When in guest (X86_FEATURE_HYPERVISOR), local_db_save() will read
per-cpu cpu_dr7 before clear dr7 register.

local_db_save() is called at the start of exc_debug_kernel().
To avoid recursive #DB, we have to disallow data breakpoints
on cpu_dr7.

Fixes: 84b6a3491567a("x86/entry: Optimize local_db_save() for virt")
Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>
---
 arch/x86/kernel/hw_breakpoint.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index bc7493a0736f..de34dd49d317 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -307,6 +307,14 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end)
 				(unsigned long)&per_cpu(cpu_tlbstate, cpu),
 				sizeof(struct tlb_state)))
 			return true;
+
+		/*
+		 * When in guest (X86_FEATURE_HYPERVISOR), local_db_save()
+		 * will read per-cpu cpu_dr7 before clear dr7 register.
+		 */
+		if (within_area(addr, end, (unsigned long)&per_cpu(cpu_dr7, cpu),
+				sizeof(cpu_dr7)))
+			return true;
 	}
 
 	return false;
-- 
2.19.1.6.gb485710b

Re: [PATCH 1/2] x86/hw_breakpoint: Prevent data breakpoints on __per_cpu_offset

[Andy Lutomirski]

On Thu, Feb 4, 2021 at 6:26 AM Lai Jiangshan <jiangshanlai@gmail.com> wrote:
>
> From: Lai Jiangshan <laijs@linux.alibaba.com>
>
> When FSGSBASE is enabled, paranoid_entry() fetches the per-CPU
> GSBASE value via __per_cpu_offset or pcpu_unit_offsets.
>
> When data breakpoint is set on __per_cpu_offset[cpu] (read-write
> operation), the specific cpu will be stuck in the infinite #DB loop.
> RCU will try to send NMI to the specific cpu, but it is not working
> either since NMI also relies on paranoid_entry().

Should we consider having a .percpu..noinstr section and having
objtool enforce this?

Re: [PATCH 1/2] x86/hw_breakpoint: Prevent data breakpoints on __per_cpu_offset

[Thomas Gleixner]

On Thu, Feb 04 2021 at 16:11, Andy Lutomirski wrote:
> On Thu, Feb 4, 2021 at 6:26 AM Lai Jiangshan <jiangshanlai@gmail.com> wrote:
>> When FSGSBASE is enabled, paranoid_entry() fetches the per-CPU
>> GSBASE value via __per_cpu_offset or pcpu_unit_offsets.
>>
>> When data breakpoint is set on __per_cpu_offset[cpu] (read-write
>> operation), the specific cpu will be stuck in the infinite #DB loop.
>> RCU will try to send NMI to the specific cpu, but it is not working
>> either since NMI also relies on paranoid_entry().
>
> Should we consider having a .percpu..noinstr section and having
> objtool enforce this?

I think so.

Thanks,

        tglx

Re: [PATCH 1/2] x86/hw_breakpoint: Prevent data breakpoints on __per_cpu_offset

[Peter Zijlstra]

On Fri, Feb 05, 2021 at 12:45:54PM +0100, Thomas Gleixner wrote:
> On Thu, Feb 04 2021 at 16:11, Andy Lutomirski wrote:
> > On Thu, Feb 4, 2021 at 6:26 AM Lai Jiangshan <jiangshanlai@gmail.com> wrote:
> >> When FSGSBASE is enabled, paranoid_entry() fetches the per-CPU
> >> GSBASE value via __per_cpu_offset or pcpu_unit_offsets.
> >>
> >> When data breakpoint is set on __per_cpu_offset[cpu] (read-write
> >> operation), the specific cpu will be stuck in the infinite #DB loop.
> >> RCU will try to send NMI to the specific cpu, but it is not working
> >> either since NMI also relies on paranoid_entry().
> >
> > Should we consider having a .percpu..noinstr section and having
> > objtool enforce this?
> 
> I think so.

I'll put it on the TODO list somewhere ...

[tip: x86/urgent] x86/debug: Prevent data breakpoints on cpu_dr7

[tip-bot2 for Lai Jiangshan]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     3943abf2dbfae9ea4d2da05c1db569a0603f76da
Gitweb:        https://git.kernel.org/tip/3943abf2dbfae9ea4d2da05c1db569a0603f76da
Author:        Lai Jiangshan <laijs@linux.alibaba.com>
AuthorDate:    Thu, 04 Feb 2021 23:27:07 +08:00
Committer:     Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Fri, 05 Feb 2021 20:13:12 +01:00

x86/debug: Prevent data breakpoints on cpu_dr7

local_db_save() is called at the start of exc_debug_kernel(), reads DR7 and
disables breakpoints to prevent recursion.

When running in a guest (X86_FEATURE_HYPERVISOR), local_db_save() reads the
per-cpu variable cpu_dr7 to check whether a breakpoint is active or not
before it accesses DR7.

A data breakpoint on cpu_dr7 therefore results in infinite #DB recursion.

Disallow data breakpoints on cpu_dr7 to prevent that.

Fixes: 84b6a3491567a("x86/entry: Optimize local_db_save() for virt")
Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20210204152708.21308-2-jiangshanlai@gmail.com

---
 arch/x86/kernel/hw_breakpoint.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index 012ed82..668a4a6 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -307,6 +307,14 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end)
 				(unsigned long)&per_cpu(cpu_tlbstate, cpu),
 				sizeof(struct tlb_state)))
 			return true;
+
+		/*
+		 * When in guest (X86_FEATURE_HYPERVISOR), local_db_save()
+		 * will read per-cpu cpu_dr7 before clear dr7 register.
+		 */
+		if (within_area(addr, end, (unsigned long)&per_cpu(cpu_dr7, cpu),
+				sizeof(cpu_dr7)))
+			return true;
 	}
 
 	return false;

[tip: x86/urgent] x86/debug: Prevent data breakpoints on __per_cpu_offset

[tip-bot2 for Lai Jiangshan]

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     c4bed4b96918ff1d062ee81fdae4d207da4fa9b0
Gitweb:        https://git.kernel.org/tip/c4bed4b96918ff1d062ee81fdae4d207da4fa9b0
Author:        Lai Jiangshan <laijs@linux.alibaba.com>
AuthorDate:    Thu, 04 Feb 2021 23:27:06 +08:00
Committer:     Thomas Gleixner <tglx@linutronix.de>
CommitterDate: Fri, 05 Feb 2021 20:13:11 +01:00

x86/debug: Prevent data breakpoints on __per_cpu_offset

When FSGSBASE is enabled, paranoid_entry() fetches the per-CPU GSBASE value
via __per_cpu_offset or pcpu_unit_offsets.

When a data breakpoint is set on __per_cpu_offset[cpu] (read-write
operation), the specific CPU will be stuck in an infinite #DB loop.

RCU will try to send an NMI to the specific CPU, but it is not working
either since NMI also relies on paranoid_entry(). Which means it's
undebuggable.

Fixes: eaad981291ee3("x86/entry/64: Introduce the FIND_PERCPU_BASE macro")
Signed-off-by: Lai Jiangshan <laijs@linux.alibaba.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/r/20210204152708.21308-1-jiangshanlai@gmail.com

---
 arch/x86/kernel/hw_breakpoint.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index 6694c0f..012ed82 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -269,6 +269,20 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end)
 			CPU_ENTRY_AREA_TOTAL_SIZE))
 		return true;
 
+	/*
+	 * When FSGSBASE is enabled, paranoid_entry() fetches the per-CPU
+	 * GSBASE value via __per_cpu_offset or pcpu_unit_offsets.
+	 */
+#ifdef CONFIG_SMP
+	if (within_area(addr, end, (unsigned long)__per_cpu_offset,
+			sizeof(unsigned long) * nr_cpu_ids))
+		return true;
+#else
+	if (within_area(addr, end, (unsigned long)&pcpu_unit_offsets,
+			sizeof(pcpu_unit_offsets)))
+		return true;
+#endif
+
 	for_each_possible_cpu(cpu) {
 		/* The original rw GDT is being used after load_direct_gdt() */
 		if (within_area(addr, end, (unsigned long)get_cpu_gdt_rw(cpu),