* [PATCH v3] x86: add NOPL and CMOV emulation [not found] <YNWAwVfzSdML/WhO@hirez.programming.kicks-ass.net> @ 2021-06-26 13:03 ` Marcos Del Sol Vives 2021-06-27 10:57 ` David Laight ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: Marcos Del Sol Vives @ 2021-06-26 13:03 UTC (permalink / raw) To: x86; +Cc: Marcos Del Sol Vives, linux-kernel NOPL and CMOV are a set of instructions that were introduced to the x86 architecture with the i686 ISA in 1995, first implemented in the Intel Pentium Pro and Pentium II processors. While virtually all desktop and server systems are modern enough to support these instructions, in the embedded market things are a little bit different: DM&P is still manufacturing their i586-only Vortex86 SoCs, and many embedded devices still in use are stuck on legacy SoCs that either are i586-only (AMD Elan, AMD Geode GX1) or implement i686 but lack the NOPL instructions (Transmeta devices, AMD Geode LX). This is a problem because most modern Linux distributions, such as Debian and all its derivatives, have started compiling targeting i686, leaving old embedded devices using binary distributions without an easy upgrade path. This ultimately results in most of these embedded devices running years old, insecure and obsolete installations, and this itself can be seen on the DM&P's own supported OS page where the newest supported desktop Linux distro is Ubuntu 18.04, already three years old. The emulation of these instructions thus allow upgrading to newer distributions just by replacing the kernel, keeping all precompiled binaries intact. The way this emulation is implemented is fairly simple: it uses the illegal instruction handler to trap these unsupported instructions, and evaluates them in software. Thus a kernel compiled with this feature can be still used on a machine that fully supports the i686 ISA with no performance penalty at all. Cc: linux-kernel@vger.kernel.org Signed-off-by: Marcos Del Sol Vives <marcos@orca.pet> --- arch/x86/Kconfig | 20 ++++ arch/x86/include/asm/soft86.h | 24 +++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/soft86.c | 193 ++++++++++++++++++++++++++++++++++ arch/x86/kernel/traps.c | 5 + 5 files changed, 243 insertions(+) create mode 100644 arch/x86/include/asm/soft86.h create mode 100644 arch/x86/kernel/soft86.c diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 0045e1b44190..631884bbf8a3 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1749,6 +1749,26 @@ config MATH_EMULATION If you are not sure, say Y; apart from resulting in a 66 KB bigger kernel, it won't hurt. +config X86_INSN_EMU + bool "Instruction emulation" + help + Linux can be compiled to emulate some instructions transparently to + an application, allowing older processors to run modern software + without recompilation, albeit with a significant performance hit. + + Currently supported instructions are: + - CMOVxx (conditional moves). + - NOPL (long NOPs). + + Emulating these two instructions allow i686 binaries to run + unmodified on devices that only support i586 (Intel Pentium 1, + AMD Geode GX1, Cyrix III, Vortex86SX/MX/DX, WinChips), or are i686 + but miss some of the instructions (Transmeta Crusoe/Efficeon, + AMD Geode LX) + + This emulation is only used if the processor is unable execute said + instructions, and will not be used if supported natively. + config MTRR def_bool y prompt "MTRR (Memory Type Range Register) support" if EXPERT diff --git a/arch/x86/include/asm/soft86.h b/arch/x86/include/asm/soft86.h new file mode 100644 index 000000000000..67b9e084a0a1 --- /dev/null +++ b/arch/x86/include/asm/soft86.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ + +#ifndef _ASM_X86_SOFT86_H +#define _ASM_X86_SOFT86_H + +/* + * Software execution of x86 opcodes + * + * Copyright (C) 2021, Marcos Del Sol Vives <marcos@orca.pet> + */ + +#include <linux/types.h> +#include <asm/ptrace.h> + +#ifdef CONFIG_X86_INSN_EMU +bool soft86_execute(struct pt_regs *regs); +#else +static inline bool soft86_execute(struct pt_regs *regs) +{ + return false; +} +#endif + +#endif /* _ASM_X86_SOFT86_H */ diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile index 0f66682ac02a..d538f4f5641c 100644 --- a/arch/x86/kernel/Makefile +++ b/arch/x86/kernel/Makefile @@ -143,6 +143,7 @@ obj-$(CONFIG_PERF_EVENTS) += perf_regs.o obj-$(CONFIG_TRACING) += tracepoint.o obj-$(CONFIG_SCHED_MC_PRIO) += itmt.o obj-$(CONFIG_X86_UMIP) += umip.o +obj-$(CONFIG_X86_INSN_EMU) += soft86.o obj-$(CONFIG_UNWINDER_ORC) += unwind_orc.o obj-$(CONFIG_UNWINDER_FRAME_POINTER) += unwind_frame.o diff --git a/arch/x86/kernel/soft86.c b/arch/x86/kernel/soft86.c new file mode 100644 index 000000000000..98d9e08599ff --- /dev/null +++ b/arch/x86/kernel/soft86.c @@ -0,0 +1,193 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Software execution of x86 opcodes + * + * Copyright (c) 2021, Marcos Del Sol Vives <marcos@orca.pet> + */ + +#include <linux/uaccess.h> + +#include <asm/insn.h> +#include <asm/insn-eval.h> +#include <asm/soft86.h> + +static bool cmov_check_condition(struct insn *insn, struct pt_regs *regs) +{ + bool result, invert; + int condition, flags; + + /* + * Bits 3-1 of the second opcode byte specify the condition. + * + * Bit 0 of the second opcode byte is a flag - if set, the result must + * be inverted. + */ + condition = (insn->opcode.bytes[1] >> 1) & 0x7; + invert = insn->opcode.bytes[1] & 1; + + flags = regs->flags; + switch (condition) { + case 0: + /* + * 0F 40 CMOVO + * 0F 41 CMOVNO + */ + result = flags & X86_EFLAGS_OF; + break; + + case 1: + /* + * 0F 42 CMOVC/CMOVNAE + * 0F 43 CMOVNC/CMOVNB + */ + result = flags & X86_EFLAGS_CF; + break; + + case 2: + /* + * 0F 44 CMOVE/CMOVZ + * 0F 45 CMOVNE/CMOVNZ + */ + result = flags & X86_EFLAGS_ZF; + break; + + case 3: + /* + * 0F 46 CMOVNA/CMOVBE + * 0F 47 CMOVA/CMOVNBE + */ + result = (flags & X86_EFLAGS_CF) || + (flags & X86_EFLAGS_ZF); + break; + + case 4: + /* + * 0F 48 CMOVS + * 0F 49 CMOVNS + */ + result = flags & X86_EFLAGS_SF; + break; + + case 5: + /* + * 0F 4A CMOVP + * 0F 4B CMOVNP + */ + result = flags & X86_EFLAGS_PF; + break; + + case 6: + /* + * 0F 4C CMOVL/CMOVNGE + * 0F 4D CMOVNL/CMOVGE + */ + result = !!(flags & X86_EFLAGS_SF) != + !!(flags & X86_EFLAGS_OF); + break; + + case 7: + /* + * 0F 4E CMOVLE/CMOVNG + * 0F 4F CMOVNLE/CMOVG + */ + result = (flags & X86_EFLAGS_ZF) || + !!(flags & X86_EFLAGS_SF) != + !!(flags & X86_EFLAGS_OF); + break; + } + + if (invert) + result = !result; + + return result; +} + +static bool cmov_do_move(struct insn *insn, struct pt_regs *regs) +{ + int reg_off, rm_off; + void __user *src; + unsigned char *reg_bytes; + + reg_bytes = (unsigned char *)regs; + + /* Destination, from the REG part of the ModRM */ + reg_off = insn_get_modrm_reg_off(insn, regs); + if (reg_off < 0) + return false; + + /* Register to register move */ + if (X86_MODRM_MOD(insn->modrm.value) == 3) { + rm_off = insn_get_modrm_rm_off(insn, regs); + if (rm_off < 0) + return false; + + memcpy(reg_bytes + reg_off, reg_bytes + rm_off, + insn->addr_bytes); + } else { + /* Source from the RM part of the ModRM */ + src = insn_get_addr_ref(insn, regs); + if (src == (void __user *)-1L) + return false; + + if (copy_from_user(reg_bytes + reg_off, src, + insn->addr_bytes) != 0) + return false; + } + + return true; +} + +static bool cmov_execute(struct insn *insn, struct pt_regs *regs) +{ + /* CMOV is only supported for 16 and 32-bit registers */ + if (insn->addr_bytes != 2 && insn->addr_bytes != 4) + return false; + + /* If condition is met, execute the move */ + if (cmov_check_condition(insn, regs)) { + /* Return false if the operands were invalid */ + if (!cmov_do_move(insn, regs)) + return false; + } + + return true; +} + +bool soft86_execute(struct pt_regs *regs) +{ + int nr_copied; + unsigned char buf[MAX_INSN_SIZE]; + struct insn insn; + bool ret; + + /* Read from userspace */ + nr_copied = insn_fetch_from_user(regs, buf); + if (!nr_copied) + return false; + + /* Attempt to decode it */ + if (!insn_decode_from_regs(&insn, regs, buf, nr_copied)) + return false; + + /* 0x0F is the two byte opcode escape */ + if (insn.opcode.bytes[0] != 0x0F) + return false; + + switch (insn.opcode.bytes[1]) { + case 0x1F: + /* NOPL, so do nothing */ + ret = true; + break; + + case 0x40 ... 0x4F: + /* CMOVxx */ + ret = cmov_execute(&insn, regs); + break; + } + + /* Increment the instruction pointer if succeeded */ + if (ret) + regs->ip += insn.length; + + return ret; +} diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 853ea7a80806..a61d4d1d1bf1 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -58,6 +58,7 @@ #include <asm/fpu/xstate.h> #include <asm/vm86.h> #include <asm/umip.h> +#include <asm/soft86.h> #include <asm/insn.h> #include <asm/insn-eval.h> #include <asm/vdso.h> @@ -215,6 +216,10 @@ void handle_invalid_op(struct pt_regs *regs) static inline void handle_invalid_op(struct pt_regs *regs) #endif { + /* Attempt instruction emulation */ + if (user_mode(regs) && soft86_execute(regs)) + return; + do_error_trap(regs, 0, "invalid opcode", X86_TRAP_UD, SIGILL, ILL_ILLOPN, error_get_trap_addr(regs)); } -- 2.25.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* RE: [PATCH v3] x86: add NOPL and CMOV emulation 2021-06-26 13:03 ` [PATCH v3] x86: add NOPL and CMOV emulation Marcos Del Sol Vives @ 2021-06-27 10:57 ` David Laight 2021-06-28 0:52 ` Marcos Del Sol Vives 2021-06-27 19:20 ` Ondrej Zary 2021-06-29 16:34 ` Borislav Petkov 2 siblings, 1 reply; 8+ messages in thread From: David Laight @ 2021-06-27 10:57 UTC (permalink / raw) To: 'Marcos Del Sol Vives', x86; +Cc: linux-kernel From: Marcos Del Sol Vives > Sent: 26 June 2021 14:03 > > NOPL and CMOV are a set of instructions that were introduced to the > x86 architecture with the i686 ISA in 1995, first implemented in the > Intel Pentium Pro and Pentium II processors. > > While virtually all desktop and server systems are modern enough to > support these instructions, in the embedded market things are a little > bit different: DM&P is still manufacturing their i586-only Vortex86 > SoCs, and many embedded devices still in use are stuck on legacy SoCs > that either are i586-only (AMD Elan, AMD Geode GX1) or implement i686 > but lack the NOPL instructions (Transmeta devices, AMD Geode LX). > > This is a problem because most modern Linux distributions, such as > Debian and all its derivatives, have started compiling targeting i686, > leaving old embedded devices using binary distributions without an > easy upgrade path. > > This ultimately results in most of these embedded devices running > years old, insecure and obsolete installations, and this itself can be > seen on the DM&P's own supported OS page where the newest supported > desktop Linux distro is Ubuntu 18.04, already three years old. > > The emulation of these instructions thus allow upgrading to newer > distributions just by replacing the kernel, keeping all precompiled > binaries intact. ... Does this really help? 1) Trapping and emulating the instructions will be slow. 2) All 64bit cpus support these instructions - so these must be 32bit. I believe the main distributions are about to drip 32bit support. It also has to be said that using Ubuntu in an embedded device is about as sensible as running windows. There is far too much 'crud' running that you don't need and is only likely to result in security breaches. Much better is something based on busybox+buildroot where you have (almost) complete control of the system userspace and can easily build a kernel that only has support for the required hardware from one of the LTS kernel trees. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales) ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] x86: add NOPL and CMOV emulation 2021-06-27 10:57 ` David Laight @ 2021-06-28 0:52 ` Marcos Del Sol Vives 2021-06-28 7:54 ` David Laight 0 siblings, 1 reply; 8+ messages in thread From: Marcos Del Sol Vives @ 2021-06-28 0:52 UTC (permalink / raw) To: David Laight, x86; +Cc: linux-kernel El 27/06/2021 a las 12:57, David Laight escribió: > From: Marcos Del Sol Vives >> Sent: 26 June 2021 14:03 >> >> NOPL and CMOV are a set of instructions that were introduced to the >> x86 architecture with the i686 ISA in 1995, first implemented in the >> Intel Pentium Pro and Pentium II processors. >> >> While virtually all desktop and server systems are modern enough to >> support these instructions, in the embedded market things are a little >> bit different: DM&P is still manufacturing their i586-only Vortex86 >> SoCs, and many embedded devices still in use are stuck on legacy SoCs >> that either are i586-only (AMD Elan, AMD Geode GX1) or implement i686 >> but lack the NOPL instructions (Transmeta devices, AMD Geode LX). >> >> This is a problem because most modern Linux distributions, such as >> Debian and all its derivatives, have started compiling targeting i686, >> leaving old embedded devices using binary distributions without an >> easy upgrade path. >> >> This ultimately results in most of these embedded devices running >> years old, insecure and obsolete installations, and this itself can be >> seen on the DM&P's own supported OS page where the newest supported >> desktop Linux distro is Ubuntu 18.04, already three years old. >> >> The emulation of these instructions thus allow upgrading to newer >> distributions just by replacing the kernel, keeping all precompiled >> binaries intact. > ... > > Does this really help? > 1) Trapping and emulating the instructions will be slow. This is true. An emulated CMOV is almost 50 times slower than conditional branch on my Vortex86MX. However, in practice this results in very little slowdown - GCC seems to really avoid emitting CMOVs, preferring conditional branches instead, so applications are still completely useable. > 2) All 64bit cpus support these instructions - so these must be 32bit. > I believe the main distributions are about to drip 32bit support. While it is true that Ubuntu has dropped support for 32 bit devices, Debian, OpenSUSE and many others don't seem to plan on doing so anytime soon. > It also has to be said that using Ubuntu in an embedded device > is about as sensible as running windows. > There is far too much 'crud' running that you don't need and > is only likely to result in security breaches. > > Much better is something based on busybox+buildroot where you > have (almost) complete control of the system userspace and > can easily build a kernel that only has support for the required > hardware from one of the LTS kernel trees. These SoCs are not the average ARM-based SoC used in very high volume products such as WiFi routers where you want a million of units all executing the same thing. For those I totally agree you'd use a fully customized operating system on a build root image. Vortex86 devices are on the other hand used generally in low-volume, niche applications, where the premium of a x86-compatible Vortex86 device that can boot any standard x86 operating systems and applications is preferrable to the cost of developing a fully customized operating system image. I've seen such devices used by small companies to develop point of sale machines, information screen, kiosks, etc... where they benefit from being easy to set up and, being a standard operating system, allow easily tweaking to the customer's needs. Myself, I am using an eBox 3350MX-AP (which sports a Vortex86MX with 512MB of RAM) as a small, low-power web server running Debian sid. If I were to use buildroot, I'd lose completely the ability to install new features or apply updates without having to rebuild the entire image, which is a complex and time-consuming task. ^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [PATCH v3] x86: add NOPL and CMOV emulation 2021-06-28 0:52 ` Marcos Del Sol Vives @ 2021-06-28 7:54 ` David Laight 0 siblings, 0 replies; 8+ messages in thread From: David Laight @ 2021-06-28 7:54 UTC (permalink / raw) To: 'Marcos Del Sol Vives', x86; +Cc: linux-kernel ... > I've seen such devices used by small companies to develop point of sale > machines, information screen, kiosks, etc... where they benefit from > being easy to set up and, being a standard operating system, allow > easily tweaking to the customer's needs. And systems where you really don't want to be running a major distribution since you almost certainly don't want to write the infrastructure to do security updates every few weeks. David - Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales) ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] x86: add NOPL and CMOV emulation 2021-06-26 13:03 ` [PATCH v3] x86: add NOPL and CMOV emulation Marcos Del Sol Vives 2021-06-27 10:57 ` David Laight @ 2021-06-27 19:20 ` Ondrej Zary 2021-06-29 16:34 ` Borislav Petkov 2 siblings, 0 replies; 8+ messages in thread From: Ondrej Zary @ 2021-06-27 19:20 UTC (permalink / raw) To: Marcos Del Sol Vives; +Cc: x86, linux-kernel On Saturday 26 June 2021 15:03:14 Marcos Del Sol Vives wrote: > NOPL and CMOV are a set of instructions that were introduced to the > x86 architecture with the i686 ISA in 1995, first implemented in the > Intel Pentium Pro and Pentium II processors. > > While virtually all desktop and server systems are modern enough to > support these instructions, in the embedded market things are a little > bit different: DM&P is still manufacturing their i586-only Vortex86 > SoCs, and many embedded devices still in use are stuck on legacy SoCs > that either are i586-only (AMD Elan, AMD Geode GX1) or implement i686 > but lack the NOPL instructions (Transmeta devices, AMD Geode LX). > > This is a problem because most modern Linux distributions, such as > Debian and all its derivatives, have started compiling targeting i686, > leaving old embedded devices using binary distributions without an > easy upgrade path. > > This ultimately results in most of these embedded devices running > years old, insecure and obsolete installations, and this itself can be > seen on the DM&P's own supported OS page where the newest supported > desktop Linux distro is Ubuntu 18.04, already three years old. > > The emulation of these instructions thus allow upgrading to newer > distributions just by replacing the kernel, keeping all precompiled > binaries intact. Great, this would allow me to update my old Pentium 133 box from unsupported Debian 8 to 9, 10 and even the upcoming 11. -- Ondrej Zary ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] x86: add NOPL and CMOV emulation 2021-06-26 13:03 ` [PATCH v3] x86: add NOPL and CMOV emulation Marcos Del Sol Vives 2021-06-27 10:57 ` David Laight 2021-06-27 19:20 ` Ondrej Zary @ 2021-06-29 16:34 ` Borislav Petkov 2021-06-29 20:45 ` Marcos Del Sol Vives 2 siblings, 1 reply; 8+ messages in thread From: Borislav Petkov @ 2021-06-29 16:34 UTC (permalink / raw) To: Marcos Del Sol Vives; +Cc: x86, linux-kernel On Sat, Jun 26, 2021 at 03:03:14PM +0200, Marcos Del Sol Vives wrote: > +config X86_INSN_EMU > + bool "Instruction emulation" > + help > + Linux can be compiled to emulate some instructions transparently to > + an application, allowing older processors to run modern software > + without recompilation, albeit with a significant performance hit. > + > + Currently supported instructions are: > + - CMOVxx (conditional moves). > + - NOPL (long NOPs). > + > + Emulating these two instructions allow i686 binaries to run > + unmodified on devices that only support i586 (Intel Pentium 1, > + AMD Geode GX1, Cyrix III, Vortex86SX/MX/DX, WinChips), or are i686 > + but miss some of the instructions (Transmeta Crusoe/Efficeon, > + AMD Geode LX) > + > + This emulation is only used if the processor is unable execute said "... unable to execute... " > + instructions, and will not be used if supported natively. And this is making me wonder whether this needs to be a Kconfig option at all or not simply enabled by default. Or is there a reason not to emulate instructions? We could do a pr_info_once("Emulating x86 instructions... "); or so to let people know that we're not running natively in unexpected configurations. For example if (cpu_feature_enabled(X86_FEATURE_CMOV)) pr_warn_once("Emulating x86 CMOV even if CPU claims it supports it!"); and other sanity-checks like that. Which would allow us to support any mode - not only protected but also long mode, in case we need that emulation there too for whatever reason. And I'm trying to think of some sort of a test case for this because we'd need something like that but the only thing I can think of is something like: UD2 CMOV... in userspace and then in the #UD handler - since we will have a soft86_execute() call there - you could advance the rIP by 2 bytes (UD2 is 0f 0b), i.e., do regs->ip += 2 and then land rIP at the CMOV and then try to emulate it. And have that testing controllable by a cmdline param or so so that it is not enabled by default. And then stick the test program in tools/testing/selftests/x86/ Something like that, at least. > +static bool cmov_check_condition(struct insn *insn, struct pt_regs *regs) > +{ > + bool result, invert; > + int condition, flags; > + > + /* > + * Bits 3-1 of the second opcode byte specify the condition. > + * > + * Bit 0 of the second opcode byte is a flag - if set, the result must > + * be inverted. > + */ This comment goes over the function. Along with documenting what that function returns. > +static bool cmov_do_move(struct insn *insn, struct pt_regs *regs) So this function could use some explanation what it is doing - basically, shuffling data between pt_regs members to simulate the register moves. Along with an explicit "CMOV: Conditionally moves from first operand (mem/reg) into second operand (reg)." to explain the direction of the movement so that people don't have to decipher this function each time. > +{ > + int reg_off, rm_off; > + void __user *src; > + unsigned char *reg_bytes; > + > + reg_bytes = (unsigned char *)regs; > + > + /* Destination, from the REG part of the ModRM */ > + reg_off = insn_get_modrm_reg_off(insn, regs); > + if (reg_off < 0) > + return false; > + > + /* Register to register move */ > + if (X86_MODRM_MOD(insn->modrm.value) == 3) { > + rm_off = insn_get_modrm_rm_off(insn, regs); > + if (rm_off < 0) > + return false; > + > + memcpy(reg_bytes + reg_off, reg_bytes + rm_off, > + insn->addr_bytes); > + } else { > + /* Source from the RM part of the ModRM */ > + src = insn_get_addr_ref(insn, regs); > + if (src == (void __user *)-1L) > + return false; > + > + if (copy_from_user(reg_bytes + reg_off, src, > + insn->addr_bytes) != 0) > + return false; > + } > + > + return true; > +} > + > +static bool cmov_execute(struct insn *insn, struct pt_regs *regs) > +{ > + /* CMOV is only supported for 16 and 32-bit registers */ I don't understand this one: addr_bytes are either 4 or 8 AFAICT. > + if (insn->addr_bytes != 2 && insn->addr_bytes != 4) > + return false; > + > + /* If condition is met, execute the move */ > + if (cmov_check_condition(insn, regs)) { Just like with the rest of the flow, do the same here: if (!cmov_check_condition(insn, regs)) return false; if (!cmov_do_move... > + /* Return false if the operands were invalid */ > + if (!cmov_do_move(insn, regs)) > + return false; > + } > + > + return true; > +} > + > +bool soft86_execute(struct pt_regs *regs) > +{ > + int nr_copied; > + unsigned char buf[MAX_INSN_SIZE]; > + struct insn insn; > + bool ret; The tip-tree preferred ordering of variable declarations at the beginning of a function is reverse fir tree order:: struct long_struct_name *descriptive_name; unsigned long foo, bar; unsigned int tmp; int ret; The above is faster to parse than the reverse ordering:: int ret; unsigned int tmp; unsigned long foo, bar; struct long_struct_name *descriptive_name; And even more so than random ordering:: unsigned long foo, bar; int ret; struct long_struct_name *descriptive_name; unsigned int tmp; Please fix all your functions. > + /* Read from userspace */ > + nr_copied = insn_fetch_from_user(regs, buf); > + if (!nr_copied) > + return false; > + > + /* Attempt to decode it */ > + if (!insn_decode_from_regs(&insn, regs, buf, nr_copied)) > + return false; > + > + /* 0x0F is the two byte opcode escape */ > + if (insn.opcode.bytes[0] != 0x0F) > + return false; That function is a generic one but you start looking at bytes. What you should do instead is something like: ret = emulate_nopl(insn); if (!ret) goto done; ret = emulate_cmov(insn); if (!ret) goto done; ... done: regs->ip += insn.length; and move all the opcode checking in the respective functions. This way each function does one thing and one thing only and is self-contained. Also, the logic should be flipped: the functions should return 0 on success and !0 on failure so the first function which returned 0, would mean it has detected and emulated the insn properly. If none of them manage to succeed, you return false at the end. All in all, fun stuff! Thx. -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] x86: add NOPL and CMOV emulation 2021-06-29 16:34 ` Borislav Petkov @ 2021-06-29 20:45 ` Marcos Del Sol Vives 2021-06-30 9:38 ` Borislav Petkov 0 siblings, 1 reply; 8+ messages in thread From: Marcos Del Sol Vives @ 2021-06-29 20:45 UTC (permalink / raw) To: Borislav Petkov; +Cc: x86, linux-kernel After some further testing I am not fully confident if this patch should be merged. I spent yesterday night trying to debug why the Debian installation was behaving so erratically. It'd boot all the way to the commandline, ping successfully the internet, serve a website over HTTP, connect via SSH to remote machines and build stuff using GCC. However, I could neither install new packages, nor have sshd running, nor properly execute "mv", which would actually move the file but then complain about running out of memory. All of this without a trace of an exception or any kind of error, neither on the program themselves nor on dmesg. All the documentation I had previously read suggested that only CMOV and NOPL had been introduced with the i686, and hence these were the two instructions I emulated only. As stated previously this is also enough to boot Debian mostly flawless. However, and contrary to what I thought, the i686 also saw the introduction of a handful of other x87 instructions (FCMOVB, FCMOVBE, FCMOVE, FCMOVNB, FCMOVNBE, FCMOVNE, FCMOVNU, FCMOVU, FCOMI, FCOMIP, FUCOMI, FUCOMIP) Running the program below: ---- #include <stdio.h> #include <stdint.h> int main() { float val = 6.21; uint16_t status; #ifdef __x86_64__ uint64_t eflags; #else uint32_t eflags; #endif asm( /* Set EFLAGS to a known value */ "xor %0, %0\n" "test %0, %0\n" /* Execute FUCOMI and get x87 FLAGS and x86 EFLAGS */ "flds %2\n" "fucomi %%st(0), %%st(0)\n" "fnstsw %w0\n" "pushf\n" "pop %1\n" :"=a"(status),"=r"(eflags) :"m"(val) ); printf("Float value: %f\n", val); printf("FPU status: %04X EFLAGS: %08X\n", status, (int) eflags); return 0; } --- On my modern i7 outputs: Float value: 6.210000 FPU status: 3800 EFLAGS: 00000242 However on my i586 Vortex86MX I get: Float value: nan FPU status: 3800 EFLAGS: 00000246 The reason why it display 6.21 as "nan" is that libc is using FUCOMI (floating point unsorted comparison) to test for NaNs by comparing the number to print against itself, as a comparison involving NaNs is an invalid operation. If the floating value was either a normal number or infinity, this works and EFLAGS are updated according to the value. If the value is NaN however, the FPU sets a couple flags (PF, CF at least, I think OF too) to 1 to signal that an illegal operation happened, which is checked by using a JP (jump if parity set) to branch to the code path that displays "nan" instead of the value. This i586 (not sure if specific to this processor in particular, or it's standard within all i586) is that it __always__ sets the parity bit on a FUCOMI (bit 2 of EFLAGS), so all numbers are displayed as "nan", regardless of their value. The worst part of all is that these unsupported instructions seem not to raise the illegal opcode exception, so it might not be possible to emulate them at all, making this patch completely pointless. There doesn't seem to be any flag on the FPU's status signaling an error has happened either. El 29/06/2021 a las 18:34, Borislav Petkov escribió: > On Sat, Jun 26, 2021 at 03:03:14PM +0200, Marcos Del Sol Vives wrote: >> +config X86_INSN_EMU >> + bool "Instruction emulation" >> + help >> + Linux can be compiled to emulate some instructions transparently to >> + an application, allowing older processors to run modern software >> + without recompilation, albeit with a significant performance hit. >> + >> + Currently supported instructions are: >> + - CMOVxx (conditional moves). >> + - NOPL (long NOPs). >> + >> + Emulating these two instructions allow i686 binaries to run >> + unmodified on devices that only support i586 (Intel Pentium 1, >> + AMD Geode GX1, Cyrix III, Vortex86SX/MX/DX, WinChips), or are i686 >> + but miss some of the instructions (Transmeta Crusoe/Efficeon, >> + AMD Geode LX) >> + >> + This emulation is only used if the processor is unable execute said > > "... unable to execute..." > >> + instructions, and will not be used if supported natively. > > And this is making me wonder whether this needs to be a Kconfig option > at all or not simply enabled by default. Or is there a reason not to > emulate instructions? I added that option because enabling this features on i686/i786 machines will just increase code bloat for no reason. Also, I am not an expert in cryptography but I think CMOV instructions with their fixed duration are widely used for executing algorithms in constant time and avoiding timing leaks. Thus given emulating them would throw that fixed timing out of the window, I preferred to leave this emulation disabled by default, so only someone who understands the security implications this might carry would enable it. > > We could do a > > pr_info_once("Emulating x86 instructions... "); > > or so to let people know that we're not running natively in unexpected > configurations. For example > > if (cpu_feature_enabled(X86_FEATURE_CMOV)) > pr_warn_once("Emulating x86 CMOV even if CPU claims it supports it!"); > > and other sanity-checks like that. > > Which would allow us to support any mode - not only protected but also > long mode, in case we need that emulation there too for whatever reason. > > And I'm trying to think of some sort of a test case for this because > we'd need something like that but the only thing I can think of is > something like: > > UD2 > CMOV... > > in userspace and then in the #UD handler - since we will have a > soft86_execute() call there - you could advance the rIP by 2 bytes (UD2 > is 0f 0b), i.e., do regs->ip += 2 and then land rIP at the CMOV and then > try to emulate it. > > And have that testing controllable by a cmdline param or so so that it > is not enabled by default. > > And then stick the test program in tools/testing/selftests/x86/ > > Something like that, at least. > >> +static bool cmov_check_condition(struct insn *insn, struct pt_regs *regs) >> +{ >> + bool result, invert; >> + int condition, flags; >> + >> + /* >> + * Bits 3-1 of the second opcode byte specify the condition. >> + * >> + * Bit 0 of the second opcode byte is a flag - if set, the result must >> + * be inverted. >> + */ > > This comment goes over the function. Along with documenting what that > function returns. > >> +static bool cmov_do_move(struct insn *insn, struct pt_regs *regs) > > So this function could use some explanation what it is doing - > basically, shuffling data between pt_regs members to simulate the > register moves. > > Along with an explicit > > "CMOV: Conditionally moves from first operand (mem/reg) into second > operand (reg)." > > to explain the direction of the movement so that people don't have to > decipher this function each time. > >> +{ >> + int reg_off, rm_off; >> + void __user *src; >> + unsigned char *reg_bytes; >> + >> + reg_bytes = (unsigned char *)regs; >> + >> + /* Destination, from the REG part of the ModRM */ >> + reg_off = insn_get_modrm_reg_off(insn, regs); >> + if (reg_off < 0) >> + return false; >> + >> + /* Register to register move */ >> + if (X86_MODRM_MOD(insn->modrm.value) == 3) { >> + rm_off = insn_get_modrm_rm_off(insn, regs); >> + if (rm_off < 0) >> + return false; >> + >> + memcpy(reg_bytes + reg_off, reg_bytes + rm_off, >> + insn->addr_bytes); >> + } else { >> + /* Source from the RM part of the ModRM */ >> + src = insn_get_addr_ref(insn, regs); >> + if (src == (void __user *)-1L) >> + return false; >> + >> + if (copy_from_user(reg_bytes + reg_off, src, >> + insn->addr_bytes) != 0) >> + return false; >> + } >> + >> + return true; >> +} >> + >> +static bool cmov_execute(struct insn *insn, struct pt_regs *regs) >> +{ >> + /* CMOV is only supported for 16 and 32-bit registers */ > > I don't understand this one: addr_bytes are either 4 or 8 AFAICT. Based on experimental testing (so it might be wrong indeed) by encoding multiple different CMOVs using GCC with both 16 and 32 bits as destination and pr_info'ing the fields of the insn structure, addr_bytes seemed to contain the length of the data to copy, hence why I did this. Again, it might be wrong. > >> + if (insn->addr_bytes != 2 && insn->addr_bytes != 4) >> + return false; >> + >> + /* If condition is met, execute the move */ >> + if (cmov_check_condition(insn, regs)) { > > Just like with the rest of the flow, do the same here: > > if (!cmov_check_condition(insn, regs)) > return false; > > if (!cmov_do_move... > >> + /* Return false if the operands were invalid */ >> + if (!cmov_do_move(insn, regs)) >> + return false; >> + } >> + >> + return true; >> +} >> + >> +bool soft86_execute(struct pt_regs *regs) >> +{ >> + int nr_copied; >> + unsigned char buf[MAX_INSN_SIZE]; >> + struct insn insn; >> + bool ret; > > The tip-tree preferred ordering of variable declarations at the > beginning of a function is reverse fir tree order:: > > struct long_struct_name *descriptive_name; > unsigned long foo, bar; > unsigned int tmp; > int ret; > > The above is faster to parse than the reverse ordering:: > > int ret; > unsigned int tmp; > unsigned long foo, bar; > struct long_struct_name *descriptive_name; > > And even more so than random ordering:: > > unsigned long foo, bar; > int ret; > struct long_struct_name *descriptive_name; > unsigned int tmp; > > Please fix all your functions. > >> + /* Read from userspace */ >> + nr_copied = insn_fetch_from_user(regs, buf); >> + if (!nr_copied) >> + return false; >> + >> + /* Attempt to decode it */ >> + if (!insn_decode_from_regs(&insn, regs, buf, nr_copied)) >> + return false; >> + >> + /* 0x0F is the two byte opcode escape */ >> + if (insn.opcode.bytes[0] != 0x0F) >> + return false; > > That function is a generic one but you start looking at bytes. What you > should do instead is something like: > > ret = emulate_nopl(insn); > if (!ret) > goto done; > > ret = emulate_cmov(insn); > if (!ret) > goto done; > > ... > > done: > regs->ip += insn.length; > > and move all the opcode checking in the respective functions. This way > each function does one thing and one thing only and is self-contained. > That's how the V1 patch mostly worked - way more modular, using an array of different isolated functions and iterating through them until either succeeded, but Peter Zijlstra asked me to change it to this approach as it would be faster. > Also, the logic should be flipped: the functions should return 0 on > success and !0 on failure so the first function which returned 0, would > mean it has detected and emulated the insn properly. > > If none of them manage to succeed, you return false at the end. > But I am using "bool"s not ints as return values. It doesn't make sense to return a "true" (non-zero) if the operation failed. > All in all, fun stuff! > > Thx. > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3] x86: add NOPL and CMOV emulation 2021-06-29 20:45 ` Marcos Del Sol Vives @ 2021-06-30 9:38 ` Borislav Petkov 0 siblings, 0 replies; 8+ messages in thread From: Borislav Petkov @ 2021-06-30 9:38 UTC (permalink / raw) To: Marcos Del Sol Vives; +Cc: x86, linux-kernel On Tue, Jun 29, 2021 at 10:45:17PM +0200, Marcos Del Sol Vives wrote: > All the documentation I had previously read suggested that only CMOV and > NOPL had been introduced with the i686, and hence these were the two > instructions I emulated only. As stated previously this is also enough to > boot Debian mostly flawless. > > However, and contrary to what I thought, the i686 also saw the introduction > of a handful of other x87 instructions (FCMOVB, FCMOVBE, FCMOVE, FCMOVNB, > FCMOVNBE, FCMOVNE, FCMOVNU, FCMOVU, FCOMI, FCOMIP, FUCOMI, FUCOMIP) Yah, looka here: https://en.wikipedia.org/wiki/FCMOV So before we play with this further, you could try to add a "nofpu" kernel cmdline param which does what fpu__init_system_early_generic() does: setup_clear_cpu_cap(X86_FEATURE_FPU) to stop the kernel from setting up FPU support and see how far you can get there. I'm afraid glibc does its own feature detection so it will see the FPU CPUID bit but if the kernel doesn't support an FPU - and glibc needs the kernel to handle the context - then maybe it'll stop using FPU instructions. But you'll have to try it because I fear no one even tested such a thing. Good luck. :) -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-06-30 9:38 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <YNWAwVfzSdML/WhO@hirez.programming.kicks-ass.net> 2021-06-26 13:03 ` [PATCH v3] x86: add NOPL and CMOV emulation Marcos Del Sol Vives 2021-06-27 10:57 ` David Laight 2021-06-28 0:52 ` Marcos Del Sol Vives 2021-06-28 7:54 ` David Laight 2021-06-27 19:20 ` Ondrej Zary 2021-06-29 16:34 ` Borislav Petkov 2021-06-29 20:45 ` Marcos Del Sol Vives 2021-06-30 9:38 ` Borislav Petkov
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).