From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9C2EEC43219 for ; Tue, 30 Apr 2019 21:08:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 693F121743 for ; Tue, 30 Apr 2019 21:08:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727119AbfD3VIN (ORCPT ); Tue, 30 Apr 2019 17:08:13 -0400 Received: from mail.kernel.org ([198.145.29.99]:52252 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726015AbfD3VIM (ORCPT ); Tue, 30 Apr 2019 17:08:12 -0400 Received: from gandalf.local.home (cpe-66-24-58-225.stny.res.rr.com [66.24.58.225]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C8C252087B; Tue, 30 Apr 2019 21:08:09 +0000 (UTC) Date: Tue, 30 Apr 2019 17:08:08 -0400 From: Steven Rostedt To: Linus Torvalds Cc: Andy Lutomirski , Peter Zijlstra , Nicolai Stange , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , "the arch/x86 maintainers" , Josh Poimboeuf , Jiri Kosina , Miroslav Benes , Petr Mladek , Joe Lawrence , Shuah Khan , Konrad Rzeszutek Wilk , Tim Chen , Sebastian Andrzej Siewior , Mimi Zohar , Juergen Gross , Nick Desaulniers , Nayna Jain , Masahiro Yamada , Joerg Roedel , Linux List Kernel Mailing , live-patching@vger.kernel.org, "open list:KERNEL SELFTEST FRAMEWORK" Subject: Re: [RFC][PATCH] ftrace/x86: Emulate call function while updating in breakpoint handler Message-ID: <20190430170808.1053b3e2@gandalf.local.home> In-Reply-To: References: <20190428133826.3e142cfd@oasis.local.home> <20190430135602.GD2589@hirez.programming.kicks-ass.net> <20190430130359.330e895b@gandalf.local.home> <20190430132024.0f03f5b8@gandalf.local.home> <20190430134913.4e29ce72@gandalf.local.home> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 30 Apr 2019 11:33:21 -0700 Linus Torvalds wrote: > > + "ftrace_emulate_call_update_irqoff:\n\t" > > + "push %gs:ftrace_bp_call_return\n\t" > > + "sti\n\t" > > + "jmp *ftrace_update_func_call\n" > > .. and this should then use the "push push sti ret" model instead. > > Plus get updated for objtool complaints. And unfortunately, this blows up on lockdep. Lockdep notices that the return from the breakpoint handler has interrupts enabled, and will not enable them in its shadow irqs disabled variable. But then we enabled them in the trampoline, without telling lockdep and we trigger something likes this: ------------[ cut here ]------------ IRQs not enabled as expected WARNING: CPU: 2 PID: 0 at kernel/time/tick-sched.c:979 tick_nohz_idle_enter+0x44/0x8c Modules linked in: CPU: 2 PID: 0 Comm: swapper/2 Not tainted 5.1.0-rc3-test+ #123 Hardware name: MSI MS-7823/CSM-H87M-G43 (MS-7823), BIOS V1.6 02/22/2014 EIP: tick_nohz_idle_enter+0x44/0x8c Code: f0 05 00 00 00 75 26 83 b8 c4 05 00 00 00 75 1d 80 3d 5f 0f 43 c1 00 75 14 68 72 74 16 c1 c6 05 5f 0f 43 c1 01 e8 33 d7 f8 ff <0f> 0b 58 fa e8 4e 2c 04 00 bb e0 36 6b c1 64 03 1d 28 81 56 c1 8b EAX: 0000001c EBX: ee769f84 ECX: 00000000 EDX: 00000006 ESI: 00000000 EDI: 00000002 EBP: ee769f50 ESP: ee769f48 DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00210292 CR0: 80050033 CR2: 00000000 CR3: 016c4000 CR4: 001406f0 Call Trace: do_idle+0x2a/0x1fc cpu_startup_entry+0x1e/0x20 start_secondary+0x1d3/0x1ec startup_32_smp+0x164/0x168 I have to fool lockdep with the following: if (regs->flags & X86_EFLAGS_IF) { regs->flags &= ~X86_EFLAGS_IF; regs->ip = (unsigned long) ftrace_emulate_call_irqoff; /* Tell lockdep here we are enabling interrupts */ trace_hardirqs_on(); } else { regs->ip = (unsigned long) ftrace_emulate_call_irqon; } -- Steve From mboxrd@z Thu Jan 1 00:00:00 1970 From: rostedt at goodmis.org (Steven Rostedt) Date: Tue, 30 Apr 2019 17:08:08 -0400 Subject: [RFC][PATCH] ftrace/x86: Emulate call function while updating in breakpoint handler In-Reply-To: References: <20190428133826.3e142cfd@oasis.local.home> <20190430135602.GD2589@hirez.programming.kicks-ass.net> <20190430130359.330e895b@gandalf.local.home> <20190430132024.0f03f5b8@gandalf.local.home> <20190430134913.4e29ce72@gandalf.local.home> Message-ID: <20190430170808.1053b3e2@gandalf.local.home> On Tue, 30 Apr 2019 11:33:21 -0700 Linus Torvalds wrote: > > + "ftrace_emulate_call_update_irqoff:\n\t" > > + "push %gs:ftrace_bp_call_return\n\t" > > + "sti\n\t" > > + "jmp *ftrace_update_func_call\n" > > .. and this should then use the "push push sti ret" model instead. > > Plus get updated for objtool complaints. And unfortunately, this blows up on lockdep. Lockdep notices that the return from the breakpoint handler has interrupts enabled, and will not enable them in its shadow irqs disabled variable. But then we enabled them in the trampoline, without telling lockdep and we trigger something likes this: ------------[ cut here ]------------ IRQs not enabled as expected WARNING: CPU: 2 PID: 0 at kernel/time/tick-sched.c:979 tick_nohz_idle_enter+0x44/0x8c Modules linked in: CPU: 2 PID: 0 Comm: swapper/2 Not tainted 5.1.0-rc3-test+ #123 Hardware name: MSI MS-7823/CSM-H87M-G43 (MS-7823), BIOS V1.6 02/22/2014 EIP: tick_nohz_idle_enter+0x44/0x8c Code: f0 05 00 00 00 75 26 83 b8 c4 05 00 00 00 75 1d 80 3d 5f 0f 43 c1 00 75 14 68 72 74 16 c1 c6 05 5f 0f 43 c1 01 e8 33 d7 f8 ff <0f> 0b 58 fa e8 4e 2c 04 00 bb e0 36 6b c1 64 03 1d 28 81 56 c1 8b EAX: 0000001c EBX: ee769f84 ECX: 00000000 EDX: 00000006 ESI: 00000000 EDI: 00000002 EBP: ee769f50 ESP: ee769f48 DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00210292 CR0: 80050033 CR2: 00000000 CR3: 016c4000 CR4: 001406f0 Call Trace: do_idle+0x2a/0x1fc cpu_startup_entry+0x1e/0x20 start_secondary+0x1d3/0x1ec startup_32_smp+0x164/0x168 I have to fool lockdep with the following: if (regs->flags & X86_EFLAGS_IF) { regs->flags &= ~X86_EFLAGS_IF; regs->ip = (unsigned long) ftrace_emulate_call_irqoff; /* Tell lockdep here we are enabling interrupts */ trace_hardirqs_on(); } else { regs->ip = (unsigned long) ftrace_emulate_call_irqon; } -- Steve From mboxrd@z Thu Jan 1 00:00:00 1970 From: rostedt@goodmis.org (Steven Rostedt) Date: Tue, 30 Apr 2019 17:08:08 -0400 Subject: [RFC][PATCH] ftrace/x86: Emulate call function while updating in breakpoint handler In-Reply-To: References: <20190428133826.3e142cfd@oasis.local.home> <20190430135602.GD2589@hirez.programming.kicks-ass.net> <20190430130359.330e895b@gandalf.local.home> <20190430132024.0f03f5b8@gandalf.local.home> <20190430134913.4e29ce72@gandalf.local.home> Message-ID: <20190430170808.1053b3e2@gandalf.local.home> Content-Type: text/plain; charset="UTF-8" Message-ID: <20190430210808.fpqIj3Tsd-Bjgbp8gtxUiwWKmTvxT0up80lzaK80LKs@z> On Tue, 30 Apr 2019 11:33:21 -0700 Linus Torvalds wrote: > > + "ftrace_emulate_call_update_irqoff:\n\t" > > + "push %gs:ftrace_bp_call_return\n\t" > > + "sti\n\t" > > + "jmp *ftrace_update_func_call\n" > > .. and this should then use the "push push sti ret" model instead. > > Plus get updated for objtool complaints. And unfortunately, this blows up on lockdep. Lockdep notices that the return from the breakpoint handler has interrupts enabled, and will not enable them in its shadow irqs disabled variable. But then we enabled them in the trampoline, without telling lockdep and we trigger something likes this: ------------[ cut here ]------------ IRQs not enabled as expected WARNING: CPU: 2 PID: 0 at kernel/time/tick-sched.c:979 tick_nohz_idle_enter+0x44/0x8c Modules linked in: CPU: 2 PID: 0 Comm: swapper/2 Not tainted 5.1.0-rc3-test+ #123 Hardware name: MSI MS-7823/CSM-H87M-G43 (MS-7823), BIOS V1.6 02/22/2014 EIP: tick_nohz_idle_enter+0x44/0x8c Code: f0 05 00 00 00 75 26 83 b8 c4 05 00 00 00 75 1d 80 3d 5f 0f 43 c1 00 75 14 68 72 74 16 c1 c6 05 5f 0f 43 c1 01 e8 33 d7 f8 ff <0f> 0b 58 fa e8 4e 2c 04 00 bb e0 36 6b c1 64 03 1d 28 81 56 c1 8b EAX: 0000001c EBX: ee769f84 ECX: 00000000 EDX: 00000006 ESI: 00000000 EDI: 00000002 EBP: ee769f50 ESP: ee769f48 DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00210292 CR0: 80050033 CR2: 00000000 CR3: 016c4000 CR4: 001406f0 Call Trace: do_idle+0x2a/0x1fc cpu_startup_entry+0x1e/0x20 start_secondary+0x1d3/0x1ec startup_32_smp+0x164/0x168 I have to fool lockdep with the following: if (regs->flags & X86_EFLAGS_IF) { regs->flags &= ~X86_EFLAGS_IF; regs->ip = (unsigned long) ftrace_emulate_call_irqoff; /* Tell lockdep here we are enabling interrupts */ trace_hardirqs_on(); } else { regs->ip = (unsigned long) ftrace_emulate_call_irqon; } -- Steve