From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CBA6C4363A for ; Wed, 28 Oct 2020 08:38:15 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9DF6C2462E for ; Wed, 28 Oct 2020 08:38:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="jpbPmvpm"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=kernel.org header.i=@kernel.org header.b="fPiYVwPD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DF6C2462E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=o5CRU5HX47LwnrClGoe+RXCa2bOvSYbQNrQoCne/8Wk=; b=jpbPmvpmqP6198qm8iZUTk6bd GG9kkmgYvRhc+Tosn3icdmSUPU3tT++ePZPA2xxThZ4P4Uke9GMFG84R+fuZYtGKZ3UIOWrHvB0Is L1+w0tpYchrmNjk0SwiXG8c/692vZ9p/twJWw+wmISwZRDi5LGEyd/Z/qtADzPuQ8CIEbm62H6Oqt Q/eqTwZhWnp++LdDwG7sDBpM5qGHxSMASt69BtiBDKp6+58V/qsuSVHRzC3YYKgJbXSLTWH0jx/TU 19ZLlwiJwKU8IzF7kwyC+ko+nxu+d9SCwm4vZH9ojrgT5EQXlMaq/z3OnVGD5lWkRsFrC1nsVyOYP kddkMxq9w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kXgx0-0002Gi-C2; Wed, 28 Oct 2020 08:36:58 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kXgwr-0002E9-Nw for linux-arm-kernel@lists.infradead.org; Wed, 28 Oct 2020 08:36:51 +0000 Received: from willie-the-truck (236.31.169.217.in-addr.arpa [217.169.31.236]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E57AF2465B; Wed, 28 Oct 2020 08:36:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1603874209; bh=wwKuRzNW3omr4pLPmb4gRrjZ4cW6YNhF1S1JPbChWHI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=fPiYVwPDrt+4HgrAVxrMqxEkbTrRCHmocs5oayS8szOtg0JfA9v9d5OxcFvQD4Abz IapGYO7zSEyCWGeygTCWzqAN4njvLEQNlMH6OX2ODD9DozEt5xQJrksl4j2IFxSyNL k4d+a7orBfi3Ot8KMAARJf1Nhsk/hx4aLEIDgGMM= Date: Wed, 28 Oct 2020 08:36:44 +0000 From: Will Deacon To: Jean-Philippe Brucker Subject: Re: [PATCH] arm64: Fix early single-stepping Message-ID: <20201028083643.GA27678@willie-the-truck> References: <20201026172907.1468294-1-jean-philippe@linaro.org> <20201027191318.aba935f7ccf00af9acd89388@kernel.org> <20201027194258.43b157ac0bbccd918fc8756a@kernel.org> <20201027115909.GB1514990@myrica> <20201027123317.GA26351@willie-the-truck> <20201027224922.7b032857d53bbfdc4484f768@kernel.org> <20201028082820.GA2328726@myrica> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20201028082820.GA2328726@myrica> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201028_043649_972395_7F41E2F7 X-CRM114-Status: GOOD ( 33.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: catalin.marinas@arm.com, Masami Hiramatsu , linux-arm-kernel@lists.infradead.org, dianders@chromium.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Oct 28, 2020 at 09:28:20AM +0100, Jean-Philippe Brucker wrote: > On Tue, Oct 27, 2020 at 10:49:22PM +0900, Masami Hiramatsu wrote: > > On Tue, 27 Oct 2020 12:33:18 +0000 > > Will Deacon wrote: > > > > > On Tue, Oct 27, 2020 at 12:59:09PM +0100, Jean-Philippe Brucker wrote: > > > > On Tue, Oct 27, 2020 at 07:42:58PM +0900, Masami Hiramatsu wrote: > > > > > On Tue, 27 Oct 2020 19:13:18 +0900 > > > > > Masami Hiramatsu wrote: > > > > > > > > > > > On Mon, 26 Oct 2020 18:29:09 +0100 > > > > > > Jean-Philippe Brucker wrote: > > > > > > > > > > > > > To use debug features such as single-step, the OS lock must be unlocked > > > > > > > in the debug registers. Currently this is done in postcore_initcall > > > > > > > which is now too late. > > > > > > > > > > > > > > Commit 36dadef23fcc ("kprobes: Init kprobes in early_initcall") enabled > > > > > > > using kprobes from early_initcall, when OS lock is still locked. So when > > > > > > > kprobe attempts to single-step a patched instruction, instead of > > > > > > > trapping, execution continues until it throws an undef exception: > > > > > > > > > > > > > > [ 0.064233] Kprobe smoke test: started > > > > > > > [ 0.151133] ------------[ cut here ]------------ > > > > > > > [ 0.151458] kernel BUG at arch/arm64/kernel/traps.c:406! > > > > > > > [ 0.151812] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP > > > > > > > ... > > > > > > > [ 0.162689] Call trace: > > > > > > > [ 0.163014] do_undefinstr+0x1d4/0x1f4 > > > > > > > [ 0.163336] el1_sync_handler+0xbc/0x140 > > > > > > > [ 0.163839] el1_sync+0x80/0x100 > > > > > > > [ 0.164154] 0xffffffc01001d004 > > > > > > > [ 0.164527] init_kprobes+0x13c/0x154 > > > > > > > [ 0.164968] do_one_initcall+0x54/0x2e0 > > > > > > > [ 0.165322] kernel_init_freeable+0xf4/0x258 > > > > > > > [ 0.165783] kernel_init+0x20/0x12c > > > > > > > [ 0.166117] ret_from_fork+0x10/0x30 > > > > > > > [ 0.166595] Code: 97ffff53 a9425bf5 17ffff9b f9001bf7 (d4210000) > > > > > > > [ 0.167084] ---[ end trace 36778fdf576e9a79 ]--- > > > > > > > > > > > > > > To fix this, unlock the OS lock as early as possible. Do it in > > > > > > > traps_init() for CPU0, since KGDB wants to use single-step from that > > > > > > > point on according to commit b322c65f8ca3 ("arm64: Call > > > > > > > debug_traps_init() from trap_init() to help early kgdb"). > > > > > > > For secondary CPUs, setup the CPU hotplug handler at early_initcall. > > > > > > > > > > > > > > Fixes: 36dadef23fcc ("kprobes: Init kprobes in early_initcall") > > > > > > > Signed-off-by: Jean-Philippe Brucker > > > > > > > > > > > > Hi Jean, > > > > > > > > > > > > How have you confirmed this fixes the issue? > > > > > > On my environment, this doesn't fix the issue. > > > > > > > > > > Oops, it was my mistake. I missed to boot up with Xen. (so I find another bug...) > > > > > Anyway this works for me too. > > > > > > > > No worries :) Although now I've been wondering whether it would be better > > > > to just disable the OS lock lazily, on the first call to > > > > enable_debug_monitors(). It might add a tiny performance penalty but would > > > > avoid this problem reappearing if one of the debugger needs to start even > > > > earlier in the future. > > > > > > I'm still uneasy about enabling KDE with the watchpoint registers in an > > > unknown state, so I think this needs more work. > > > > Hmm, how we reset it in the early stage? reset watchpoint registers first? > > Yes, I think so. Same order problem as the OS lock, they need to be reset > before enable_debug_monitors(). On CPU0 that would be before > early_initcall and for secondaries the hotplug notifier needs to be > installed earlier as well. I'll send a v2. Cheers. An alternative (which I think would be better in the long run anyway) would be to avoid using hardware step in kprobes and instead rely on a BRK instruction to trap after running the trampoline. Then we just need to ensure that the BRK handlers are up and running in time (which might require an early handler, like we have for KASAN). Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel