From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AA69C433E0 for ; Mon, 1 Feb 2021 15:22:55 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CDC8964EA0 for ; Mon, 1 Feb 2021 15:22:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDC8964EA0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=lnVn7vcE//im/E+ZwPrlEkF53h9aw15yLwzyXgtdaew=; b=rNacIDOpD00Q/xk6784Jp43R7 HFcGFU+1ZW8xoEP5B6jlAXl7KycD8lE/nsVQjGsEjVPHgxh+jWT6vHlItRa55fkDq8Y8nWPaE1it4 n4j8j29s0As4yVCr8cQA7+eMS/7fvQlLAuDagQ7qNiVLvFjmgpSDCOn7QmEeRmWm9C0PguIjFbEv6 1mLop+rEF/qUpBwOZFgG1QdCZuHFHosw/yFuKm/r03tNkDiDJr9fSZAWzdQxqozP+iwN1lDrqR1Ge 0Y3bcysSGs8B9nYntjPvNoiDmOGnCg6RcAHdkoGxsyPlCa6Go7aaFcRuT/cvQ3JXbvmyyB7d1OdCW cTrItrQcQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6b1Q-0003qU-Ig; Mon, 01 Feb 2021 15:21:48 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l6b1O-0003py-KT for linux-arm-kernel@lists.infradead.org; Mon, 01 Feb 2021 15:21:47 +0000 Received: from [192.168.254.32] (unknown [47.187.219.45]) by linux.microsoft.com (Postfix) with ESMTPSA id 6081320B7192; Mon, 1 Feb 2021 07:21:44 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 6081320B7192 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1612192904; bh=Q9CJaRjEAzL2igcWrsiE4GiPNDboN2d2kbAu6KuTEFw=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=HFcLaIkUAJihtCGRL/7xN1BaS2VOx1DDd/2IiIexZxXrMtZci1sldKkTRfX7nnpOF hoWmimrkMaB5sPL7FlBEt9QeXoRld48INiv/SymOj9XRxI1n+3S0ihFPMhJpvj2VCb 5mPxMLJraFNAHWd+99+ZexNKOWIcdNFNYGWdUHgE= Subject: Re: [RFC PATCH 0/3] arm64: Implement reliable stack trace To: Josh Poimboeuf , Mark Brown References: <20201012172605.10715-1-broonie@kernel.org> <13095563-ff6d-b806-1bf3-efde4383456e@linux.microsoft.com> <20210128142250.GC4537@sirena.org.uk> <20210128152649.6zin3hzim3etbv2p@treble> From: "Madhavan T. Venkataraman" Message-ID: Date: Mon, 1 Feb 2021 09:21:43 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20210128152649.6zin3hzim3etbv2p@treble> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210201_102146_927574_C74E6B47 X-CRM114-Status: GOOD ( 28.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Julien Thierry , Catalin Marinas , Miroslav Benes , Will Deacon , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 1/28/21 9:26 AM, Josh Poimboeuf wrote: >> If we're trusting the compiler we can probably just do that without any >> explicit support from the compiler - it should be doing the standard >> stuff unless we explicitly ask it to and if it isn't then it might be a >> result of a mismatch in assumptions rather than a deliberate decision to >> do something non-standard. My understanding with objtool is that a big >> part of the idea is to provide a static check that the binary we end up >> with matches the assumptions that we are making so the fact that it's a >> separate implementation is important. > For C code, even if we trusted the compiler (which we don't), we still > have inline asm which the compiler doesn't have any visibility to, which > is more than capable of messing up frame pointers (we had several cases > of this in x86). > > Getting the assembler to annotate which functions are FP could be > interesting but: > > a) good luck getting the assembler folks to do that; they tend to be > insistent on being ignorant of code semantics; > > b) assembly often resembles spaghetti and the concept of a function is > quite fluid; in fact many functions aren't annotated as such. OK. Before this whole discussion, I did not know that the compiler cannot be trusted. So, it looks like objtool is definitely needed. However, I believe we can minimize the work objtool does by using a shadow stack. I read Mark Brown's response to my shadow stack email. I agree with him. The shadow stack looks promising. So, here is my suggestion for the shadow stack. This is just to start the discussion on the shadow stack. Prolog and epilog for C functions ================================= Some shadow stack prolog and epilog are needed. Let us add a new option to the compiler to generate extra no-ops at the beginning of a function for the prolog and just before return for the epilog so some other entity such as objtool can add its own prolog and epilog. This is so we don't have to trust the compiler and can maintain our own prolog and epilog. Objtool will check for the no-ops. If they are present, it will replace the no-ops with the shadow stack prolog and epilog. It can also check the frame pointer prolog and epilog. Then, it will set a flag in the symbol table entry of the function to indicate that the function has a proper prolog and epilog. Prolog and epilog for assembly functions ======================================== The no-ops and frame pointer prolog and epilog can be added to assembly functions manually. Objtool will process them as above. Decoding ======== To do all this, objtool has to decode only the following instructions. - no-op - return instruction - store register pair in frame pointer prolog - load register pair in frame pointer epilog This simplifies the objtool part a lot. AFAIK, all instructions in ARM64 are 32 bits wide. So, objtool does not have to decode an instruction to know its length. Objtool has to scan a function for the return instruction to know the location(s) of the epilog. I guess objtool still has to figure out unreachable code, alternatives and all that sort of thing. But that logic is already there. Will alternatives every contain the return instruction? If not, objtool can skip processing alternatives. Shadow stack ============ Allocation ---------- Allocate the shadow stack and the regular stack adjacent to each other or at a fixed distance from each other (may be a guard page in-between?). This is so the shadow stack can be accessed from any stack address using a simple calculation. Top of shadow stack ------------------- We can either use a compact shadow stack or a parallel shadow stack. There are trade-offs in each. In either case, we need to know where the top of the shadow stack is. We could designate a register for that. But then, we have to make sure that the register is not used anywhere else. That is a problem. The alternative is to reserve the first 8 bytes in the shadow stack for the top of stack pointer. Prolog ====== Push the previous frame pointer and link register (register that contains the return address) on the shadow stack. Epilog ====== Pop the shadow stack. Scratch registers ================= A couple of scratch registers have to be used for the prolog and epilog. Perhaps x16 and x17 can be used? They are supposed to be caller-saved intra procedure call scratch registers. Unwinder logic ============== The unwinder will walk the stack using frame pointers like it does currently. As it unwinds the regular stack, it will also unwind the shadow stack: However, at each step, it needs to perform some additional checks: symbol = lookup symbol table entry for pc if (!symbol) return -EINVAL; if (symbol does not have proper prolog and epilog) return -EINVAL; Compare the information stored on the regular stack with that stored on the shadow stack. if (the info does not match) return -EINVAL; Success for this frame longjmp style situations ======================== Let us say we unwind the regular stack by some number of frames. We need to unwind the shadow stack as well. After unwinding the regular stack first, we can just take the current frame pointer of the regular stack and locate the entry on the shadow stack that matches with that and make that entry the top of the shadow stack. Summary ======= I think a shadow stack will work perfectly for livepatch and will need only a little help from objtool. I don't know what the performance will be like though. I have probably missed some corner cases. Please comment. Madhavan _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel