From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58351C433F5 for ; Fri, 15 Oct 2021 19:13:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3C24160F11 for ; Fri, 15 Oct 2021 19:13:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242659AbhJOTQB (ORCPT ); Fri, 15 Oct 2021 15:16:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242668AbhJOTPy (ORCPT ); Fri, 15 Oct 2021 15:15:54 -0400 Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [IPv6:2a00:1450:4864:20::131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D345C061570 for ; Fri, 15 Oct 2021 12:13:46 -0700 (PDT) Received: by mail-lf1-x131.google.com with SMTP id n8so45516189lfk.6 for ; Fri, 15 Oct 2021 12:13:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HBzr0B3tGGPW4EBa2LvZed6scqvnq5Oml+sl03YQDio=; b=BA/5lSdu+5ROrNUsKhvi+0xgDXMbeH1IeW/bUOWfcmhvn1b5gplsN/lhjVCre6mi7A T02zPFwlJIn5jPNjwH4oOtQyucvs6lBBcy+OrsPY3YL/FYdY0J233zlFzQbbtj4HgTc2 L9417guhXF0JmPaEr+m4IC2Neu+S/CP0pe8W1DeWsGEaRfcbKuXMoiVVLIsmZUAZYhDW i839MhmZ7hNt4oiL/fD/Ou4ujKkHeGGDRRmS1Lm9czlqguFwu1qDQyHbuiiIuclaTR8n zgTo8mkTKOfB8QZ+Z9qaTsaUNQSAUoM7AdYTjJhuqe3g1u8w9OuBkFHS5pOcW6IIJz3a cmzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HBzr0B3tGGPW4EBa2LvZed6scqvnq5Oml+sl03YQDio=; b=knKxQy4e2pT148EqRtCag84syxUzG5f/n8Hm/QMuysXHlkVFjFVQazIXo+eZ6QEF0o vI9OzKKl/mMV7ZmP8mrRtnj98hnoj7UaJi0UiEbMNzwAiilBMP9fbEz0dBLF5Cek2zsM 3YH+dSk//hIJi8ZWzYRrwZGTgGYJNeyKMJLp7lXvUqklFp0HcpE6dRHAUFbE1EaepZvA J++bAEdzVbdnL7Ibkgs9dexOqVgzuRYROu8YAlcP1EsaVSHj3XeZnwHYGr9l0D5bWpvl ZmZCTySAMop73/E+TtVsMaYOmFXdV4L1zOGVHXKNTbf100i3z9LxOzj9LaqHKqbl0Ard yyTQ== X-Gm-Message-State: AOAM5319cTR0QJ80Lrr3ykJAGoqzvK32pcHB1U2Qola9iGnnzQU709VS Oznfauw1LoeXavi/XUoIeaPr4bUEjo5EVvhLRd+JmQ== X-Google-Smtp-Source: ABdhPJz8/p6IXPlhwfOmIFLJc3rweNG7wsVdft9QVM6g2UNGOxECgfrMdFD6Bhxm1ySa6xlaQxoW0bTsBpferjnjiCE= X-Received: by 2002:a05:6512:3ba3:: with SMTP id g35mr13415925lfv.651.1634325224740; Fri, 15 Oct 2021 12:13:44 -0700 (PDT) MIME-Version: 1.0 References: <1634167668-60198-1-git-send-email-ashimida@linux.alibaba.com> <722d9662-e27c-2efb-e8cf-d505b6950475@linux.alibaba.com> In-Reply-To: <722d9662-e27c-2efb-e8cf-d505b6950475@linux.alibaba.com> From: Nick Desaulniers Date: Fri, 15 Oct 2021 12:13:33 -0700 Message-ID: Subject: Re: [PATCH] [PATCH V4]ARM64: SCS: Add gcc plugin to support Shadow Call Stack To: Dan Li Cc: masahiroy@kernel.org, michal.lkml@markovi.net, catalin.marinas@arm.com, will@kernel.org, keescook@chromium.org, nathan@kernel.org, tglx@linutronix.de, akpm@linux-foundation.org, samitolvanen@google.com, frederic@kernel.org, rppt@kernel.org, mark.rutland@arm.com, yifeifz2@illinois.edu, rostedt@goodmis.org, viresh.kumar@linaro.org, andreyknvl@gmail.com, colin.king@canonical.com, ojeda@kernel.org, luc.vanoostenryck@gmail.com, elver@google.com, nivedita@alum.mit.edu, ardb@kernel.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hardening@vger.kernel.org, clang-built-linux@googlegroups.com Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Oct 15, 2021 at 11:29 AM Dan Li wrote: > > > > On 10/15/21 2:44 AM, Nick Desaulniers wrote: > > On Wed, Oct 13, 2021 at 4:28 PM Dan Li wrote: > >> --- a/include/linux/compiler-gcc.h > >> +++ b/include/linux/compiler-gcc.h > >> @@ -50,6 +50,10 @@ > >> #define __latent_entropy __attribute__((latent_entropy)) > >> #endif > >> > >> +#if defined(SHADOW_CALL_STACK_PLUGIN) && !defined(__CHECKER__) > >> +#define __noscs __attribute__((no_shadow_call_stack)) > >> +#endif > > > > Cool this is a nice addition, and something I don't think that clang > > has. For any new feature, having a function attribute to disable it > > at the function granularity is nice, and plays better with LTO than -f > > group flags. Though that begs the question: what happens if a __noscs > > callee is inlined into a non-__noscs caller, or vice versa? > Thanks Nick, > > According to my understanding, all inline optimizations in gcc should > happen before inserting scs insns (scs and paciasp/autiasp use the > same insertion point). Therefore, the check for the __noscs attribute > will also occur after all inlining is completed. > > As in the following example: > - Since __noscs attribute is specified, scs_test1 does not insert scs insns > - Since normal functions scs_test2/3 uses x30, it needs to insert scs insns > - Since __noscs attribute is specified, scs_test4 after inlining does not > need to insert scs insns > > __always_inline __noscs void scs_test1(void) > { > asm volatile("mov x1, x1\n\t":::"x30"); > } > > //scs insns inserted after function inline > void scs_test2(void) > { > scs_test1(); > } That may be surprising to developers. Perhaps __always_inline on scs_test1 is distracting this test case, but I suspect it may not make a difference. This particular issue comes up time and again with stack protectors; ie. the callee is marked no stack protector, then gets inlined into a caller and suddenly gets a stack protector. > > __always_inline void scs_test3(void) > { > asm volatile("mov x3, x3\n\t":::"x30"); > } > > //no scs insns inserted > __noscs void scs_test4(void) > { > scs_test3(); > } > > ffff800010012900 : > ffff800010012900: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff800010012904: 910003fd mov x29, sp > ffff800010012908: aa0103e1 mov x1, x1 > ffff80001001290c: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012910: d65f03c0 ret > > ffff800010012914 : > ffff800010012914: f800865e str x30, [x18], #8 > ffff800010012918: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff80001001291c: 910003fd mov x29, sp > ffff800010012920: aa0103e1 mov x1, x1 > ffff800010012924: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012928: f85f8e5e ldr x30, [x18, #-8]! > ffff80001001292c: d65f03c0 ret > > ffff800010012930 : > ffff800010012930: f800865e str x30, [x18], #8 > ffff800010012934: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff800010012938: 910003fd mov x29, sp > ffff80001001293c: aa0303e3 mov x3, x3 > ffff800010012940: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012944: f85f8e5e ldr x30, [x18, #-8]! > ffff800010012948: d65f03c0 ret > ffff80001001294c: d503201f nop > > ffff800010012950 : > ffff800010012950: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff800010012954: 910003fd mov x29, sp > ffff800010012958: aa0303e3 mov x3, x3 > ffff80001001295c: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012960: d65f03c0 ret > > I noticed that __noscs isn't actually applied anywhere in the kernel, > > yet, at least in this series. Were there any places necessary that > > you've found thus far? > At present, I have not found a function that must use the __noscs > attribute in the kernel. I have only used this attribute in test cases. -- Thanks, ~Nick Desaulniers From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E13C9C433FE for ; Fri, 15 Oct 2021 19:15:00 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AA82B60F11 for ; Fri, 15 Oct 2021 19:15:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org AA82B60F11 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Igb39uH79fZjL7BoJ/4UjIpVrtJa5f8XQQFk9thMXGs=; b=EgSnwomgzTob+s Lo9YwIepzWEf0vZVOLft11ARKZ6nc7NIkwjPbaYMRSs0pFWCkDjvf9S/WTztVArMEz3/RV2NX46b8 qwuX8xCijZ4Rc7pYzXrMBctJVyPK0ksvRmBzqj9AYAHSXf1Tqu50eShsmZxjuejRoVDKRaBiHbDKD e8z7/oBcm+E8CdG3dfQGZOkC5mqo+X2sRPVKvWdo0zLHrVDi+JP/chYsdAn/mq929UR/1EqOMu9pY ii3IKaEPAN77xQIispGepntmLEq0vergd52OPp23GJKWnXu0TICLJXw1QmAXZjn9gkJMpev7bHeS+ bVHoowBkWAPga8FcVRDQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mbSeL-008am1-Le; Fri, 15 Oct 2021 19:13:49 +0000 Received: from mail-lf1-x134.google.com ([2a00:1450:4864:20::134]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mbSeI-008alV-M9 for linux-arm-kernel@lists.infradead.org; Fri, 15 Oct 2021 19:13:48 +0000 Received: by mail-lf1-x134.google.com with SMTP id n8so45516186lfk.6 for ; Fri, 15 Oct 2021 12:13:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=HBzr0B3tGGPW4EBa2LvZed6scqvnq5Oml+sl03YQDio=; b=BA/5lSdu+5ROrNUsKhvi+0xgDXMbeH1IeW/bUOWfcmhvn1b5gplsN/lhjVCre6mi7A T02zPFwlJIn5jPNjwH4oOtQyucvs6lBBcy+OrsPY3YL/FYdY0J233zlFzQbbtj4HgTc2 L9417guhXF0JmPaEr+m4IC2Neu+S/CP0pe8W1DeWsGEaRfcbKuXMoiVVLIsmZUAZYhDW i839MhmZ7hNt4oiL/fD/Ou4ujKkHeGGDRRmS1Lm9czlqguFwu1qDQyHbuiiIuclaTR8n zgTo8mkTKOfB8QZ+Z9qaTsaUNQSAUoM7AdYTjJhuqe3g1u8w9OuBkFHS5pOcW6IIJz3a cmzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HBzr0B3tGGPW4EBa2LvZed6scqvnq5Oml+sl03YQDio=; b=wB4F4Xh+Dx69tC7ehOpZisPC+VR9NSCpgsYxcbDWcVDgAxPnbfadJ/eADdjHwwy+gF z60KOeWg+WCWZcvC7Nfvtor0jiyyOoGKQtVr3MC52ZhUQY5K1cHJCgBe2u0qDzB2ixnZ feMyAynZp/ZXa6OTRksmeMmZ+A1W4CBOyfvGi88dsg4gsx3jCIJc4Lm+9frJlzTGtucC cMhUbmbna3m0FixxCZisFhBRrIwX7eWBl8WRMkFWS4oGkEHiGfQvIQmKHRx6vFvaOtg6 HIROG0cOna48VodRJSwf9ut+434PRj0l4CFEdz2X1aGFUZPLLnbx1FyN1z3immhA2Maj bukw== X-Gm-Message-State: AOAM530ju3MoztgtIX7ZfwIF6bIkEqDFoK2278a0AdDXPtT6aQ7s5K/c gEb9dZtFGMra60PrnBCkDQnvWfyZn5lLfSyIF/0HxQ== X-Google-Smtp-Source: ABdhPJz8/p6IXPlhwfOmIFLJc3rweNG7wsVdft9QVM6g2UNGOxECgfrMdFD6Bhxm1ySa6xlaQxoW0bTsBpferjnjiCE= X-Received: by 2002:a05:6512:3ba3:: with SMTP id g35mr13415925lfv.651.1634325224740; Fri, 15 Oct 2021 12:13:44 -0700 (PDT) MIME-Version: 1.0 References: <1634167668-60198-1-git-send-email-ashimida@linux.alibaba.com> <722d9662-e27c-2efb-e8cf-d505b6950475@linux.alibaba.com> In-Reply-To: <722d9662-e27c-2efb-e8cf-d505b6950475@linux.alibaba.com> From: Nick Desaulniers Date: Fri, 15 Oct 2021 12:13:33 -0700 Message-ID: Subject: Re: [PATCH] [PATCH V4]ARM64: SCS: Add gcc plugin to support Shadow Call Stack To: Dan Li Cc: masahiroy@kernel.org, michal.lkml@markovi.net, catalin.marinas@arm.com, will@kernel.org, keescook@chromium.org, nathan@kernel.org, tglx@linutronix.de, akpm@linux-foundation.org, samitolvanen@google.com, frederic@kernel.org, rppt@kernel.org, mark.rutland@arm.com, yifeifz2@illinois.edu, rostedt@goodmis.org, viresh.kumar@linaro.org, andreyknvl@gmail.com, colin.king@canonical.com, ojeda@kernel.org, luc.vanoostenryck@gmail.com, elver@google.com, nivedita@alum.mit.edu, ardb@kernel.org, linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-hardening@vger.kernel.org, clang-built-linux@googlegroups.com X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211015_121346_751286_8405D50A X-CRM114-Status: GOOD ( 25.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Oct 15, 2021 at 11:29 AM Dan Li wrote: > > > > On 10/15/21 2:44 AM, Nick Desaulniers wrote: > > On Wed, Oct 13, 2021 at 4:28 PM Dan Li wrote: > >> --- a/include/linux/compiler-gcc.h > >> +++ b/include/linux/compiler-gcc.h > >> @@ -50,6 +50,10 @@ > >> #define __latent_entropy __attribute__((latent_entropy)) > >> #endif > >> > >> +#if defined(SHADOW_CALL_STACK_PLUGIN) && !defined(__CHECKER__) > >> +#define __noscs __attribute__((no_shadow_call_stack)) > >> +#endif > > > > Cool this is a nice addition, and something I don't think that clang > > has. For any new feature, having a function attribute to disable it > > at the function granularity is nice, and plays better with LTO than -f > > group flags. Though that begs the question: what happens if a __noscs > > callee is inlined into a non-__noscs caller, or vice versa? > Thanks Nick, > > According to my understanding, all inline optimizations in gcc should > happen before inserting scs insns (scs and paciasp/autiasp use the > same insertion point). Therefore, the check for the __noscs attribute > will also occur after all inlining is completed. > > As in the following example: > - Since __noscs attribute is specified, scs_test1 does not insert scs insns > - Since normal functions scs_test2/3 uses x30, it needs to insert scs insns > - Since __noscs attribute is specified, scs_test4 after inlining does not > need to insert scs insns > > __always_inline __noscs void scs_test1(void) > { > asm volatile("mov x1, x1\n\t":::"x30"); > } > > //scs insns inserted after function inline > void scs_test2(void) > { > scs_test1(); > } That may be surprising to developers. Perhaps __always_inline on scs_test1 is distracting this test case, but I suspect it may not make a difference. This particular issue comes up time and again with stack protectors; ie. the callee is marked no stack protector, then gets inlined into a caller and suddenly gets a stack protector. > > __always_inline void scs_test3(void) > { > asm volatile("mov x3, x3\n\t":::"x30"); > } > > //no scs insns inserted > __noscs void scs_test4(void) > { > scs_test3(); > } > > ffff800010012900 : > ffff800010012900: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff800010012904: 910003fd mov x29, sp > ffff800010012908: aa0103e1 mov x1, x1 > ffff80001001290c: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012910: d65f03c0 ret > > ffff800010012914 : > ffff800010012914: f800865e str x30, [x18], #8 > ffff800010012918: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff80001001291c: 910003fd mov x29, sp > ffff800010012920: aa0103e1 mov x1, x1 > ffff800010012924: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012928: f85f8e5e ldr x30, [x18, #-8]! > ffff80001001292c: d65f03c0 ret > > ffff800010012930 : > ffff800010012930: f800865e str x30, [x18], #8 > ffff800010012934: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff800010012938: 910003fd mov x29, sp > ffff80001001293c: aa0303e3 mov x3, x3 > ffff800010012940: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012944: f85f8e5e ldr x30, [x18, #-8]! > ffff800010012948: d65f03c0 ret > ffff80001001294c: d503201f nop > > ffff800010012950 : > ffff800010012950: a9bf7bfd stp x29, x30, [sp, #-16]! > ffff800010012954: 910003fd mov x29, sp > ffff800010012958: aa0303e3 mov x3, x3 > ffff80001001295c: a8c17bfd ldp x29, x30, [sp], #16 > ffff800010012960: d65f03c0 ret > > I noticed that __noscs isn't actually applied anywhere in the kernel, > > yet, at least in this series. Were there any places necessary that > > you've found thus far? > At present, I have not found a function that must use the __noscs > attribute in the kernel. I have only used this attribute in test cases. -- Thanks, ~Nick Desaulniers _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel