From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1044C1C32 for ; Sat, 7 Jan 2023 15:37:23 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id y1so4839363plb.2 for ; Sat, 07 Jan 2023 07:37:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=fDmGZxyb74ofTslZ4RiBsZsZcQbDnfGdhk5APp6eMRs=; b=FXtAlMhf9oxk2BS3hRJnGA+Z35M1zHAepCr/pNYQhR4hvq3jIwCsnFWZBKLESIbi86 mAE1BUitRV+Dyy9qGv7XJ/W9qZ6//X0RbZnf9YpNEJqD8c/s8JFL2YfqM4jNMLQEDNeu A0SJDsjitCduhGbS/cNZqjnF4/cYjgRwi/twkee2ZkZkbj28mdphTXi8258Dy53yVwO3 Vp0vxY9lcs7Li/rkvLqtYVqnqOHkDQAKsVV9ep7Wf9CEu6Lo4wn8FD2la7ku4UpPr2qJ u40OTEafTGWLRIBk1kHshAxzzplyWyOOSER238Q8fPJ10bclR9LOAa6Kb24Ug9hscb7c vw3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fDmGZxyb74ofTslZ4RiBsZsZcQbDnfGdhk5APp6eMRs=; b=EA9XEf3JVv13Pq9gnE9AiL20C7BPGwEkqzeLr+8wsj1KlBO8SxlawhiFoAwd6pTW71 u0/p3rPoUI5SU5OWM6oLBajEintVZygPiFhuo1TAZUKIho6RRdrF0OSu8zv8KoUtItA4 pRLNpiYZ9b/JYHxCW+C2S7b0RfMi86Uzo8DyPqgy7fE7ydrgouhp9s4+1drnMUWZQMKJ Y7NzQC3DUQxdwqqwgrgxo05CVWiE3ugHFuAF1ic5Xp5XtnAbYtVSS/fTY3SQe79vUC1X c6H9TCsUEEufJUWo7JEDPhQGEsFPIH08nYBixIwES/bO+AxXY7ZVjmNUqw6HwtEwX7b7 9Gcw== X-Gm-Message-State: AFqh2kqA3V/dfZgkZbOj9tX+0MevQl3iQ3shJA1x4Y2WolHOx4D1zB83 enKsmTLMxqKHOIVgmnJKg+Y= X-Google-Smtp-Source: AMrXdXt6+BA6TF0dfWsIgapu+pu1tyw89wv3yhd2Ckc8NeP9Pz2cJcf6EzanqlPqCmYvAHQ94Zma9A== X-Received: by 2002:a17:90a:64c9:b0:226:2f1c:f167 with SMTP id i9-20020a17090a64c900b002262f1cf167mr34292797pjm.15.1673105843435; Sat, 07 Jan 2023 07:37:23 -0800 (PST) Received: from localhost ([124.248.219.206]) by smtp.gmail.com with ESMTPSA id s90-20020a17090a69e300b0022630ba1c80sm2566397pjj.42.2023.01.07.07.37.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 07 Jan 2023 07:37:23 -0800 (PST) Date: Sat, 7 Jan 2023 07:37:19 -0800 From: Dan Li To: Mark Rutland Cc: Masahiro Yamada , Michal Marek , Nick Desaulniers , Catalin Marinas , Will Deacon , Sami Tolvanen , Kees Cook , Nathan Chancellor , Tom Rix , Peter Zijlstra , "Paul E. McKenney" , Josh Poimboeuf , Frederic Weisbecker , "Eric W. Biederman" , Marco Elver , Christophe Leroy , Song Liu , Andrew Morton , Uros Bizjak , Kumar Kartikeya Dwivedi , Juergen Gross , Luis Chamberlain , Borislav Petkov , Masami Hiramatsu , Dmitry Torokhov , Aaron Tomlin , Kalesh Singh , Yuntao Wang , Changbin Du , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev Subject: Re: [RFC/RFT] CFI: Add support for gcc CFI in aarch64 Message-ID: <20230107153719.cpuq5yrc7v67f2uy@ubuntu> References: <20221219061758.23321-1-ashimida.1990@gmail.com> Precedence: bulk X-Mailing-List: llvm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20171215 Hi Mark, Sorry for the late reply. On 01/03, Mark Rutland wrote: > On Sun, Dec 18, 2022 at 10:17:58PM -0800, Dan Li wrote: > > If there are user mode programs or other systems that want to use > > this feature, it may be more convenient to use a callback (so this > > compilation option is set to -fsanitize=cfi instead of kcfi). > > I appreciate that may be nicer for userspace, but it would be far nicer for the > kernel if we could have a kcfi mode that behaves the same as LLVM, using a BRK. > That's going to be simpler for the kernel to deal with, and should result in > nicer code / smaller binary size (for the reasons given above). > > Can we please have an LLVM-compatible KCFI mode, and have the -fsanitize=cfi be > a separate option from -fsanitize=kcfi? Ok, in the next version I will change to the same option as clang :) > > > 2. A reserved typeid (such as 0x0U on the aarch64 platform) is always > > inserted in front of functions that should not be called indirectly. > > Functions that can be called indirectly will not use this hash value, > > which prevents instructions/data before the function from being used > > as a typeid by an attacker. > > That sounds sensible, though it meanse we'll need to go audit all the assembly > without type annotations. > > I presume that "functions that should not be called indirectly" only includes > those which are not directly visible outside the compilation unit AND whose > address is never taken / escaped from the compilation unit. Is that the case? Yes. > > > 3. Some bits are ignored in the typeid to avoid conflicts between the > > typeid and the instruction set of a specific platform, thereby > > preventing an attacker from bypassing the CFI check by using the > > instruction as a typeid, such as on the aarch64 platform: > > * If the following instruction sequence exists: > > 400620: a9be7bfd stp x29, x30, [sp, #-32]! > > 400624: 910003fd mov x29, sp > > 400628: f9000bf3 str x19, [sp, #16] > > * If the expected typeid of the indirect call is exactly 0x910003fd, > > the attacker can jump to the next instruction position of any > > "mov x29,sp" instruction (such as 0x400628 here). > > Which bits exactly are ignored on arm64? > > e.g. are these encoded into UDF immediates? In aarch64, I currently ignore bit [28:27]. IUCC, according to the manual[1], it is a UDF instruction only when the upper 16 bits are all 0. But due to this has too much impact on the entropy of typeid, so I (not rigorously) only ignore 2 bits here, and most of the instruction codes covered by it belong to 'Reserved' or 'UNALLOCATED' (probably not a good idea). But as Kees said, if clang doesn't handle it here, in order to be consistent, I think it's better for gcc to not handle it when implementing kernel cfi. [1] https://developer.arm.com/documentation/ddi0602/2022-06/Index-by-Encoding?lang=en > > As a general thing, how does this work with -fpatchable-function-entry=M,N, > where N is non-zero? > > We still need to fix that for LLVM, and it would be good to align on the same behaviour. > Yeah, it makes sense. Currently, it is consistent with llvm. Taking -fpatchable-function-entry=2,1 as an example, the currently generated code is as follows: __cfi_main: .4byte 0x439d3502 .global main .section __patchable_function_entries .align 3 .8byte .LPFE3 .text .LPFE3: nop .type main, %function main: nop .LFB2: .cfi_startproc stp x29, x30, [sp, -32]! Finally, do we want to generate code like this? nop .4byte 0x439d3502 main: nop ... Thanks, Dan. > > > > -- > > 2.17.1 > > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 74E1AC46467 for ; Sat, 7 Jan 2023 15:38:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=iZd4cfFQNkrQ4aYQOUIbZLjzeZLE6OWldV1Ovf6tGcc=; b=nTWa9cURSl4aax vy8zOb4c/SMpNHO6wmfuRoCfxqrcRmrN0ejcD+jknOB9DLJ3uajd0MVbeSVRLJxSS3pjpucJ4BFd5 NYC6JZsP3CBfR2+XOu7qduOA/qSdowYOKZIG8GzKoz5DBg3Tt9PNGJnSZDB6MrCO76/nWiJkMZer5 UbjMabZqEcCz+t80JGQcq2mTmwNSoBBhoxQtbYPyBjxJ/N+fltWDTtXNFBVLjubQjny341aLWnG5T A5l6TwhS+1kFjH/2djI8fpCkdBYPd1AAf2UoGMEywrQ/WhGSxQrNDcg6YzXKa7slOHGAm0ZDUppPv 8syIoLIhpYX4CEn1HItw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pEBGE-007WlK-Ch; Sat, 07 Jan 2023 15:37:30 +0000 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pEBGA-007WkE-2X for linux-arm-kernel@lists.infradead.org; Sat, 07 Jan 2023 15:37:28 +0000 Received: by mail-pl1-x633.google.com with SMTP id jl4so4813576plb.8 for ; Sat, 07 Jan 2023 07:37:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date :message-id:reply-to; bh=fDmGZxyb74ofTslZ4RiBsZsZcQbDnfGdhk5APp6eMRs=; b=FXtAlMhf9oxk2BS3hRJnGA+Z35M1zHAepCr/pNYQhR4hvq3jIwCsnFWZBKLESIbi86 mAE1BUitRV+Dyy9qGv7XJ/W9qZ6//X0RbZnf9YpNEJqD8c/s8JFL2YfqM4jNMLQEDNeu A0SJDsjitCduhGbS/cNZqjnF4/cYjgRwi/twkee2ZkZkbj28mdphTXi8258Dy53yVwO3 Vp0vxY9lcs7Li/rkvLqtYVqnqOHkDQAKsVV9ep7Wf9CEu6Lo4wn8FD2la7ku4UpPr2qJ u40OTEafTGWLRIBk1kHshAxzzplyWyOOSER238Q8fPJ10bclR9LOAa6Kb24Ug9hscb7c vw3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fDmGZxyb74ofTslZ4RiBsZsZcQbDnfGdhk5APp6eMRs=; b=gGYp8s9DHAtFzpTbvs7K8JGxhssT0iHQ7MVu3hjPHIfL5aaDddzNdFOf2TWKwJvj8p UUrPo0MJ3ZB7CmJIiZYxwb1g7JXH/sPEsNg0wL7ZWUBDpIdPsS1HjFQMdLjSa2XkZqJL UXLUqKLKxR+Omhcul9cm2PAwkCNHbJa//pLjlPTbG9f74oBYponAvidF0axioNNdUeFH QJmpUuCya9mS8rm8ZZ5kHwaxazIUnZlBZwaJZvbDi1bdwgVlNu43ZMF6LkiDrODMu+cm zlSLzGX1g4Hy7uIvHJY0VcbwFgg11eLAO77m+eKRL9UxJjzfm8LhEfyzArVPtSU5eiYR Y29g== X-Gm-Message-State: AFqh2ko2HiWfW9f2JI/gRqDVts2sCrfqaE7LO/W9dzA+NR4U57amYRPA Vqef1ODguBsFu6pEK6Gkjnw= X-Google-Smtp-Source: AMrXdXt6+BA6TF0dfWsIgapu+pu1tyw89wv3yhd2Ckc8NeP9Pz2cJcf6EzanqlPqCmYvAHQ94Zma9A== X-Received: by 2002:a17:90a:64c9:b0:226:2f1c:f167 with SMTP id i9-20020a17090a64c900b002262f1cf167mr34292797pjm.15.1673105843435; Sat, 07 Jan 2023 07:37:23 -0800 (PST) Received: from localhost ([124.248.219.206]) by smtp.gmail.com with ESMTPSA id s90-20020a17090a69e300b0022630ba1c80sm2566397pjj.42.2023.01.07.07.37.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 07 Jan 2023 07:37:23 -0800 (PST) Date: Sat, 7 Jan 2023 07:37:19 -0800 From: Dan Li To: Mark Rutland Cc: Masahiro Yamada , Michal Marek , Nick Desaulniers , Catalin Marinas , Will Deacon , Sami Tolvanen , Kees Cook , Nathan Chancellor , Tom Rix , Peter Zijlstra , "Paul E. McKenney" , Josh Poimboeuf , Frederic Weisbecker , "Eric W. Biederman" , Marco Elver , Christophe Leroy , Song Liu , Andrew Morton , Uros Bizjak , Kumar Kartikeya Dwivedi , Juergen Gross , Luis Chamberlain , Borislav Petkov , Masami Hiramatsu , Dmitry Torokhov , Aaron Tomlin , Kalesh Singh , Yuntao Wang , Changbin Du , linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, llvm@lists.linux.dev Subject: Re: [RFC/RFT] CFI: Add support for gcc CFI in aarch64 Message-ID: <20230107153719.cpuq5yrc7v67f2uy@ubuntu> References: <20221219061758.23321-1-ashimida.1990@gmail.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20171215 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230107_073726_161420_11ACEB8F X-CRM114-Status: GOOD ( 41.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Mark, Sorry for the late reply. On 01/03, Mark Rutland wrote: > On Sun, Dec 18, 2022 at 10:17:58PM -0800, Dan Li wrote: > > If there are user mode programs or other systems that want to use > > this feature, it may be more convenient to use a callback (so this > > compilation option is set to -fsanitize=cfi instead of kcfi). > > I appreciate that may be nicer for userspace, but it would be far nicer for the > kernel if we could have a kcfi mode that behaves the same as LLVM, using a BRK. > That's going to be simpler for the kernel to deal with, and should result in > nicer code / smaller binary size (for the reasons given above). > > Can we please have an LLVM-compatible KCFI mode, and have the -fsanitize=cfi be > a separate option from -fsanitize=kcfi? Ok, in the next version I will change to the same option as clang :) > > > 2. A reserved typeid (such as 0x0U on the aarch64 platform) is always > > inserted in front of functions that should not be called indirectly. > > Functions that can be called indirectly will not use this hash value, > > which prevents instructions/data before the function from being used > > as a typeid by an attacker. > > That sounds sensible, though it meanse we'll need to go audit all the assembly > without type annotations. > > I presume that "functions that should not be called indirectly" only includes > those which are not directly visible outside the compilation unit AND whose > address is never taken / escaped from the compilation unit. Is that the case? Yes. > > > 3. Some bits are ignored in the typeid to avoid conflicts between the > > typeid and the instruction set of a specific platform, thereby > > preventing an attacker from bypassing the CFI check by using the > > instruction as a typeid, such as on the aarch64 platform: > > * If the following instruction sequence exists: > > 400620: a9be7bfd stp x29, x30, [sp, #-32]! > > 400624: 910003fd mov x29, sp > > 400628: f9000bf3 str x19, [sp, #16] > > * If the expected typeid of the indirect call is exactly 0x910003fd, > > the attacker can jump to the next instruction position of any > > "mov x29,sp" instruction (such as 0x400628 here). > > Which bits exactly are ignored on arm64? > > e.g. are these encoded into UDF immediates? In aarch64, I currently ignore bit [28:27]. IUCC, according to the manual[1], it is a UDF instruction only when the upper 16 bits are all 0. But due to this has too much impact on the entropy of typeid, so I (not rigorously) only ignore 2 bits here, and most of the instruction codes covered by it belong to 'Reserved' or 'UNALLOCATED' (probably not a good idea). But as Kees said, if clang doesn't handle it here, in order to be consistent, I think it's better for gcc to not handle it when implementing kernel cfi. [1] https://developer.arm.com/documentation/ddi0602/2022-06/Index-by-Encoding?lang=en > > As a general thing, how does this work with -fpatchable-function-entry=M,N, > where N is non-zero? > > We still need to fix that for LLVM, and it would be good to align on the same behaviour. > Yeah, it makes sense. Currently, it is consistent with llvm. Taking -fpatchable-function-entry=2,1 as an example, the currently generated code is as follows: __cfi_main: .4byte 0x439d3502 .global main .section __patchable_function_entries .align 3 .8byte .LPFE3 .text .LPFE3: nop .type main, %function main: nop .LFB2: .cfi_startproc stp x29, x30, [sp, -32]! Finally, do we want to generate code like this? nop .4byte 0x439d3502 main: nop ... Thanks, Dan. > > > > -- > > 2.17.1 > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel