From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51F9CC4332F for ; Fri, 22 Apr 2022 11:34:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346437AbiDVLgw (ORCPT ); Fri, 22 Apr 2022 07:36:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45406 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242181AbiDVLgu (ORCPT ); Fri, 22 Apr 2022 07:36:50 -0400 Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E04056404 for ; Fri, 22 Apr 2022 04:33:56 -0700 (PDT) Received: by mail-wr1-x42d.google.com with SMTP id k22so10647146wrd.2 for ; Fri, 22 Apr 2022 04:33:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=references:user-agent:from:to:cc:subject:date:in-reply-to :message-id:mime-version; bh=uz+KcNIj+v9WSFSUjytskdEFRoZIQjG/EVCHIE7PoXA=; b=gzbQuJCj0I2T04uY0UYnVWLijPTDjp9cU8gjljlQbHYCpA8RUc0SI+1iOS8JCVDX30 wHYpTjb2TVGttz0rLW8wUREcv0cSVao2eWrMuveVbBcuvb69FNpN1EiELHQyz7N6bS5c 3TUxO7P8UkMUTNvh+TLzQ0YBXD9IfwQPSNMvk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:references:user-agent:from:to:cc:subject:date :in-reply-to:message-id:mime-version; bh=uz+KcNIj+v9WSFSUjytskdEFRoZIQjG/EVCHIE7PoXA=; b=HuBNXeH3jkqXSruZBi2KuG4vznjPGvTO8puPyilw9INZ9iI3ODZKALK86HVpaAEEg3 ETp5R7+5PoTQ3e7sE5cgAOiulUlaAElOT7yFC7OF85N5F+mSHvfKj1jO2896SjoFwRVU qsKLSTyk1fx4vVkItne/wdWCzZ1Pmx/4w0qChYnhTcf/DoxwY1at86yk6rMvcdbqwVEv n1InTVdzM+3Fc1Kle2HBgBV3sVGP5sRVWvhNJW4bo97Mni4wdlLiqY0w0cDJ5lrLgIUl kSXbRd9trJcfUSM+hUPihUSUxx7h7pmzFioGwPNBekRbzSzQPWQ/z5lsGFBj8C1/4YfF m8Og== X-Gm-Message-State: AOAM532XYgzPlQeIpRJnJrvlTNscR0F574bbIqbArHLJCQNvUf+pz8Bs UU1gHx0OLJLuNa8q7vElNUYyeg== X-Google-Smtp-Source: ABdhPJytQeZpvy34ZOAAMQB7w6jF7X6Nf/OgCkIilBTulJnvg3gtBspcR68ouVwuJUzFMunZvSQqjw== X-Received: by 2002:a5d:4e08:0:b0:20a:8f9e:beef with SMTP id p8-20020a5d4e08000000b0020a8f9ebeefmr3571401wrt.8.1650627235044; Fri, 22 Apr 2022 04:33:55 -0700 (PDT) Received: from cloudflare.com (79.184.126.143.ipv4.supernova.orange.pl. [79.184.126.143]) by smtp.gmail.com with ESMTPSA id r25-20020adfa159000000b0020ac9758f17sm1338619wrr.23.2022.04.22.04.33.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Apr 2022 04:33:54 -0700 (PDT) References: <20220414162220.1985095-1-xukuohai@huawei.com> <20220414162220.1985095-5-xukuohai@huawei.com> User-agent: mu4e 1.6.10; emacs 27.2 From: Jakub Sitnicki To: Xu Kuohai Cc: bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, Catalin Marinas , Will Deacon , Steven Rostedt , Ingo Molnar , Daniel Borkmann , Alexei Starovoitov , Zi Shen Lim , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , "David S . Miller" , Hideaki YOSHIFUJI , David Ahern , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, hpa@zytor.com, Shuah Khan , Mark Rutland , Ard Biesheuvel , Pasha Tatashin , Peter Collingbourne , Daniel Kiss , Sudeep Holla , Steven Price , Marc Zyngier , Mark Brown , Kumar Kartikeya Dwivedi , Delyan Kratunov , kernel-team@cloudflare.com Subject: Re: [PATCH bpf-next v2 4/6] bpf, arm64: Impelment bpf_arch_text_poke() for arm64 Date: Fri, 22 Apr 2022 12:54:02 +0200 In-reply-to: <20220414162220.1985095-5-xukuohai@huawei.com> Message-ID: <87levxfj32.fsf@cloudflare.com> MIME-Version: 1.0 Content-Type: text/plain Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Xu, Thanks for working on this. We are also looking forward to using fentry hooks on arm64. In particular, attaching to entry/exit into/from XDP progs. On Thu, Apr 14, 2022 at 12:22 PM -04, Xu Kuohai wrote: > Impelment bpf_arch_text_poke() for arm64, so bpf trampoline code can use > it to replace nop with jump, or replace jump with nop. > > Signed-off-by: Xu Kuohai > Acked-by: Song Liu > --- > arch/arm64/net/bpf_jit_comp.c | 52 +++++++++++++++++++++++++++++++++++ > 1 file changed, 52 insertions(+) > > diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c > index 8ab4035dea27..1a1c3ea75ee2 100644 > --- a/arch/arm64/net/bpf_jit_comp.c > +++ b/arch/arm64/net/bpf_jit_comp.c > @@ -9,6 +9,7 @@ > > #include > #include > +#include > #include > #include > #include > @@ -18,6 +19,7 @@ > #include > #include > #include > +#include > #include > > #include "bpf_jit.h" > @@ -1529,3 +1531,53 @@ void bpf_jit_free_exec(void *addr) > { > return vfree(addr); > } > + > +static int gen_branch_or_nop(enum aarch64_insn_branch_type type, void *ip, > + void *addr, u32 *insn) > +{ > + if (!addr) > + *insn = aarch64_insn_gen_nop(); > + else > + *insn = aarch64_insn_gen_branch_imm((unsigned long)ip, > + (unsigned long)addr, > + type); > + > + return *insn != AARCH64_BREAK_FAULT ? 0 : -EFAULT; > +} > + > +int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type, > + void *old_addr, void *new_addr) > +{ > + int ret; > + u32 old_insn; > + u32 new_insn; > + u32 replaced; > + enum aarch64_insn_branch_type branch_type; > + > + if (poke_type == BPF_MOD_CALL) > + branch_type = AARCH64_INSN_BRANCH_LINK; This path, bpf_arch_text_poke(, BPF_MOD_CALL, ...), is what we hit when attaching a BPF program entry. It is exercised by selftest #232 xdp_bpf2bpf. However, with this patchset alone it will not work because we don't emit, yet, the ftrace patch (MOV X9, LR; NOP) as a part of BPF prog prologue, like ftrace_init_nop() does. So patching attempt will fail. I think that is what you mentioned to in your reply to Hou [1] So my question is - is support for attaching to BPF progs in scope for this patchset? If no, then perhaps it would be better for now to fail early with something like -EOPNOTSUPP when poke_type is BPF_MOD_CALL, rather then attempt to patch the code. If you plan to enable it as a part of this patchset, then I've given it a quick try, and it seems that not a lot is needed get fentry to BPF attachment to work. I'm including the diff for my quick and dirty attempt below. With that patch on top, the xdp_bpf2bpf tests pass: #232 xdp_bpf2bpf:OK [1] https://lore.kernel.org/bpf/d8c4f1fb-a020-9457-44e2-dc63982a9213@huawei.com/ > + else > + branch_type = AARCH64_INSN_BRANCH_NOLINK; > + > + if (gen_branch_or_nop(branch_type, ip, old_addr, &old_insn) < 0) > + return -EFAULT; > + > + if (gen_branch_or_nop(branch_type, ip, new_addr, &new_insn) < 0) > + return -EFAULT; > + > + mutex_lock(&text_mutex); > + if (aarch64_insn_read(ip, &replaced)) { > + ret = -EFAULT; > + goto out; > + } > + > + if (replaced != old_insn) { > + ret = -EFAULT; > + goto out; > + } > + > + ret = aarch64_insn_patch_text_nosync((void *)ip, new_insn); > +out: > + mutex_unlock(&text_mutex); The body of this critical section is identical as ftrace_modify_code(). Perhaps we could export it and reuse? > + return ret; > +} --- diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 5f6bd755050f..94d8251500ab 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -240,9 +240,9 @@ static bool is_lsi_offset(int offset, int scale) /* Tail call offset to jump into */ #if IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) || \ IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL) -#define PROLOGUE_OFFSET 9 +#define PROLOGUE_OFFSET 11 #else -#define PROLOGUE_OFFSET 8 +#define PROLOGUE_OFFSET 10 #endif static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf) @@ -281,6 +281,10 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf) * */ + /* Set up ftrace patch (initially in disabled state) */ + emit(A64_MOV(1, A64_R(9), A64_LR), ctx); + emit(A64_NOP, ctx); + /* Sign lr */ if (IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL)) emit(A64_PACIASP, ctx); @@ -1888,10 +1892,16 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type, u32 replaced; enum aarch64_insn_branch_type branch_type; - if (poke_type == BPF_MOD_CALL) + if (poke_type == BPF_MOD_CALL) { branch_type = AARCH64_INSN_BRANCH_LINK; - else + /* + * Adjust addr to point at the BL in the callsite. + * See ftrace_init_nop() for the callsite sequence. + */ + ip = (void *)((unsigned long)ip + AARCH64_INSN_SIZE); + } else { branch_type = AARCH64_INSN_BRANCH_NOLINK; + } if (gen_branch_or_nop(branch_type, ip, old_addr, &old_insn) < 0) return -EFAULT; From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D4068C433F5 for ; Fri, 22 Apr 2022 11:35:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-ID:In-reply-to: Date:Subject:Cc:To:From:References:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=thghAyBnw0n5EDAUcE+7CtoCwLhlprpiHPWKrtbli3c=; b=T/AZvMN/XVGcVb 4BurFyqGLoTt03ALCGsXm3Dv/hOwhXT6ZyJvZj4lw+fama18Fwh0Ro1Oj8XXTUX6ShR0LmWYwad2E H6mgM9bfK2UfwjjcS/0RCsXS5cv+kTXvfeAMIEUOUyln2PT4UXN4rShnAS9cyvz+38fbGG8GsCOJ1 6u7eWr/Jru6iH1oTAnL/cHTKKG6tfceJGeCb+fYHShAfbCJYjWa5itwooLvQjlM1jUbMRYl5qPQx+ ktPBtiR8oIOQf4RrYYFwPAZGFBNEkeseZFNZrHp6OZT+YjtGcYiMoL19vbB62NG1pwZf8LTve34NK kXZKj3A/1++zl5Yqy4JQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nhrY3-000Dxn-57; Fri, 22 Apr 2022 11:34:03 +0000 Received: from mail-wr1-x434.google.com ([2a00:1450:4864:20::434]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nhrXz-000DwX-Nh for linux-arm-kernel@lists.infradead.org; Fri, 22 Apr 2022 11:34:01 +0000 Received: by mail-wr1-x434.google.com with SMTP id e28so427666wra.8 for ; Fri, 22 Apr 2022 04:33:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=references:user-agent:from:to:cc:subject:date:in-reply-to :message-id:mime-version; bh=uz+KcNIj+v9WSFSUjytskdEFRoZIQjG/EVCHIE7PoXA=; b=gzbQuJCj0I2T04uY0UYnVWLijPTDjp9cU8gjljlQbHYCpA8RUc0SI+1iOS8JCVDX30 wHYpTjb2TVGttz0rLW8wUREcv0cSVao2eWrMuveVbBcuvb69FNpN1EiELHQyz7N6bS5c 3TUxO7P8UkMUTNvh+TLzQ0YBXD9IfwQPSNMvk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:references:user-agent:from:to:cc:subject:date :in-reply-to:message-id:mime-version; bh=uz+KcNIj+v9WSFSUjytskdEFRoZIQjG/EVCHIE7PoXA=; b=gEKfPgviPssSyHWzWMbUcSICfy24zRAVhch8jlIyN7oPSLCBNuUMuJhPbM+6H3UWit 8NoNxjUHqveDc/dDXAfTgu56rFbf97kwjhfZAK6AupLXMG4v8Z4n8+Vn2OAEuYQAaSbJ FzR7LLbcddmCYrpCdPvd3tB6I7UpcTK9MCrWARbtKw2PE0NB5Uk+Htfgm+Lye1HEIAEb 3emE1/kzHg5yMY6JSjaeSISVkwWFwr9OwGy97jPYdiVO3ynENwAKDHjkUmTM4lPOkKNn tCwj3eU+ZnU/mwMa5dkPPKowizsqTCrqFjGVD78YVOPFgDJWOqZSLiEAY2AmmFGiSZ03 feLw== X-Gm-Message-State: AOAM532hGeUCFHwNWUoY+B0H0aGG1zc66nwjnyMrkjngWjDsWe4t8nRs IQpu2K+1NYG+sgdTNBPxWEqyBg== X-Google-Smtp-Source: ABdhPJytQeZpvy34ZOAAMQB7w6jF7X6Nf/OgCkIilBTulJnvg3gtBspcR68ouVwuJUzFMunZvSQqjw== X-Received: by 2002:a5d:4e08:0:b0:20a:8f9e:beef with SMTP id p8-20020a5d4e08000000b0020a8f9ebeefmr3571401wrt.8.1650627235044; Fri, 22 Apr 2022 04:33:55 -0700 (PDT) Received: from cloudflare.com (79.184.126.143.ipv4.supernova.orange.pl. [79.184.126.143]) by smtp.gmail.com with ESMTPSA id r25-20020adfa159000000b0020ac9758f17sm1338619wrr.23.2022.04.22.04.33.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Apr 2022 04:33:54 -0700 (PDT) References: <20220414162220.1985095-1-xukuohai@huawei.com> <20220414162220.1985095-5-xukuohai@huawei.com> User-agent: mu4e 1.6.10; emacs 27.2 From: Jakub Sitnicki To: Xu Kuohai Cc: bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, Catalin Marinas , Will Deacon , Steven Rostedt , Ingo Molnar , Daniel Borkmann , Alexei Starovoitov , Zi Shen Lim , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , John Fastabend , KP Singh , "David S . Miller" , Hideaki YOSHIFUJI , David Ahern , Thomas Gleixner , Borislav Petkov , Dave Hansen , x86@kernel.org, hpa@zytor.com, Shuah Khan , Mark Rutland , Ard Biesheuvel , Pasha Tatashin , Peter Collingbourne , Daniel Kiss , Sudeep Holla , Steven Price , Marc Zyngier , Mark Brown , Kumar Kartikeya Dwivedi , Delyan Kratunov , kernel-team@cloudflare.com Subject: Re: [PATCH bpf-next v2 4/6] bpf, arm64: Impelment bpf_arch_text_poke() for arm64 Date: Fri, 22 Apr 2022 12:54:02 +0200 In-reply-to: <20220414162220.1985095-5-xukuohai@huawei.com> Message-ID: <87levxfj32.fsf@cloudflare.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220422_043359_829270_C419F683 X-CRM114-Status: GOOD ( 29.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Xu, Thanks for working on this. We are also looking forward to using fentry hooks on arm64. In particular, attaching to entry/exit into/from XDP progs. On Thu, Apr 14, 2022 at 12:22 PM -04, Xu Kuohai wrote: > Impelment bpf_arch_text_poke() for arm64, so bpf trampoline code can use > it to replace nop with jump, or replace jump with nop. > > Signed-off-by: Xu Kuohai > Acked-by: Song Liu > --- > arch/arm64/net/bpf_jit_comp.c | 52 +++++++++++++++++++++++++++++++++++ > 1 file changed, 52 insertions(+) > > diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c > index 8ab4035dea27..1a1c3ea75ee2 100644 > --- a/arch/arm64/net/bpf_jit_comp.c > +++ b/arch/arm64/net/bpf_jit_comp.c > @@ -9,6 +9,7 @@ > > #include > #include > +#include > #include > #include > #include > @@ -18,6 +19,7 @@ > #include > #include > #include > +#include > #include > > #include "bpf_jit.h" > @@ -1529,3 +1531,53 @@ void bpf_jit_free_exec(void *addr) > { > return vfree(addr); > } > + > +static int gen_branch_or_nop(enum aarch64_insn_branch_type type, void *ip, > + void *addr, u32 *insn) > +{ > + if (!addr) > + *insn = aarch64_insn_gen_nop(); > + else > + *insn = aarch64_insn_gen_branch_imm((unsigned long)ip, > + (unsigned long)addr, > + type); > + > + return *insn != AARCH64_BREAK_FAULT ? 0 : -EFAULT; > +} > + > +int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type, > + void *old_addr, void *new_addr) > +{ > + int ret; > + u32 old_insn; > + u32 new_insn; > + u32 replaced; > + enum aarch64_insn_branch_type branch_type; > + > + if (poke_type == BPF_MOD_CALL) > + branch_type = AARCH64_INSN_BRANCH_LINK; This path, bpf_arch_text_poke(, BPF_MOD_CALL, ...), is what we hit when attaching a BPF program entry. It is exercised by selftest #232 xdp_bpf2bpf. However, with this patchset alone it will not work because we don't emit, yet, the ftrace patch (MOV X9, LR; NOP) as a part of BPF prog prologue, like ftrace_init_nop() does. So patching attempt will fail. I think that is what you mentioned to in your reply to Hou [1] So my question is - is support for attaching to BPF progs in scope for this patchset? If no, then perhaps it would be better for now to fail early with something like -EOPNOTSUPP when poke_type is BPF_MOD_CALL, rather then attempt to patch the code. If you plan to enable it as a part of this patchset, then I've given it a quick try, and it seems that not a lot is needed get fentry to BPF attachment to work. I'm including the diff for my quick and dirty attempt below. With that patch on top, the xdp_bpf2bpf tests pass: #232 xdp_bpf2bpf:OK [1] https://lore.kernel.org/bpf/d8c4f1fb-a020-9457-44e2-dc63982a9213@huawei.com/ > + else > + branch_type = AARCH64_INSN_BRANCH_NOLINK; > + > + if (gen_branch_or_nop(branch_type, ip, old_addr, &old_insn) < 0) > + return -EFAULT; > + > + if (gen_branch_or_nop(branch_type, ip, new_addr, &new_insn) < 0) > + return -EFAULT; > + > + mutex_lock(&text_mutex); > + if (aarch64_insn_read(ip, &replaced)) { > + ret = -EFAULT; > + goto out; > + } > + > + if (replaced != old_insn) { > + ret = -EFAULT; > + goto out; > + } > + > + ret = aarch64_insn_patch_text_nosync((void *)ip, new_insn); > +out: > + mutex_unlock(&text_mutex); The body of this critical section is identical as ftrace_modify_code(). Perhaps we could export it and reuse? > + return ret; > +} --- diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 5f6bd755050f..94d8251500ab 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -240,9 +240,9 @@ static bool is_lsi_offset(int offset, int scale) /* Tail call offset to jump into */ #if IS_ENABLED(CONFIG_ARM64_BTI_KERNEL) || \ IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL) -#define PROLOGUE_OFFSET 9 +#define PROLOGUE_OFFSET 11 #else -#define PROLOGUE_OFFSET 8 +#define PROLOGUE_OFFSET 10 #endif static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf) @@ -281,6 +281,10 @@ static int build_prologue(struct jit_ctx *ctx, bool ebpf_from_cbpf) * */ + /* Set up ftrace patch (initially in disabled state) */ + emit(A64_MOV(1, A64_R(9), A64_LR), ctx); + emit(A64_NOP, ctx); + /* Sign lr */ if (IS_ENABLED(CONFIG_ARM64_PTR_AUTH_KERNEL)) emit(A64_PACIASP, ctx); @@ -1888,10 +1892,16 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type poke_type, u32 replaced; enum aarch64_insn_branch_type branch_type; - if (poke_type == BPF_MOD_CALL) + if (poke_type == BPF_MOD_CALL) { branch_type = AARCH64_INSN_BRANCH_LINK; - else + /* + * Adjust addr to point at the BL in the callsite. + * See ftrace_init_nop() for the callsite sequence. + */ + ip = (void *)((unsigned long)ip + AARCH64_INSN_SIZE); + } else { branch_type = AARCH64_INSN_BRANCH_NOLINK; + } if (gen_branch_or_nop(branch_type, ip, old_addr, &old_insn) < 0) return -EFAULT; _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel