From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Google-Smtp-Source: AH8x226N2Pdt19pa0jspiDt3R6FyW/1suScz6MC6b8EOh7ZOe/RiqRXW9fd/R6oHd+2itQz7VGlv ARC-Seal: i=1; a=rsa-sha256; t=1517256669; cv=none; d=google.com; s=arc-20160816; b=AMg94GkYuJ5DG4elkkMxHtBIOXXNFGJ7mO7whLjbyiNfdP5V42LabOiRMgQsBHB3df 63NdS1pKtb3NsmW2CKJUGgSmZBZQHLCsDHZvodZO4UNaxtrW3coQ4DN3Fm9X+AwJflAR zHz24WfNp8sswlvLDbSm5qE7m49vfNLk7jMQ9x0NFiubBsSjqAW2K5x8tlyhPSCSgYP/ a/s6oYzbqyf+quV6Zjk18QAIYS5TiOMOFXEx0ZKZSWxDfbHHbVgsE9tNdihfvjI5jMCz JcV62meSqTsoFHlx9MK4NWLoW90pqX9Dwu1BJHJBkANtNhWTPnwGzwNsyqTG7uJcVERO 70Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=/6ksdEcS4dfmrfqfhkuaWDvSnB8mbiG2vLFEsY+CwNs=; b=CM8Ah/PQS4r7BAr53gDGnZxgwQBP7Y2esA70Z04IZtWW7jz/UMmuzf8iJRhFEsm9ts KElFpJKBNfDz0spESWYdkCa6+QZYEff3HNRkPBhSEn9r/JAtELCe0XB8i1yeuYxZcMn3 j4C2OeU88xj7pmgKijXKLZkZdYvF4boUx1BLn1Ge0XjGLwb0PKab2fKAY0SX44KPpEno W3DNM5UCPAaqYg5Qn93ACF/iY5WKFe+4Z/LN7tSx2BUONUlerkwfTnw1tfbEvBQGwDCs ZzmaxNaJbFKqpqME6IVGRYMGO+7ILx4A30c3htNR+Pu7E+HKcTLInHs0LQLWTRdvXME9 RoGg== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning gregkh@linuxfoundation.org does not designate 90.92.71.90 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning gregkh@linuxfoundation.org does not designate 90.92.71.90 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "ast@kernel.org, stable@vger.kernel.org, Daniel Borkmann" , Alexei Starovoitov , Daniel Borkmann Subject: [PATCH 4.14 70/71] bpf, arm64: fix stack_depth tracking in combination with tail calls Date: Mon, 29 Jan 2018 13:57:38 +0100 Message-Id: <20180129123832.239908860@linuxfoundation.org> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180129123827.271171825@linuxfoundation.org> References: <20180129123827.271171825@linuxfoundation.org> User-Agent: quilt/0.65 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 X-getmail-retrieved-from-mailbox: INBOX X-GMAIL-LABELS: =?utf-8?b?IlxcU2VudCI=?= X-GMAIL-THRID: =?utf-8?q?1590958929253125626?= X-GMAIL-MSGID: =?utf-8?q?1590958929253125626?= X-Mailing-List: linux-kernel@vger.kernel.org List-ID: 4.14-stable review patch. If anyone has any objections, please let me know. ------------------ From: Daniel Borkmann [ upstream commit a2284d912bfc865cdca4c00488e08a3550f9a405 ] Using dynamic stack_depth tracking in arm64 JIT is currently broken in combination with tail calls. In prologue, we cache ctx->stack_size and adjust SP reg for setting up function call stack, and tearing it down again in epilogue. Problem is that when doing a tail call, the cached ctx->stack_size might not be the same. One way to fix the problem with minimal overhead is to re-adjust SP in emit_bpf_tail_call() and properly adjust it to the current program's ctx->stack_size. Tested on Cavium ThunderX ARMv8. Fixes: f1c9eed7f437 ("bpf, arm64: take advantage of stack_depth tracking") Signed-off-by: Daniel Borkmann Signed-off-by: Alexei Starovoitov Signed-off-by: Greg Kroah-Hartman --- arch/arm64/net/bpf_jit_comp.c | 20 +++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -148,7 +148,8 @@ static inline int epilogue_offset(const /* Stack must be multiples of 16B */ #define STACK_ALIGN(sz) (((sz) + 15) & ~15) -#define PROLOGUE_OFFSET 8 +/* Tail call offset to jump into */ +#define PROLOGUE_OFFSET 7 static int build_prologue(struct jit_ctx *ctx) { @@ -200,19 +201,19 @@ static int build_prologue(struct jit_ctx /* Initialize tail_call_cnt */ emit(A64_MOVZ(1, tcc, 0, 0), ctx); - /* 4 byte extra for skb_copy_bits buffer */ - ctx->stack_size = prog->aux->stack_depth + 4; - ctx->stack_size = STACK_ALIGN(ctx->stack_size); - - /* Set up function call stack */ - emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_size), ctx); - cur_offset = ctx->idx - idx0; if (cur_offset != PROLOGUE_OFFSET) { pr_err_once("PROLOGUE_OFFSET = %d, expected %d!\n", cur_offset, PROLOGUE_OFFSET); return -1; } + + /* 4 byte extra for skb_copy_bits buffer */ + ctx->stack_size = prog->aux->stack_depth + 4; + ctx->stack_size = STACK_ALIGN(ctx->stack_size); + + /* Set up function call stack */ + emit(A64_SUB_I(1, A64_SP, A64_SP, ctx->stack_size), ctx); return 0; } @@ -260,11 +261,12 @@ static int emit_bpf_tail_call(struct jit emit(A64_LDR64(prg, tmp, prg), ctx); emit(A64_CBZ(1, prg, jmp_offset), ctx); - /* goto *(prog->bpf_func + prologue_size); */ + /* goto *(prog->bpf_func + prologue_offset); */ off = offsetof(struct bpf_prog, bpf_func); emit_a64_mov_i64(tmp, off, ctx); emit(A64_LDR64(tmp, prg, tmp), ctx); emit(A64_ADD_I(1, tmp, tmp, sizeof(u32) * PROLOGUE_OFFSET), ctx); + emit(A64_ADD_I(1, A64_SP, A64_SP, ctx->stack_size), ctx); emit(A64_BR(tmp), ctx); /* out: */