bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Björn Töpel" <bjorn.topel@gmail.com>
To: netdev@vger.kernel.org, ast@kernel.org, daniel@iogearbox.net
Cc: "Björn Töpel" <bjorn.topel@intel.com>,
	bpf@vger.kernel.org, magnus.karlsson@gmail.com,
	magnus.karlsson@intel.com, jonathan.lemon@gmail.com,
	ecree@solarflare.com, thoiland@redhat.com,
	andrii.nakryiko@gmail.com
Subject: [PATCH bpf-next v3 6/6] bpf, x86: align dispatcher branch targets to 16B
Date: Mon,  9 Dec 2019 14:55:22 +0100	[thread overview]
Message-ID: <20191209135522.16576-7-bjorn.topel@gmail.com> (raw)
In-Reply-To: <20191209135522.16576-1-bjorn.topel@gmail.com>

From: Björn Töpel <bjorn.topel@intel.com>

From Intel 64 and IA-32 Architectures Optimization Reference Manual,
3.4.1.4 Code Alignment, Assembly/Compiler Coding Rule 11: All branch
targets should be 16-byte aligned.

This commits aligns branch targets according to the Intel manual.

The nops used to align branch targets make the dispatcher larger, and
therefore the number of supported dispatch points/programs are
descreased from 64 to 48.

Signed-off-by: Björn Töpel <bjorn.topel@intel.com>
---
 arch/x86/net/bpf_jit_comp.c | 30 +++++++++++++++++++++++++++++-
 kernel/bpf/dispatcher.c     |  2 +-
 2 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 3ce7ad41bd6f..4c8a2d1f8470 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -1548,6 +1548,26 @@ static int emit_cond_near_jump(u8 **pprog, void *func, void *ip, u8 jmp_cond)
 	return 0;
 }
 
+static void emit_nops(u8 **pprog, unsigned int len)
+{
+	unsigned int i, noplen;
+	u8 *prog = *pprog;
+	int cnt = 0;
+
+	while (len > 0) {
+		noplen = len;
+
+		if (noplen > ASM_NOP_MAX)
+			noplen = ASM_NOP_MAX;
+
+		for (i = 0; i < noplen; i++)
+			EMIT1(ideal_nops[noplen][i]);
+		len -= noplen;
+	}
+
+	*pprog = prog;
+}
+
 static int emit_fallback_jump(u8 **pprog)
 {
 	u8 *prog = *pprog;
@@ -1570,8 +1590,8 @@ static int emit_fallback_jump(u8 **pprog)
 
 static int emit_bpf_dispatcher(u8 **pprog, int a, int b, s64 *progs)
 {
+	u8 *jg_reloc, *jg_target, *prog = *pprog;
 	int pivot, err, jg_bytes = 1, cnt = 0;
-	u8 *jg_reloc, *prog = *pprog;
 	s64 jg_offset;
 
 	if (a == b) {
@@ -1620,6 +1640,14 @@ static int emit_bpf_dispatcher(u8 **pprog, int a, int b, s64 *progs)
 	if (err)
 		return err;
 
+	/* From Intel 64 and IA-32 Architectures Optimization
+	 * Reference Manual, 3.4.1.4 Code Alignment, Assembly/Compiler
+	 * Coding Rule 11: All branch targets should be 16-byte
+	 * aligned.
+	 */
+	jg_target = PTR_ALIGN(prog, 16);
+	if (jg_target != prog)
+		emit_nops(&prog, jg_target - prog);
 	jg_offset = prog - jg_reloc;
 	emit_code(jg_reloc - jg_bytes, jg_offset, jg_bytes);
 
diff --git a/kernel/bpf/dispatcher.c b/kernel/bpf/dispatcher.c
index de6b1f20b920..5f8ce701bcad 100644
--- a/kernel/bpf/dispatcher.c
+++ b/kernel/bpf/dispatcher.c
@@ -25,7 +25,7 @@
  * allocated on first use, and never freed.
  */
 
-#define BPF_DISPATCHER_MAX 64 /* Fits in 2048B */
+#define BPF_DISPATCHER_MAX 48 /* Fits in 2048B */
 
 struct bpf_disp_prog {
 	struct bpf_prog *prog;
-- 
2.20.1


  parent reply	other threads:[~2019-12-09 13:55 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-09 13:55 [PATCH bpf-next v3 0/6] Introduce the BPF dispatcher Björn Töpel
2019-12-09 13:55 ` [PATCH bpf-next v3 1/6] bpf: move trampoline JIT image allocation to a function Björn Töpel
2019-12-09 13:55 ` [PATCH bpf-next v3 2/6] bpf: introduce BPF dispatcher Björn Töpel
2019-12-10  5:50   ` Alexei Starovoitov
2019-12-10  5:54     ` Björn Töpel
2019-12-09 13:55 ` [PATCH bpf-next v3 3/6] bpf, xdp: start using the BPF dispatcher for XDP Björn Töpel
2019-12-09 13:55 ` [PATCH bpf-next v3 4/6] bpf: start using the BPF dispatcher in BPF_TEST_RUN Björn Töpel
2019-12-09 13:55 ` [PATCH bpf-next v3 5/6] selftests: bpf: add xdp_perf test Björn Töpel
2019-12-10 11:05   ` Jesper Dangaard Brouer
2019-12-10 11:56     ` Björn Töpel
2019-12-09 13:55 ` Björn Töpel [this message]
2019-12-09 15:00 ` [PATCH bpf-next v3 0/6] Introduce the BPF dispatcher Toke Høiland-Jørgensen
2019-12-09 17:42   ` Björn Töpel
2019-12-11 12:38     ` Björn Töpel
2019-12-11 13:17       ` Toke Høiland-Jørgensen
2019-12-09 17:00 ` Jesper Dangaard Brouer
2019-12-09 17:45   ` Björn Töpel
2019-12-09 19:50     ` Jesper Dangaard Brouer
2019-12-10 19:28 ` Samudrala, Sridhar
2019-12-10 20:04   ` Björn Töpel
2019-12-10 19:59 ` Björn Töpel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191209135522.16576-7-bjorn.topel@gmail.com \
    --to=bjorn.topel@gmail.com \
    --cc=andrii.nakryiko@gmail.com \
    --cc=ast@kernel.org \
    --cc=bjorn.topel@intel.com \
    --cc=bpf@vger.kernel.org \
    --cc=daniel@iogearbox.net \
    --cc=ecree@solarflare.com \
    --cc=jonathan.lemon@gmail.com \
    --cc=magnus.karlsson@gmail.com \
    --cc=magnus.karlsson@intel.com \
    --cc=netdev@vger.kernel.org \
    --cc=thoiland@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).