bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: Edward Cree <ecree@solarflare.com>
Cc: "Toke Høiland-Jørgensen" <toke@redhat.com>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Martin KaFai Lau" <kafai@fb.com>,
	"Song Liu" <songliubraving@fb.com>, "Yonghong Song" <yhs@fb.com>,
	"Marek Majkowski" <marek@cloudflare.com>,
	"Lorenz Bauer" <lmb@cloudflare.com>,
	"Alan Maguire" <alan.maguire@oracle.com>,
	"Jesper Dangaard Brouer" <brouer@redhat.com>,
	"David Miller" <davem@davemloft.net>,
	netdev@vger.kernel.org, bpf@vger.kernel.org
Subject: Re: static and dynamic linking. Was: [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF
Date: Tue, 12 Nov 2019 15:18:26 -0800	[thread overview]
Message-ID: <20191112231822.o3gir44yskmntgnq@ast-mbp.dhcp.thefacebook.com> (raw)
In-Reply-To: <8c251f3d-67bd-9bc2-8037-a15d93b48674@solarflare.com>

On Tue, Nov 12, 2019 at 09:25:06PM +0000, Edward Cree wrote:
> On 12/11/2019 19:52, Alexei Starovoitov wrote:
> > We haven't yet defined what 'extern' keyword in the program means.
> > There are a preliminary patches for llvm to support extern variables. Extern
> > functions are not done yet. We have to walk before we run. With dynamic linking
> > I'm proposing an api for the kernel that I think will work regardless of how
> > libbpf and llvm decide to define the meaning of 'extern'.
> Fwiw the 'natural' C way of doing it would be that for any extern symbol in
>  the C file, the ELF file gets a symbol entry with st_shndx=SHN_UNDEF, and
>  code in .text that uses that symbol gets relocation entries.  That's (AIUI)
>  how it works on 'normal' architectures, and that's what my ebld linker
>  understands; when it sees a definition in another file for that symbol
>  (matched just by the symbol name) it applies all the relocations of the
>  symbol to the appropriate progbits.
> I don't really see what else you could define 'extern' to mean.

That's exactly the problem with standard 'extern'. ELF preserves the name only.
There is no type. I think size is there, but linkers ignore it. There is also
no way to place extern into a section. Currently SEC("..") is a standard way to
annotate bpf programs. I think reliable 'extern' has to have more than just
name. 'extern int foo;' can be a reference to 'int foo;' in another BPF ELF
file, or it can be a reference to 'int foo;' in already loaded BPF prog, or it
can be a reference to 'int foo;' inside the kernel itself, or it can be a
reference to pseudo variable that libbpf should replace. For example 'extern
int kernel_version;' or 'extern int CONFIG_HZ;' would be useful extern-like
variables that program might want to use. Disambiguating by name is probably
not enough. We can define an order of resolution. libbpf will search in other
.o first, then will search in loaded bpf progs, than in kernel, and if all
fails than will resolve things like 'extern int CONFIG_HZ' on its own. It feels
fragile though. I think we need to be able to specify something like section to
extern variables and functions. That would be a compiler extension.

> > Partial verification should be available regardless of
> > whether kernel performs dynamic linking or libbpf staticly links multiple .o
> > together.
> It's not clear to me how partial verification would work for statically
>  linked programs — could you elaborate on this?

I was imagining that the verifier will do per-function verification
of program with sub-programs instead of analyzing from root.
Today the verifier is essentially told: Here is XDP program. Check that it's
valid with any valid context. Now imagine the verifier sees a function:
int foo(struct xdp_md *ctx);
The verifier can check that the function is safe when 'ctx' is valid. There is
no program type for the verifier to deal with. Such 'foo' is an abstract
program. It receives a valid pointer to xpd_md and can call any helper that
accepts xdp_md.

Applying the same logic for two arguments:
int foo(struct xdp_md *arg1, struct __sk_buff *arg2);
The verifier can check that such function is safe when both arg1 and arg2 are
valid pointers. This is not a realistic function. It illustrates the idea that
programs/functions can be abstract. Valid pointers of different types are
received and can be further passed into different helpers when types match.

The next step is to extend this thought process to integers.
int foo(struct xdp_md *arg1, int arg2);
The verifier can check that the program is valid for any valid arg1 and
arg2 = mark_reg_unbounded().

Support for pointers to non-context structs are a bit harder. That needs
more thinking. It should be possible to support:
int foo(struct xdp_md *arg1, char *arg2, bpf_size_t arg3);
The verifier will validate that arg2 array is accessed in [0, arg3] range.

I think that will cover most of the use cases. It won't be possible to support
aribtrary passing of pointers the way current bpf2bpf calls support, but I
think it's an acceptable limitation for partial verification.


  reply	other threads:[~2019-11-12 23:18 UTC|newest]

Thread overview: 55+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-07 17:20 [PATCH bpf-next v3 0/5] xdp: Support multiple programs on a single interface through chain calls Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF programs after each other Toke Høiland-Jørgensen
2019-10-07 20:42   ` Alexei Starovoitov
2019-10-08  8:07     ` Toke Høiland-Jørgensen
2019-10-09  1:51       ` Alexei Starovoitov
2019-10-09  8:03         ` Toke Høiland-Jørgensen
2019-10-10  4:41           ` Alexei Starovoitov
2019-10-14 12:35             ` Toke Høiland-Jørgensen
2019-10-14 17:08               ` John Fastabend
2019-10-14 18:48                 ` Toke Høiland-Jørgensen
2019-10-15 16:30                   ` Edward Cree
2019-10-15 16:42                     ` Toke Høiland-Jørgensen
2019-10-15 18:33                       ` Edward Cree
2019-10-17 12:11                         ` Toke Høiland-Jørgensen
2019-10-22 17:27                           ` Edward Cree
2019-10-22 18:07                             ` Toke Høiland-Jørgensen
2019-11-12  2:51                               ` static and dynamic linking. Was: [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF Alexei Starovoitov
2019-11-12 16:20                                 ` Toke Høiland-Jørgensen
2019-11-12 19:52                                   ` Alexei Starovoitov
2019-11-12 21:25                                     ` Edward Cree
2019-11-12 23:18                                       ` Alexei Starovoitov [this message]
2019-11-13 18:30                                         ` Edward Cree
2019-11-13 18:51                                           ` Andrii Nakryiko
2019-11-15  2:13                                           ` Alexei Starovoitov
2019-11-15 16:56                                             ` John Fastabend
2019-11-12 23:25                                     ` John Fastabend
2019-11-13  0:21                                       ` Alexei Starovoitov
2019-11-13  5:33                                         ` John Fastabend
2019-11-15  1:50                                           ` Alexei Starovoitov
2019-11-15 16:39                                             ` John Fastabend
2019-11-14 15:41                                     ` Toke Høiland-Jørgensen
2019-11-12 16:32                                 ` Edward Cree
2019-11-15 11:48                                 ` Lorenz Bauer
2019-11-15 23:02                                   ` Alexei Starovoitov
2019-11-18 13:29                                     ` Lorenz Bauer
2019-10-21 23:51                         ` [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF programs after each other Edward Cree
2019-10-16  2:28               ` Alexei Starovoitov
2019-10-16  8:27                 ` Jesper Dangaard Brouer
2019-10-16 10:35                   ` Daniel Borkmann
2019-10-16 11:16                     ` Toke Høiland-Jørgensen
2019-10-16 13:51                 ` Toke Høiland-Jørgensen
2019-10-19 20:09                   ` bpf indirect calls Alexei Starovoitov
2019-10-20 10:58                     ` Toke Høiland-Jørgensen
2019-10-25 16:30                       ` Alexei Starovoitov
2019-10-27 12:15                         ` Toke Høiland-Jørgensen
2019-10-09 10:19         ` [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF programs after each other Jesper Dangaard Brouer
2019-10-09 17:57           ` Alexei Starovoitov
2019-10-07 17:20 ` [PATCH bpf-next v3 2/5] bpf: Add support for setting chain call sequence for programs Toke Høiland-Jørgensen
2019-10-07 20:38   ` Daniel Borkmann
2019-10-08  8:09     ` Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 3/5] tools: Update bpf.h header for program chain calls Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 4/5] libbpf: Add syscall wrappers for BPF_PROG_CHAIN_* commands Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 5/5] selftests: Add tests for XDP chain calls Toke Høiland-Jørgensen
2019-10-07 18:58 ` [PATCH bpf-next v3 0/5] xdp: Support multiple programs on a single interface through " John Fastabend
2019-10-08  8:42   ` Toke Høiland-Jørgensen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191112231822.o3gir44yskmntgnq@ast-mbp.dhcp.thefacebook.com \
    --to=alexei.starovoitov@gmail.com \
    --cc=alan.maguire@oracle.com \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=ecree@solarflare.com \
    --cc=john.fastabend@gmail.com \
    --cc=kafai@fb.com \
    --cc=lmb@cloudflare.com \
    --cc=marek@cloudflare.com \
    --cc=netdev@vger.kernel.org \
    --cc=songliubraving@fb.com \
    --cc=toke@redhat.com \
    --cc=yhs@fb.com \
    --subject='Re: static and dynamic linking. Was: [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).