bpf.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Edward Cree <ecree@solarflare.com>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: "Toke Høiland-Jørgensen" <toke@redhat.com>,
	"John Fastabend" <john.fastabend@gmail.com>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Martin KaFai Lau" <kafai@fb.com>,
	"Song Liu" <songliubraving@fb.com>, "Yonghong Song" <yhs@fb.com>,
	"Marek Majkowski" <marek@cloudflare.com>,
	"Lorenz Bauer" <lmb@cloudflare.com>,
	"Alan Maguire" <alan.maguire@oracle.com>,
	"Jesper Dangaard Brouer" <brouer@redhat.com>,
	"David Miller" <davem@davemloft.net>,
	netdev@vger.kernel.org, bpf@vger.kernel.org
Subject: Re: static and dynamic linking. Was: [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF
Date: Wed, 13 Nov 2019 18:30:04 +0000	[thread overview]
Message-ID: <0c90adc4-5992-8648-88bf-4993252e8992@solarflare.com> (raw)
In-Reply-To: <20191112231822.o3gir44yskmntgnq@ast-mbp.dhcp.thefacebook.com>

On 12/11/2019 23:18, Alexei Starovoitov wrote:
> On Tue, Nov 12, 2019 at 09:25:06PM +0000, Edward Cree wrote:
>> Fwiw the 'natural' C way of doing it would be that for any extern symbol in
>>  the C file, the ELF file gets a symbol entry with st_shndx=SHN_UNDEF, and
>>  code in .text that uses that symbol gets relocation entries.  That's (AIUI)
>>  how it works on 'normal' architectures, and that's what my ebld linker
>>  understands; when it sees a definition in another file for that symbol
>>  (matched just by the symbol name) it applies all the relocations of the
>>  symbol to the appropriate progbits.
>> I don't really see what else you could define 'extern' to mean.
> That's exactly the problem with standard 'extern'. ELF preserves the name only.
> There is no type.
But if you have BTFs, then you can look up each symbol's type at link time and
 check that they match.  Point being that the BTF is for validation (and
 therefore strictly optional at this point) rather than using it to identify a
 symbol.  Trouble with the latter is that BTF ids get renumbered on linking, so
 any references to them have to change, whereas symbol names stay the same.

> There is also
> no way to place extern into a section. Currently SEC("..") is a standard way to
> annotate bpf programs.
While the symbol itself doesn't have a section, each _use_ of the symbol has a
 reloc, and the SHT_REL[A] in which that reloc resides has a sh_info specifying
 "the section header index of the section to which the relocation applies."  So
 can't that be used if symbol visibility needs to depend on section?  Tbh I
 can't exactly see why externs need placing in a section in the first place.

> I think reliable 'extern' has to have more than just
> name. 'extern int foo;' can be a reference to 'int foo;' in another BPF ELF
> file, or it can be a reference to 'int foo;' in already loaded BPF prog, or it
> can be a reference to 'int foo;' inside the kernel itself, or it can be a
> reference to pseudo variable that libbpf should replace. For example 'extern
> int kernel_version;' or 'extern int CONFIG_HZ;' would be useful extern-like
> variables that program might want to use. Disambiguating by name is probably
> not enough. We can define an order of resolution. libbpf will search in other
> .o first, then will search in loaded bpf progs, than in kernel, and if all
> fails than will resolve things like 'extern int CONFIG_HZ' on its own. It feels
> fragile though.
It sounds perfectly reasonable and not fragile to me.  The main alternative
 I see, about equally good, is to not allow defining symbols that are already
 (non-weakly) defined; so if a bpf prog tries to globally declare "int CONFIG_HZ"
 or "int netif_receive_skb(struct sk_buff *skb)" then it gets rejected.

> I think we need to be able to specify something like section to
> extern variables and functions.
It seems unnecessary to have the user code specify this.  Another a bad
 analogy: in userland C code you don't have to annotate the function protos in
 your header files to say whether they come from another .o file, a random
 library or the libc.  You just declare "a function called this exists somewhere
 and we'll find it at link time".

> I was imagining that the verifier will do per-function verification
> of program with sub-programs instead of analyzing from root.
Ah I see.  Yes, that's a very attractive design.

If we make it from a sufficiently generic idea of pre/postconditions, then it
 could also be useful for e.g. loop bodies (user-supplied annotations that allow
 us to walk the body only once instead of N times); then a function call just
 gets standard pre/postconditions generated from its argument types if the user
 didn't specify something else.

That would then also support things like:
> The next step is to extend this thought process to integers.
> int foo(struct xdp_md *arg1, int arg2);
> The verifier can check that the program is valid for any valid arg1 and
> arg2 = mark_reg_unbounded().
... this but arg2 isn't unbounded.
However, it might be difficult to do this without exposing details of the
 verifier into the ABI.  Still, if we can it sounds like it would make John
 quite happy too.  And of course it doesn't need to have the user annotations
 from the beginning, it can start out as just the kernel generating pre/post
 conditions internally.


  reply	other threads:[~2019-11-13 18:30 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-07 17:20 [PATCH bpf-next v3 0/5] xdp: Support multiple programs on a single interface through chain calls Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF programs after each other Toke Høiland-Jørgensen
2019-10-07 20:42   ` Alexei Starovoitov
2019-10-08  8:07     ` Toke Høiland-Jørgensen
2019-10-09  1:51       ` Alexei Starovoitov
2019-10-09  8:03         ` Toke Høiland-Jørgensen
2019-10-10  4:41           ` Alexei Starovoitov
2019-10-14 12:35             ` Toke Høiland-Jørgensen
2019-10-14 17:08               ` John Fastabend
2019-10-14 18:48                 ` Toke Høiland-Jørgensen
2019-10-15 16:30                   ` Edward Cree
2019-10-15 16:42                     ` Toke Høiland-Jørgensen
2019-10-15 18:33                       ` Edward Cree
2019-10-17 12:11                         ` Toke Høiland-Jørgensen
2019-10-22 17:27                           ` Edward Cree
2019-10-22 18:07                             ` Toke Høiland-Jørgensen
2019-11-12  2:51                               ` static and dynamic linking. Was: [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF Alexei Starovoitov
2019-11-12 16:20                                 ` Toke Høiland-Jørgensen
2019-11-12 19:52                                   ` Alexei Starovoitov
2019-11-12 21:25                                     ` Edward Cree
2019-11-12 23:18                                       ` Alexei Starovoitov
2019-11-13 18:30                                         ` Edward Cree [this message]
2019-11-13 18:51                                           ` Andrii Nakryiko
2019-11-15  2:13                                           ` Alexei Starovoitov
2019-11-15 16:56                                             ` John Fastabend
2019-11-12 23:25                                     ` John Fastabend
2019-11-13  0:21                                       ` Alexei Starovoitov
2019-11-13  5:33                                         ` John Fastabend
2019-11-15  1:50                                           ` Alexei Starovoitov
2019-11-15 16:39                                             ` John Fastabend
2019-11-14 15:41                                     ` Toke Høiland-Jørgensen
2019-11-12 16:32                                 ` Edward Cree
2019-11-15 11:48                                 ` Lorenz Bauer
2019-11-15 23:02                                   ` Alexei Starovoitov
2019-11-18 13:29                                     ` Lorenz Bauer
2019-10-21 23:51                         ` [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF programs after each other Edward Cree
2019-10-16  2:28               ` Alexei Starovoitov
2019-10-16  8:27                 ` Jesper Dangaard Brouer
2019-10-16 10:35                   ` Daniel Borkmann
2019-10-16 11:16                     ` Toke Høiland-Jørgensen
2019-10-16 13:51                 ` Toke Høiland-Jørgensen
2019-10-19 20:09                   ` bpf indirect calls Alexei Starovoitov
2019-10-20 10:58                     ` Toke Høiland-Jørgensen
2019-10-25 16:30                       ` Alexei Starovoitov
2019-10-27 12:15                         ` Toke Høiland-Jørgensen
2023-09-27 13:27                     ` Matt Bobrowski
2023-09-29 21:06                       ` Alexei Starovoitov
2023-10-02 18:50                         ` Barret Rhoden
2023-10-06  9:36                         ` Matt Bobrowski
2023-10-06 18:49                           ` Alexei Starovoitov
2023-10-19 12:28                             ` Matt Bobrowski
2019-10-09 10:19         ` [PATCH bpf-next v3 1/5] bpf: Support chain calling multiple BPF programs after each other Jesper Dangaard Brouer
2019-10-09 17:57           ` Alexei Starovoitov
2019-10-07 17:20 ` [PATCH bpf-next v3 2/5] bpf: Add support for setting chain call sequence for programs Toke Høiland-Jørgensen
2019-10-07 20:38   ` Daniel Borkmann
2019-10-08  8:09     ` Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 3/5] tools: Update bpf.h header for program chain calls Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 4/5] libbpf: Add syscall wrappers for BPF_PROG_CHAIN_* commands Toke Høiland-Jørgensen
2019-10-07 17:20 ` [PATCH bpf-next v3 5/5] selftests: Add tests for XDP chain calls Toke Høiland-Jørgensen
2019-10-07 18:58 ` [PATCH bpf-next v3 0/5] xdp: Support multiple programs on a single interface through " John Fastabend
2019-10-08  8:42   ` Toke Høiland-Jørgensen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0c90adc4-5992-8648-88bf-4993252e8992@solarflare.com \
    --to=ecree@solarflare.com \
    --cc=alan.maguire@oracle.com \
    --cc=alexei.starovoitov@gmail.com \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=brouer@redhat.com \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=john.fastabend@gmail.com \
    --cc=kafai@fb.com \
    --cc=lmb@cloudflare.com \
    --cc=marek@cloudflare.com \
    --cc=netdev@vger.kernel.org \
    --cc=songliubraving@fb.com \
    --cc=toke@redhat.com \
    --cc=yhs@fb.com \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).