From: Daniel Borkmann <daniel@iogearbox.net>
To: Joe Stringer <joe@wand.net.nz>,
Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Lorenz Bauer <lmb@cloudflare.com>, Yonghong Song <yhs@fb.com>,
bpf <bpf@vger.kernel.org>, Networking <netdev@vger.kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Eric Dumazet <eric.dumazet@gmail.com>, Martin Lau <kafai@fb.com>,
john fastabend <john.fastabend@gmail.com>
Subject: Re: call for bpf progs. Re: [PATCHv2 bpf-next 5/5] selftests: bpf: add test for sk_assign
Date: Fri, 27 Mar 2020 21:16:52 +0100 [thread overview]
Message-ID: <c5e50f60-3872-b3ec-7038-737ca08f3077@iogearbox.net> (raw)
In-Reply-To: <CAOftzPjyCNGEjBm4k3aKK+=AB-1STDbYbQK5sZbK6gTAo13XuA@mail.gmail.com>
On 3/27/20 8:06 PM, Joe Stringer wrote:
> On Thu, Mar 26, 2020 at 2:07 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
>> The second concern was pruning, but iirc the experiments were inconclusive.
>> selftests/bpf only has old fb progs. Hence, I think, the step zero is for
>> everyone to contribute their bpf programs written in C. If we have both
>> cilium and cloudflare progs as selftests it will help a lot to guide such long
>> lasting verifier decisions.
>
> How would you like to handle program changes over time for this?
>
> In Cilium community we periodically rebuild bpf-next VM images for
> testing, and run every pull request against those images[0]. We also
> test against specific older kernels, currently 4.9 and 4.19. This
> allows us to get some sense for the impact of upstream changes while
> developing Cilium features, but unfortunately doesn't allow everyone
> using kernel selftests to get that feedback at least from the kernel
> tree. We also have a verifier complexity test script where we compile
> with the maximum number of features (to ideally generate the most
> complex programs possible) then attempt to load all of the various
> programs, and output the complexity count that the kernel reports[1,2]
> which we can track over time.
>
> However Cilium BPF programs are actively developing and even if we
> merge these programs into the kernel tree, they will get out-of-date
> quickly. Up until recently everything was verifying fine compiling
> with LLVM7 and loading into bpf-next. Over the past month we started
> noticing new issues not with the existing implementation, but in *new*
> BPF features. As we increased complexity, our CI started failing
> against bpf-next[3] while they loaded fine on older kernels. We ended
> up mitigating by upgrading to LLVM-10. Long story short, there's
> several moving parts; changing BPF program implementations, changing
> the compiler toolchain, changing the kernel verifier. So my question
> is basically, where's the line of responsibility for what the kernel
> selftests are responsible for vs integration tests? How do we maintain
> those over time as the BPF programs and compiler changes?
I wonder whether it would make sense to create test cases which are based
on our cilium/cilium:latest docker image. Those would be run as part of
BPF kernel selftests as well as part of our own CI for every PR. I think
it could be some basic connectivity test, service mapping, etc with a
bunch of application containers.
One nice thing that just came to mind is that it would actually allow for
easy testing of latest clang/llvm git by rerunning the test script and
remapping them as a volume, e.g.:
docker run -it -v /root/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/clang:/bin/clang \
-v /root/clang+llvm-7.0.1-x86_64-linux-gnu-ubuntu-18.04/bin/llc:/bin/llc \
cilium/cilium:latest /bin/sh
Perhaps that would be more useful and always up to date than a copy of the
code base that would get stale next day? In the end in this context kernel
changes and/or llvm changes might be of interest to check whether anything
potentially blows up, so having a self-contained packaging might be useful.
Thoughts?
> Do we just parachute the ~11K LoC of Cilium datapath into the kernel
> tree once per cycle? Or should Cilium autobuild a verifier-test docker
> image that kernel testing scripts can pull & run? Or would it be
> helpful to have a separate GitHub project similar to libbpf that pulls
> out kernel selftests, Cilium progs, fb progs, cloudflare progs, etc
> automatically and centralizes a generic suite of BPF verifier
> integration tests? Some other option?
>
> [0] https://github.com/cilium/packer-ci-build
> [1] https://github.com/cilium/cilium/blob/master/test/bpf/check-complexity.sh
> [2] https://github.com/cilium/cilium/blob/master/test/bpf/verifier-test.sh
> [3] https://github.com/cilium/cilium/issues/10517
>
next prev parent reply other threads:[~2020-03-27 20:17 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-25 5:57 [PATCHv2 bpf-next 0/5] Add bpf_sk_assign eBPF helper Joe Stringer
2020-03-25 5:57 ` [PATCHv2 bpf-next 1/5] bpf: Add socket assign support Joe Stringer
2020-03-26 6:23 ` Martin KaFai Lau
2020-03-26 6:31 ` Joe Stringer
2020-03-26 10:24 ` Lorenz Bauer
2020-03-26 22:52 ` Joe Stringer
2020-03-27 2:40 ` Joe Stringer
2020-03-25 5:57 ` [PATCHv2 bpf-next 2/5] bpf: Prefetch established socket destinations Joe Stringer
2020-03-26 21:11 ` Alexei Starovoitov
2020-03-26 21:45 ` Joe Stringer
2020-03-25 5:57 ` [PATCHv2 bpf-next 3/5] net: Track socket refcounts in skb_steal_sock() Joe Stringer
2020-03-25 5:57 ` [PATCHv2 bpf-next 4/5] bpf: Don't refcount LISTEN sockets in sk_assign() Joe Stringer
2020-03-25 10:29 ` Lorenz Bauer
2020-03-25 20:46 ` Joe Stringer
2020-03-26 10:20 ` Lorenz Bauer
2020-03-26 21:37 ` Joe Stringer
2020-03-25 5:57 ` [PATCHv2 bpf-next 5/5] selftests: bpf: add test for sk_assign Joe Stringer
2020-03-25 10:35 ` Lorenz Bauer
2020-03-25 20:55 ` Joe Stringer
2020-03-26 6:25 ` Martin KaFai Lau
2020-03-26 6:38 ` Joe Stringer
2020-03-26 23:39 ` Joe Stringer
2020-03-25 18:17 ` Yonghong Song
2020-03-25 21:20 ` Joe Stringer
2020-03-25 22:00 ` Yonghong Song
2020-03-25 23:07 ` Joe Stringer
2020-03-26 10:13 ` Lorenz Bauer
2020-03-26 21:07 ` call for bpf progs. " Alexei Starovoitov
2020-03-26 23:14 ` Yonghong Song
2020-03-27 10:02 ` Lorenz Bauer
2020-03-27 16:08 ` Alexei Starovoitov
2020-03-27 19:06 ` Joe Stringer
2020-03-27 20:16 ` Daniel Borkmann [this message]
2020-03-27 22:24 ` Alexei Starovoitov
2020-03-28 0:17 ` Andrii Nakryiko
2020-03-26 2:04 ` Andrii Nakryiko
2020-03-26 2:16 ` Andrii Nakryiko
2020-03-26 5:28 ` Joe Stringer
2020-03-26 6:31 ` Martin KaFai Lau
2020-03-26 19:36 ` Andrii Nakryiko
2020-03-26 21:38 ` Joe Stringer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c5e50f60-3872-b3ec-7038-737ca08f3077@iogearbox.net \
--to=daniel@iogearbox.net \
--cc=alexei.starovoitov@gmail.com \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=eric.dumazet@gmail.com \
--cc=joe@wand.net.nz \
--cc=john.fastabend@gmail.com \
--cc=kafai@fb.com \
--cc=lmb@cloudflare.com \
--cc=netdev@vger.kernel.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).