From: Stanislav Fomichev <sdf@google.com>
To: David Vernet <void@manifault.com>
Cc: Magnus Karlsson <magnus.karlsson@gmail.com>,
magnus.karlsson@intel.com, bjorn@kernel.org, ast@kernel.org,
daniel@iogearbox.net, netdev@vger.kernel.org,
maciej.fijalkowski@intel.com, bpf@vger.kernel.org, yhs@fb.com,
andrii@kernel.org, martin.lau@linux.dev, song@kernel.org,
john.fastabend@gmail.com, kpsingh@kernel.org, haoluo@google.com,
jolsa@kernel.org, tirthendu.sarkar@intel.com,
jonathan.lemon@gmail.com
Subject: Re: [PATCH bpf-next v2 11/15] selftests/xsk: get rid of built-in XDP program
Date: Wed, 4 Jan 2023 11:46:56 -0800 [thread overview]
Message-ID: <CAKH8qBuypCkTZYoSPHH1rbeAd3yRB_n2PidS4KN5CGfCpQTMtA@mail.gmail.com> (raw)
In-Reply-To: <Y7XP9924wTevHcBT@maniforge.lan>
On Wed, Jan 4, 2023 at 11:14 AM David Vernet <void@manifault.com> wrote:
>
> On Wed, Jan 04, 2023 at 10:19:37AM -0800, Stanislav Fomichev wrote:
> > On Wed, Jan 4, 2023 at 10:14 AM David Vernet <void@manifault.com> wrote:
> > >
> > > On Wed, Jan 04, 2023 at 01:17:40PM +0100, Magnus Karlsson wrote:
> > > > From: Magnus Karlsson <magnus.karlsson@intel.com>
> > > >
> > > > Get rid of the built-in XDP program that was part of the old libbpf
> > > > code in xsk.c and replace it with an eBPF program build using the
> > > > framework by all the other bpf selftests. This will form the base for
> > > > adding more programs in later commits.
> > > >
> > > > Signed-off-by: Magnus Karlsson <magnus.karlsson@intel.com>
> > > > ---
> > > > tools/testing/selftests/bpf/Makefile | 2 +-
> > > > .../selftests/bpf/progs/xsk_xdp_progs.c | 19 ++++
> > > > tools/testing/selftests/bpf/xsk.c | 88 ++++---------------
> > > > tools/testing/selftests/bpf/xsk.h | 6 +-
> > > > tools/testing/selftests/bpf/xskxceiver.c | 72 ++++++++-------
> > > > tools/testing/selftests/bpf/xskxceiver.h | 7 +-
> > > > 6 files changed, 88 insertions(+), 106 deletions(-)
> > > > create mode 100644 tools/testing/selftests/bpf/progs/xsk_xdp_progs.c
> > > >
> > > > diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> > > > index 205e8c3c346a..a0193a8f9da6 100644
> > > > --- a/tools/testing/selftests/bpf/Makefile
> > > > +++ b/tools/testing/selftests/bpf/Makefile
> > > > @@ -240,7 +240,7 @@ $(OUTPUT)/flow_dissector_load: $(TESTING_HELPERS)
> > > > $(OUTPUT)/test_maps: $(TESTING_HELPERS)
> > > > $(OUTPUT)/test_verifier: $(TESTING_HELPERS) $(CAP_HELPERS)
> > > > $(OUTPUT)/xsk.o: $(BPFOBJ)
> > > > -$(OUTPUT)/xskxceiver: $(OUTPUT)/xsk.o
> > > > +$(OUTPUT)/xskxceiver: $(OUTPUT)/xsk.o $(OUTPUT)/xsk_xdp_progs.skel.h
> > >
> > > Hi Magnus,
> > >
> > > This seems to break the selftests build for clang:
> > >
> > > $ pwd
> > > <redacted>/bpf-next/tools/testing/selftests/bpf
> > >
> > > $ make LLVM=1 CC=clang
> > > MKDIR libbpf
> > > HOSTCC /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep.o
> > > HOSTLD /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep-in.o
> > > LINK /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/fixdep
> > >
> > > ...
> > >
> > > GEN-SKEL [test_progs-no_alu32] test_static_linked.skel.h
> > > LINK-BPF [test_progs-no_alu32] test_usdt.bpf.o
> > > GEN-SKEL [test_progs-no_alu32] linked_vars.skel.h
> > > GEN-SKEL [test_progs-no_alu32] linked_funcs.skel.h
> > > EXT-COPY [test_progs-no_alu32] urandom_read bpf_testmod.ko liburandom_read.so xdp_synproxy sign-file ima_setup.sh verify_sig_setup.sh btf_dump_test_case_bitfields.c btf_dump_test_case_multidim.c btf_dump_test_case_namespacing.c btf_dump_test_case_ordering.c btf_dump_test_case_packing.c btf_dump_test_case_padding.c btf_dump_test_case_syntax.c
> > > GEN-SKEL [test_progs-no_alu32] linked_maps.skel.h
> > > GEN-SKEL [test_progs-no_alu32] test_subskeleton.skel.h
> > > BINARY xskxceiver
> > > BINARY bench
> > > GEN-SKEL [test_progs-no_alu32] test_subskeleton_lib.skel.h
> > > GEN-SKEL [test_progs-no_alu32] test_usdt.skel.h
> > > clang-15: error: cannot specify -o when generating multiple output files
> > > make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xskxceiver] Error 1
> > > make: *** Waiting for unfinished jobs....
> > > make[1]: Nothing to be done for 'docs'.
> > > $
> > >
> > > It's also broken on CI: https://github.com/kernel-patches/bpf/actions/runs/3837984934/jobs/6533917001
> > >
> > > Could you please look into this?
> >
> > Ugh, that's the same issue I'm getting for my xdp_hw_metadata binary.
> > And I'm still at loss on how to reproduce it locally. (I've tried 'apt
> > install clang-16 in ubuntu latest docker container' and it's still
> > fine).
>
> I was able to reproduce this issue locally:
Damn, I was missing LLVM=1, that does make it reproduce for me, thanks!
> [void@maniforge bpf]$ make -j LLVM=1 CC=clang
> GEN-SKEL [test_progs] pyperf600.skel.h
> GEN-SKEL [test_progs] test_verif_scale2.skel.h
> LINK-BPF [test_progs] test_static_linked.bpf.o
> LINK-BPF [test_progs] linked_funcs.bpf.o
>
> ...
>
> LINK-BPF [test_progs-no_alu32] test_usdt.bpf.o
> EXT-COPY [test_progs-no_alu32] urandom_read bpf_testmod.ko liburandom_read.so xdp_synproxy sign-file ima_setup.sh verify_sig_setup.sh btf_dump_test_case_bitfields.c btf_dump_test_case_multidim.c btf_dump_test_case_namespacing.c btf_dump_test_case_ordering.c btf_dump_test_case_packing.c btf_dump_test_case_padding.c btf_dump_test_case_syntax.c
> GEN-SKEL [test_progs-no_alu32] linked_funcs.skel.h
> BINARY bench
> GEN-SKEL [test_progs-no_alu32] test_subskeleton.skel.h
> BINARY xdp_hw_metadata
> GEN-SKEL [test_progs-no_alu32] test_subskeleton_lib.skel.h
> GEN-SKEL [test_progs-no_alu32] test_usdt.skel.h
> clang-15: error: cannot specify -o when generating multiple output files
> make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata] Error 1
> make: *** Waiting for unfinished jobs....
> make[1]: Nothing to be done for 'docs'.
>
> Here's the actual clang command being executed:
>
> [void@maniforge bpf]$ make LLVM=1 --dry-run xdp_hw_metadata
> printf ' %-8s%s %s%s\n' "BINARY" "" "xdp_hw_metadata" "";
> clang --target=x86_64-linux-gnu -fintegrated-as -g -O0 -rdynamic -Wall -Werror -DHAVE_GENHDR -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/include -I/home/void/upstream/bpf-next/include/generated -I/home/void/upstream/bpf-next/tools/lib -I/home/void/upstream/bpf-next/tools/include -I/home/void/upstream/bpf-next/tools/include/uapi -I/home/void/upstream/bpf-next/tools/testing/selftests/bpf -Wno-unused-command-line-argument -static xdp_hw_metadata.c /home/void/upstream/bpf-next/tools/testing/selftests/bpf/tools/build/libbpf/libbpf.a /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xsk.o /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata.skel.h /home/void/upstream/bpf-next/tools/testing/selftests/bpf/network_helpers.o -lelf -lz -lrt -lpthread -o /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata
>
> and the output using --debug=j
>
> [void@maniforge bpf]$ make LLVM=1 --debug=j xdp_hw_metadata
> Putting child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693804 on the chain.
> Live child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693804
> BINARY xdp_hw_metadata
> Reaping winning child 0x55cc78cd6670 PID 693804
> Live child 0x55cc78cd6670 (/home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata) PID 693805
> clang-15: error: cannot specify -o when generating multiple output files
> Reaping losing child 0x55cc78cd6670 PID 693805
> make: *** [Makefile:171: /home/void/upstream/bpf-next/tools/testing/selftests/bpf/xdp_hw_metadata] Error 1
> Removing child 0x55cc78cd6670 PID 693805 from chain.
>
> make is taking the xdp_hw_metadata.skel.h file and providing it as an
> input to clang. So I believe what's going on here is that the clang
> command above is actually creating two output files:
>
> 1. xdp_hw_metadata
> 2. The precompiled header generated from xdp_hw_metadata.skel.h
>
> and the error is clang reasonably saying: "I don't know which output
> file you're referring to with -o". I'm surprised that gcc doesn't
> complain about this, but I assume that it's doing the far more
> non-intuitive thing of first outputting the precompiled header as
> xdp_hw_metadata, and then immediately overwriting it with the actual
> xdp_hw_metadata binary.
Hm, I'm a bit surprised that clang can't handle .h as an extra input.
GCC seems to be doing the right thing; the headers compile to nothing
-> use that nothing as an extra input and don't complain..
Seems like the way to go is to explicitly have a '$(CC) xxx' rule that
filters it out. Will try.
> > Any pointers on how to debug those github actions locally?
> >
> > > Thanks,
> > > David
next prev parent reply other threads:[~2023-01-04 19:47 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-04 12:17 [PATCH bpf-next v2 00/15] selftests/xsk: speed-ups, fixes, and new XDP programs Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 01/15] selftests/xsk: print correct payload for packet dump Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 02/15] selftests/xsk: do not close unused file descriptors Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 03/15] selftests/xsk: submit correct number of frames in populate_fill_ring Magnus Karlsson
2023-01-09 23:01 ` Maciej Fijalkowski
2023-01-04 12:17 ` [PATCH bpf-next v2 04/15] selftests/xsk: print correct error codes when exiting Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 05/15] selftests/xsk: remove unused variable outstanding_tx Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 06/15] selftests/xsk: add debug option for creating netdevs Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 07/15] selftests/xsk: replace asm acquire/release implementations Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 08/15] selftests/xsk: remove namespaces Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 09/15] selftests/xsk: load and attach XDP program only once per mode Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 10/15] selftests/xsk: remove unnecessary code in control path Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 11/15] selftests/xsk: get rid of built-in XDP program Magnus Karlsson
2023-01-04 18:14 ` David Vernet
2023-01-04 18:19 ` Stanislav Fomichev
2023-01-04 19:13 ` David Vernet
2023-01-04 19:46 ` Stanislav Fomichev [this message]
2023-01-05 8:38 ` Magnus Karlsson
2023-01-04 23:15 ` Andrii Nakryiko
2023-01-04 23:44 ` Stanislav Fomichev
2023-01-11 19:20 ` Andrii Nakryiko
2023-01-04 12:17 ` [PATCH bpf-next v2 12/15] selftests/xsk: add test when some packets are XDP_DROPed Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 13/15] selftests/xsk: merge dual and single thread dispatchers Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 14/15] selftests/xsk: automatically restore packet stream Magnus Karlsson
2023-01-04 12:17 ` [PATCH bpf-next v2 15/15] selftests/xsk: automatically switch XDP programs Magnus Karlsson
2023-01-10 11:56 ` Maciej Fijalkowski
2023-01-10 12:08 ` Magnus Karlsson
2023-01-10 12:00 ` [PATCH bpf-next v2 00/15] selftests/xsk: speed-ups, fixes, and new " Maciej Fijalkowski
2023-01-10 12:10 ` Magnus Karlsson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAKH8qBuypCkTZYoSPHH1rbeAd3yRB_n2PidS4KN5CGfCpQTMtA@mail.gmail.com \
--to=sdf@google.com \
--cc=andrii@kernel.org \
--cc=ast@kernel.org \
--cc=bjorn@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=daniel@iogearbox.net \
--cc=haoluo@google.com \
--cc=john.fastabend@gmail.com \
--cc=jolsa@kernel.org \
--cc=jonathan.lemon@gmail.com \
--cc=kpsingh@kernel.org \
--cc=maciej.fijalkowski@intel.com \
--cc=magnus.karlsson@gmail.com \
--cc=magnus.karlsson@intel.com \
--cc=martin.lau@linux.dev \
--cc=netdev@vger.kernel.org \
--cc=song@kernel.org \
--cc=tirthendu.sarkar@intel.com \
--cc=void@manifault.com \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).