Re: [PATCH] selftests/bpf: split -extras target to -static and -gen

From: Alexei Starovoitov <alexei.starovoitov@gmail.com>
To: shuah <shuah@kernel.org>
Cc: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>,
	Andrii Nakryiko <andrii.nakryiko@gmail.com>,
	bpf <bpf@vger.kernel.org>, Jiri Benc <jbenc@redhat.com>,
	Jiri Olsa <jolsa@redhat.com>, Andrii Nakryiko <andriin@fb.com>,
	Daniel Borkmann <daniel@iogearbox.net>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>
Subject: Re: [PATCH] selftests/bpf: split -extras target to -static and -gen
Date: Wed, 27 May 2020 15:23:13 -0700	[thread overview]
Message-ID: <CAADnVQJhb0+KWY0=4WVKc8NQswDJ5pU7LW1dQE2TQuya0Pn0oA@mail.gmail.com> (raw)
In-Reply-To: <ec5f6bd9-83e9-fc55-1885-18eee404d988@kernel.org>

On Wed, May 27, 2020 at 3:01 PM shuah <shuah@kernel.org> wrote:
>
> On 5/27/20 11:05 AM, Alexei Starovoitov wrote:
> > On Wed, May 27, 2020 at 10:02 AM Yauheni Kaliuta
> > <yauheni.kaliuta@redhat.com> wrote:
> >>
> >> Hi, Alexei!
> >>
> >>>>>>> On Wed, 27 May 2020 09:48:04 -0700, Alexei Starovoitov  wrote:
> >>
> >>   > On Wed, May 27, 2020 at 12:19 AM Yauheni Kaliuta
> >>   > <yauheni.kaliuta@redhat.com> wrote:
> >>   >>
> >>   >> Hi, Alexei!
> >>   >>
> >>   >> >>>>> On Tue, 26 May 2020 22:37:39 -0700, Alexei Starovoitov  wrote:
> >>   >>
> >>   >> > On Tue, May 26, 2020 at 10:31 PM Yauheni Kaliuta
> >>   >> > <yauheni.kaliuta@redhat.com> wrote:
> >>   >> >>
> >>   >> >> Hi, Andrii!
> >>   >> >>
> >>   >> >> >>>>> On Tue, 26 May 2020 17:19:18 -0700, Andrii Nakryiko  wrote:
> >>   >> >>
> >>   >> >> > On Fri, May 22, 2020 at 1:19 AM Yauheni Kaliuta
> >>   >> >> > <yauheni.kaliuta@redhat.com> wrote:
> >>   >> >> >>
> >>   >> >> >> There is difference in depoying static and generated extra resource
> >>   >> >> >> files between in/out of tree build and flavors:
> >>   >> >> >>
> >>   >> >> >> - in case of unflavored out-of-tree build static files are not
> >>   >> >> >> available and must be copied as well as both static and generated
> >>   >> >> >> files for flavored build.
> >>   >> >> >>
> >>   >> >> >> So split the rules and variables. The name TRUNNER_EXTRA_GEN_FILES
> >>   >> >> >> is chosen in analogy to TEST_GEN_* variants.
> >>   >> >> >>
> >>   >> >>
> >>   >> >> > Can we keep them together but be smarter about what needs to
> >>   >> >> > be copied based on source/target directories? I would really
> >>   >> >> > like to not blow up all these rules.
> >>   >> >>
> >>   >> >> I can try, ok, I just find it a bit more clear. But it's good to
> >>   >> >> get some input from kselftest about OOT build in general.
> >>   >>
> >>   >> > I see no value in 'make install' of selftests/bpf
> >>   >> > and since it's broken just remove that makefile target.
> >>   >>
> >>   >> Some CI systems perform testing next stage after building were
> >>   >> build tree is not available anymore. So it's in use at the
> >>   >> moment.
> >>
> >>   > such CI systems can do 'cp -r' then
> >> >> It's a discussion for linux-kselftest@ (added).
> >>
> >> At the moment `make install` is generic kselftest functionality
> >> and since bpf is part of that infra it looks a bit strange to
> >> break it intentionally.
> >
> > selftests/bpf is only historically part of selftests.
> > It probably should stop using kselftest build infra all together.
> > We had breakages in selftests/bpf in the past only because
> > of changes in kselftests bits.
> >
>
> The question is whether or not the breakages addresses quickly.
> Also, bpf keels breaking ksleftest builds and installs because
> it has dependencies on bleeding edge tools and causes problems
> for kselftest users.
>
> You are pulling me into the discussion midway and I am missing the
> context for the discussion. There is another thread on this topic
> where Yauheni and I have been talking about bpf install.
>
> I would say bpf install has never really worked from kselftest
> install mechanism.
>
> Ideally all tests use kselftest common install rule to leverage
> the install and not have users do individual test installs.
> It isn't productive and cooperative to say let's have bpf test
> do its thing. It is part of selftests and we have to figure out
> how to have it consistently build and run.
>
> It isn't building for me on Linux 5.7-rc7 at the moment, leave
> alone install.
>
> The test Makefile has to handle OUTPUT directory. Please add me
> to the entire patch series especially if it changes selftests
> Makefile and lib.mk. I will review and try to see if we can make
> bpf install work under kselftest common infrastructure.

I prefer to keep selftests/bpf install broken.
This forced marriage between kselftests and selftests/bpf
never worked well. I think it's a time to free them up from each other.