Re: [PATCH] selftests/bpf: split -extras target to -static and -gen

From: shuah <shuah@kernel.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>,
	Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>,
	bpf <bpf@vger.kernel.org>, Jiri Benc <jbenc@redhat.com>,
	Jiri Olsa <jolsa@redhat.com>, Andrii Nakryiko <andriin@fb.com>,
	Daniel Borkmann <daniel@iogearbox.net>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>, shuah <shuah@kernel.org>
Subject: Re: [PATCH] selftests/bpf: split -extras target to -static and -gen
Date: Wed, 27 May 2020 16:01:34 -0600	[thread overview]
Message-ID: <ec5f6bd9-83e9-fc55-1885-18eee404d988@kernel.org> (raw)
In-Reply-To: <CAADnVQKbKA_Yuj7v3c6fNi7gZ8z_q_hzX2ry9optEHE3B_iWcg@mail.gmail.com>

On 5/27/20 11:05 AM, Alexei Starovoitov wrote:
> On Wed, May 27, 2020 at 10:02 AM Yauheni Kaliuta
> <yauheni.kaliuta@redhat.com> wrote:
>>
>> Hi, Alexei!
>>
>>>>>>> On Wed, 27 May 2020 09:48:04 -0700, Alexei Starovoitov  wrote:
>>
>>   > On Wed, May 27, 2020 at 12:19 AM Yauheni Kaliuta
>>   > <yauheni.kaliuta@redhat.com> wrote:
>>   >>
>>   >> Hi, Alexei!
>>   >>
>>   >> >>>>> On Tue, 26 May 2020 22:37:39 -0700, Alexei Starovoitov  wrote:
>>   >>
>>   >> > On Tue, May 26, 2020 at 10:31 PM Yauheni Kaliuta
>>   >> > <yauheni.kaliuta@redhat.com> wrote:
>>   >> >>
>>   >> >> Hi, Andrii!
>>   >> >>
>>   >> >> >>>>> On Tue, 26 May 2020 17:19:18 -0700, Andrii Nakryiko  wrote:
>>   >> >>
>>   >> >> > On Fri, May 22, 2020 at 1:19 AM Yauheni Kaliuta
>>   >> >> > <yauheni.kaliuta@redhat.com> wrote:
>>   >> >> >>
>>   >> >> >> There is difference in depoying static and generated extra resource
>>   >> >> >> files between in/out of tree build and flavors:
>>   >> >> >>
>>   >> >> >> - in case of unflavored out-of-tree build static files are not
>>   >> >> >> available and must be copied as well as both static and generated
>>   >> >> >> files for flavored build.
>>   >> >> >>
>>   >> >> >> So split the rules and variables. The name TRUNNER_EXTRA_GEN_FILES
>>   >> >> >> is chosen in analogy to TEST_GEN_* variants.
>>   >> >> >>
>>   >> >>
>>   >> >> > Can we keep them together but be smarter about what needs to
>>   >> >> > be copied based on source/target directories? I would really
>>   >> >> > like to not blow up all these rules.
>>   >> >>
>>   >> >> I can try, ok, I just find it a bit more clear. But it's good to
>>   >> >> get some input from kselftest about OOT build in general.
>>   >>
>>   >> > I see no value in 'make install' of selftests/bpf
>>   >> > and since it's broken just remove that makefile target.
>>   >>
>>   >> Some CI systems perform testing next stage after building were
>>   >> build tree is not available anymore. So it's in use at the
>>   >> moment.
>>
>>   > such CI systems can do 'cp -r' then
>> >> It's a discussion for linux-kselftest@ (added).
>>
>> At the moment `make install` is generic kselftest functionality
>> and since bpf is part of that infra it looks a bit strange to
>> break it intentionally.
> 
> selftests/bpf is only historically part of selftests.
> It probably should stop using kselftest build infra all together.
> We had breakages in selftests/bpf in the past only because
> of changes in kselftests bits.
> 

The question is whether or not the breakages addresses quickly.
Also, bpf keels breaking ksleftest builds and installs because
it has dependencies on bleeding edge tools and causes problems
for kselftest users.

You are pulling me into the discussion midway and I am missing the
context for the discussion. There is another thread on this topic
where Yauheni and I have been talking about bpf install.

I would say bpf install has never really worked from kselftest
install mechanism.

Ideally all tests use kselftest common install rule to leverage
the install and not have users do individual test installs.
It isn't productive and cooperative to say let's have bpf test
do its thing. It is part of selftests and we have to figure out
how to have it consistently build and run.

It isn't building for me on Linux 5.7-rc7 at the moment, leave
alone install.

The test Makefile has to handle OUTPUT directory. Please add me
to the entire patch series especially if it changes selftests
Makefile and lib.mk. I will review and try to see if we can make
bpf install work under kselftest common infrastructure.

I recently fixed bugs in kvm test Makefile to address the install
problems for cross-builds.

thanks,
-- Shuah