Re: [PATCH] selftests/bpf: split -extras target to -static and -gen

From: Shuah Khan <skhan@linuxfoundation.org>
To: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>,
	Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>,
	Jiri Benc <jbenc@redhat.com>, shuah <shuah@kernel.org>,
	Andrii Nakryiko <andrii.nakryiko@gmail.com>,
	bpf <bpf@vger.kernel.org>, Jiri Olsa <jolsa@redhat.com>,
	Andrii Nakryiko <andriin@fb.com>,
	Daniel Borkmann <daniel@iogearbox.net>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	skhan@linuxfoundation.org
Subject: Re: [PATCH] selftests/bpf: split -extras target to -static and -gen
Date: Thu, 28 May 2020 13:09:05 -0600	[thread overview]
Message-ID: <49931ed9-da92-4b32-ba54-aeba33166bdd@linuxfoundation.org> (raw)
In-Reply-To: <CANoWswkGoJEZVcfLiNverDWyh6skSoix=JqxeJR9K8A=H8x=rw@mail.gmail.com>

On 5/28/20 11:10 AM, Yauheni Kaliuta wrote:
> Hi, Alexei,
> 
> On Thu, May 28, 2020 at 7:14 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
>>
>> On Thu, May 28, 2020 at 12:56:31PM +0200, Greg KH wrote:
>>> On Thu, May 28, 2020 at 10:05:57AM +0200, Jiri Benc wrote:
>>>> On Wed, 27 May 2020 15:23:13 -0700, Alexei Starovoitov wrote:
>>>>> I prefer to keep selftests/bpf install broken.
>>>>> This forced marriage between kselftests and selftests/bpf
>>>>> never worked well. I think it's a time to free them up from each other.
>>>>
>>>> Alexei, it would be great if you could cooperate with other people
>>>> instead of pushing your own way. The selftests infrastructure was put
>>>> to the kernel to have one place for testing. Inventing yet another way
>>>> to add tests does not help anyone. You don't own the kernel. We're
>>>> community, we should cooperate.
>>>
>>> I agree, we rely on the infrastructure of the kselftests framework so
>>> that testing systems do not have to create "custom" frameworks to handle
>>> all of the individual variants that could easily crop up here.
>>>
>>> Let's keep it easy for people to run and use these tests, to not do so
>>> is to ensure that they are not used, which is the exact opposite goal of
>>> creating tests.
>>
>> Greg,
>>
>> It is easy for people (bpf developers) to run and use the tests.
>> Every developer runs them before submitting patches.
>> New tests is a hard requirement for any new features.
>> Maintainers run them for every push.
>>
>> What I was and will push back hard is when other people (not bpf developers)
>> come back with an excuse that some CI system has a hard time running these
>> tests. It's the problem of weak CI. That CI needs to be fixed. Not the tests.
>> The example of this is that we already have github/libbpf CI that runs
>> selftests/bpf just fine. Anyone who wants to do another CI are welcome to copy
>> paste what already works instead of burdening people (bpf developers) who run
>> and use existing tests. I frankly have no sympathy to folks who put their own
>> interest of their CI development in front of bpf community of developers.
>> The main job of CI is to help developers and maintainers.
>> Where helping means to not impose new dumb rules on developers because CI
>> framework is dumb. Fix CI instead.
>>
> 
> Any good reason why bpf selftests, residing under selftests/, should
> be an exception?
> "Breakages" is not, breakages are fixable.
> 

Let's not talk about moving tests. I don't want to discuss that until
we are all on the same page on what is the problem in adding install
support to bpf Makefile.

It is possible that there is a misunderstanding that bpf maintainer
and developer workflow will change. Which is definitely not needed.
If this patch series requires it, it isn't correct and needs to be
reworked.

thanks,
-- Shuah