Re: [PATCH] selftests/bpf: split -extras target to -static and -gen

From: Shuah Khan <skhan@linuxfoundation.org>
To: Yauheni Kaliuta <yauheni.kaliuta@redhat.com>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>,
	Greg KH <gregkh@linuxfoundation.org>,
	Jiri Benc <jbenc@redhat.com>, shuah <shuah@kernel.org>,
	Andrii Nakryiko <andrii.nakryiko@gmail.com>,
	bpf <bpf@vger.kernel.org>, Jiri Olsa <jolsa@redhat.com>,
	Andrii Nakryiko <andriin@fb.com>,
	Daniel Borkmann <daniel@iogearbox.net>,
	"open list:KERNEL SELFTEST FRAMEWORK" 
	<linux-kselftest@vger.kernel.org>,
	skhan@linuxfoundation.org
Subject: Re: [PATCH] selftests/bpf: split -extras target to -static and -gen
Date: Thu, 28 May 2020 13:34:33 -0600	[thread overview]
Message-ID: <a80b0b4c-3ba3-e88d-b8d9-0164663b9c3f@linuxfoundation.org> (raw)
In-Reply-To: <CANoWswmSGHddKpSmyxvxjFEMZDB7u0cAp=ceGB6tU2AcdGRcaw@mail.gmail.com>

On 5/28/20 1:20 PM, Yauheni Kaliuta wrote:
> On Thu, May 28, 2020 at 10:09 PM Shuah Khan <skhan@linuxfoundation.org> wrote:
>>
>> On 5/28/20 11:10 AM, Yauheni Kaliuta wrote:
>>> Hi, Alexei,
>>>
>>> On Thu, May 28, 2020 at 7:14 PM Alexei Starovoitov
>>> <alexei.starovoitov@gmail.com> wrote:
>>>>
>>>> On Thu, May 28, 2020 at 12:56:31PM +0200, Greg KH wrote:
>>>>> On Thu, May 28, 2020 at 10:05:57AM +0200, Jiri Benc wrote:
>>>>>> On Wed, 27 May 2020 15:23:13 -0700, Alexei Starovoitov wrote:
>>>>>>> I prefer to keep selftests/bpf install broken.
>>>>>>> This forced marriage between kselftests and selftests/bpf
>>>>>>> never worked well. I think it's a time to free them up from each other.
>>>>>>
>>>>>> Alexei, it would be great if you could cooperate with other people
>>>>>> instead of pushing your own way. The selftests infrastructure was put
>>>>>> to the kernel to have one place for testing. Inventing yet another way
>>>>>> to add tests does not help anyone. You don't own the kernel. We're
>>>>>> community, we should cooperate.
>>>>>
>>>>> I agree, we rely on the infrastructure of the kselftests framework so
>>>>> that testing systems do not have to create "custom" frameworks to handle
>>>>> all of the individual variants that could easily crop up here.
>>>>>
>>>>> Let's keep it easy for people to run and use these tests, to not do so
>>>>> is to ensure that they are not used, which is the exact opposite goal of
>>>>> creating tests.
>>>>
>>>> Greg,
>>>>
>>>> It is easy for people (bpf developers) to run and use the tests.
>>>> Every developer runs them before submitting patches.
>>>> New tests is a hard requirement for any new features.
>>>> Maintainers run them for every push.
>>>>
>>>> What I was and will push back hard is when other people (not bpf developers)
>>>> come back with an excuse that some CI system has a hard time running these
>>>> tests. It's the problem of weak CI. That CI needs to be fixed. Not the tests.
>>>> The example of this is that we already have github/libbpf CI that runs
>>>> selftests/bpf just fine. Anyone who wants to do another CI are welcome to copy
>>>> paste what already works instead of burdening people (bpf developers) who run
>>>> and use existing tests. I frankly have no sympathy to folks who put their own
>>>> interest of their CI development in front of bpf community of developers.
>>>> The main job of CI is to help developers and maintainers.
>>>> Where helping means to not impose new dumb rules on developers because CI
>>>> framework is dumb. Fix CI instead.
>>>>
>>>
>>> Any good reason why bpf selftests, residing under selftests/, should
>>> be an exception?
>>> "Breakages" is not, breakages are fixable.
>>>
>>
>> Let's not talk about moving tests. I don't want to discuss that until
>> we are all on the same page on what is the problem in adding install
>> support to bpf Makefile.
>>
>> It is possible that there is a misunderstanding that bpf maintainer
>> and developer workflow will change. Which is definitely not needed.
>> If this patch series requires it, it isn't correct and needs to be
>> reworked.
> 
> patch series does not change it. Running bpf tests in-place is one of
> my own usecases, I'm not going to break it :)

Great. I wasn't suggesting you would. So there is no need for the
concern that bpf developer/maintainer workflow will have to change
to add install support.

> The series tried to fix what does not work (install and build in
> $(OUTPUT) directory) but exists in the code. I'll include you in Cc
> for the respin(s) if you are interested.

Please do.

> 
> The discussion is pretty much not connected to the patchset, but about
> bpf selftests and the infra in general.
> 

Yeah. We have to get a better understanding and be on the same page on
this anyway.

thanks,
-- Shuah