[LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Petr Vorel]

Suggested-by: Cyril Hrubis <chrubis@suse.cz>
Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
 doc/maintainer-patch-review-checklist.txt | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/doc/maintainer-patch-review-checklist.txt b/doc/maintainer-patch-review-checklist.txt
index c7bb47810..4e2b267ac 100644
--- a/doc/maintainer-patch-review-checklist.txt
+++ b/doc/maintainer-patch-review-checklist.txt
@@ -34,6 +34,9 @@ New test should
   GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
   unless test is completely rewritten
 * Old copyrights should be kept unless test is completely rewritten
+* Tests for new functionality in mainline kernel should be merged after final
+  release of kernel which contains that functionality (it's not enough when the
+  feature gets into rc1, because it can be reverted in later rc if problematic).
 
 ### C tests
 * Use new https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c[C API]
-- 
2.34.1


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Cyril Hrubis]

Hi!
> Suggested-by: Cyril Hrubis <chrubis@suse.cz>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
>  doc/maintainer-patch-review-checklist.txt | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/doc/maintainer-patch-review-checklist.txt b/doc/maintainer-patch-review-checklist.txt
> index c7bb47810..4e2b267ac 100644
> --- a/doc/maintainer-patch-review-checklist.txt
> +++ b/doc/maintainer-patch-review-checklist.txt
> @@ -34,6 +34,9 @@ New test should
>    GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
>    unless test is completely rewritten
>  * Old copyrights should be kept unless test is completely rewritten
> +* Tests for new functionality in mainline kernel should be merged after final
> +  release of kernel which contains that functionality (it's not enough when the
> +  feature gets into rc1, because it can be reverted in later rc if problematic).

Sounds reasonable to me, but ideally this should be acked by the rest of
the maintainers.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Petr Vorel]

Hi all,

> Hi!
> > Suggested-by: Cyril Hrubis <chrubis@suse.cz>
> > Signed-off-by: Petr Vorel <pvorel@suse.cz>
> > ---
> >  doc/maintainer-patch-review-checklist.txt | 3 +++
> >  1 file changed, 3 insertions(+)

> > diff --git a/doc/maintainer-patch-review-checklist.txt b/doc/maintainer-patch-review-checklist.txt
> > index c7bb47810..4e2b267ac 100644
> > --- a/doc/maintainer-patch-review-checklist.txt
> > +++ b/doc/maintainer-patch-review-checklist.txt
> > @@ -34,6 +34,9 @@ New test should
> >    GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
> >    unless test is completely rewritten
> >  * Old copyrights should be kept unless test is completely rewritten
> > +* Tests for new functionality in mainline kernel should be merged after final
> > +  release of kernel which contains that functionality (it's not enough when the
> > +  feature gets into rc1, because it can be reverted in later rc if problematic).

> Sounds reasonable to me, but ideally this should be acked by the rest of
> the maintainers.
Sure, this can wait :).

Cc maintainers directly.

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Mike Frysinger]

[-- Attachment #1.1: Type: text/plain, Size: 1177 bytes --]

On 11 Dec 2021 16:19, Petr Vorel wrote:
> > > Suggested-by: Cyril Hrubis <chrubis@suse.cz>
> > > Signed-off-by: Petr Vorel <pvorel@suse.cz>
> > > ---
> > >  doc/maintainer-patch-review-checklist.txt | 3 +++
> > >  1 file changed, 3 insertions(+)
> 
> > > diff --git a/doc/maintainer-patch-review-checklist.txt b/doc/maintainer-patch-review-checklist.txt
> > > index c7bb47810..4e2b267ac 100644
> > > --- a/doc/maintainer-patch-review-checklist.txt
> > > +++ b/doc/maintainer-patch-review-checklist.txt
> > > @@ -34,6 +34,9 @@ New test should
> > >    GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
> > >    unless test is completely rewritten
> > >  * Old copyrights should be kept unless test is completely rewritten
> > > +* Tests for new functionality in mainline kernel should be merged after final
> > > +  release of kernel which contains that functionality (it's not enough when the
> > > +  feature gets into rc1, because it can be reverted in later rc if problematic).
> 
> > Sounds reasonable to me, but ideally this should be acked by the rest of
> > the maintainers.
> 
> Sure, this can wait :).

seems fine
-mike

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Enji Cooper]

> On Dec 11, 2021, at 08:55, Mike Frysinger <vapier@gentoo.org> wrote:
> 
> On 11 Dec 2021 16:19, Petr Vorel wrote:
>>>> Suggested-by: Cyril Hrubis <chrubis@suse.cz>
>>>> Signed-off-by: Petr Vorel <pvorel@suse.cz>
>>>> ---
>>>> doc/maintainer-patch-review-checklist.txt | 3 +++
>>>> 1 file changed, 3 insertions(+)
>> 
>>>> diff --git a/doc/maintainer-patch-review-checklist.txt b/doc/maintainer-patch-review-checklist.txt
>>>> index c7bb47810..4e2b267ac 100644
>>>> --- a/doc/maintainer-patch-review-checklist.txt
>>>> +++ b/doc/maintainer-patch-review-checklist.txt
>>>> @@ -34,6 +34,9 @@ New test should
>>>>   GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
>>>>   unless test is completely rewritten
>>>> * Old copyrights should be kept unless test is completely rewritten
>>>> +* Tests for new functionality in mainline kernel should be merged after final
>>>> +  release of kernel which contains that functionality (it's not enough when the
>>>> +  feature gets into rc1, because it can be reverted in later rc if problematic).
>> 
>>> Sounds reasonable to me, but ideally this should be acked by the rest of
>>> the maintainers.
>> 
>> Sure, this can wait :).
> 
> seems fine

ACK

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Li Wang]

[-- Attachment #1.1: Type: text/plain, Size: 1261 bytes --]

On Fri, Dec 10, 2021 at 9:46 PM Petr Vorel <pvorel@suse.cz> wrote:

> Suggested-by: Cyril Hrubis <chrubis@suse.cz>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
>
Reviewed-by: Li Wang <liwang@redhat.com>



> ---
>  doc/maintainer-patch-review-checklist.txt | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/doc/maintainer-patch-review-checklist.txt
> b/doc/maintainer-patch-review-checklist.txt
> index c7bb47810..4e2b267ac 100644
> --- a/doc/maintainer-patch-review-checklist.txt
> +++ b/doc/maintainer-patch-review-checklist.txt
> @@ -34,6 +34,9 @@ New test should
>    GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
>    unless test is completely rewritten
>  * Old copyrights should be kept unless test is completely rewritten
> +* Tests for new functionality in mainline kernel should be merged after
> final
> +  release of kernel which contains that functionality (it's not enough
> when the
> +  feature gets into rc1, because it can be reverted in later rc if
> problematic).
>
>  ### C tests
>  * Use new
> https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c[C
> API]
> --
> 2.34.1
>
>
> --
> Mailing list info: https://lists.linux.it/listinfo/ltp
>
>

-- 
Regards,
Li Wang

[-- Attachment #1.2: Type: text/html, Size: 2567 bytes --]

[-- Attachment #2: Type: text/plain, Size: 60 bytes --]


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Jan Stancek]

On Fri, Dec 10, 2021 at 2:46 PM Petr Vorel <pvorel@suse.cz> wrote:
>
> Suggested-by: Cyril Hrubis <chrubis@suse.cz>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
>  doc/maintainer-patch-review-checklist.txt | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/doc/maintainer-patch-review-checklist.txt b/doc/maintainer-patch-review-checklist.txt
> index c7bb47810..4e2b267ac 100644
> --- a/doc/maintainer-patch-review-checklist.txt
> +++ b/doc/maintainer-patch-review-checklist.txt
> @@ -34,6 +34,9 @@ New test should
>    GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
>    unless test is completely rewritten
>  * Old copyrights should be kept unless test is completely rewritten
> +* Tests for new functionality in mainline kernel should be merged after final
> +  release of kernel which contains that functionality (it's not enough when the
> +  feature gets into rc1, because it can be reverted in later rc if problematic).

Acked-by: Jan Stancek <jstancek@redhat.com>


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Richard Palethorpe]

Hello Petr,

Petr Vorel <pvorel@suse.cz> writes:

> Suggested-by: Cyril Hrubis <chrubis@suse.cz>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
>  doc/maintainer-patch-review-checklist.txt | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/doc/maintainer-patch-review-checklist.txt b/doc/maintainer-patch-review-checklist.txt
> index c7bb47810..4e2b267ac 100644
> --- a/doc/maintainer-patch-review-checklist.txt
> +++ b/doc/maintainer-patch-review-checklist.txt
> @@ -34,6 +34,9 @@ New test should
>    GPL-2.0-or-later; the licence for test (e.g. GPL-2.0) should not change
>    unless test is completely rewritten
>  * Old copyrights should be kept unless test is completely rewritten
> +* Tests for new functionality in mainline kernel should be merged after final
> +  release of kernel which contains that functionality (it's not enough when the
> +  feature gets into rc1, because it can be reverted in later rc if
> problematic).

What is the concern? All I can see is that we merge a test which is for
a feature that is never included

The issue is we may forget to merge patch sets for features which are
included (a far worse result). It's more stuff waiting around in the
queue. At the least we should have a procedure for tracking them (like
tagging github issues for review at each mainline release).

If a test requires a kernel config which doesn't exist in mainline we
could also look for that automatically.

>  
>  ### C tests
>  * Use new https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c[C API]
> -- 
> 2.34.1


-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Cyril Hrubis]

Hi!
> > +* Tests for new functionality in mainline kernel should be merged after final
> > +  release of kernel which contains that functionality (it's not enough when the
> > +  feature gets into rc1, because it can be reverted in later rc if
> > problematic).
> 
> What is the concern? All I can see is that we merge a test which is for
> a feature that is never included

Not only that, the interface may change subtly.

> The issue is we may forget to merge patch sets for features which are
> included (a far worse result). It's more stuff waiting around in the
> queue. At the least we should have a procedure for tracking them (like
> tagging github issues for review at each mainline release).
> 
> If a test requires a kernel config which doesn't exist in mainline we
> could also look for that automatically.

The main issue is that if we happen to release LTP meanwhile with a test
for a syscall that didn't get included in the mainline in the end we
have released LTP that is supposed to be stable and the test will start
to fail when the syscall number is allocated for something else which
will happen sooner or later.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[xuyang2018.jy]

Hi!
> Hi!
>>> +* Tests for new functionality in mainline kernel should be merged after final
>>> +  release of kernel which contains that functionality (it's not enough when the
>>> +  feature gets into rc1, because it can be reverted in later rc if
>>> problematic).
>>
>> What is the concern? All I can see is that we merge a test which is for
>> a feature that is never included
>
> Not only that, the interface may change subtly.
>
>> The issue is we may forget to merge patch sets for features which are
>> included (a far worse result). It's more stuff waiting around in the
>> queue. At the least we should have a procedure for tracking them (like
>> tagging github issues for review at each mainline release).
>>
>> If a test requires a kernel config which doesn't exist in mainline we
>> could also look for that automatically.
>
> The main issue is that if we happen to release LTP meanwhile with a test
> for a syscall that didn't get included in the mainline in the end we
> have released LTP that is supposed to be stable and the test will start
> to fail when the syscall number is allocated for something else which
> will happen sooner or later.
I know a example that is quotactl_path syscall.
>

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Richard Palethorpe]

Hello,

"xuyang2018.jy@fujitsu.com" <xuyang2018.jy@fujitsu.com> writes:

> Hi!
>> Hi!
>>>> +* Tests for new functionality in mainline kernel should be merged after final
>>>> +  release of kernel which contains that functionality (it's not enough when the
>>>> +  feature gets into rc1, because it can be reverted in later rc if
>>>> problematic).
>>>
>>> What is the concern? All I can see is that we merge a test which is for
>>> a feature that is never included
>>
>> Not only that, the interface may change subtly.

That can always happen as plenty of changes will break LTP test
expectations, but not real programs.

>>
>>> The issue is we may forget to merge patch sets for features which are
>>> included (a far worse result). It's more stuff waiting around in the
>>> queue. At the least we should have a procedure for tracking them (like
>>> tagging github issues for review at each mainline release).
>>>
>>> If a test requires a kernel config which doesn't exist in mainline we
>>> could also look for that automatically.
>>
>> The main issue is that if we happen to release LTP meanwhile with a test
>> for a syscall that didn't get included in the mainline in the end we
>> have released LTP that is supposed to be stable and the test will start
>> to fail when the syscall number is allocated for something else which
>> will happen sooner or later.
> I know a example that is quotactl_path syscall.
>>

If the real issue is LTP releases, then why not exclude tests for new
features from them? I assume it's only a small number of commits which
would need to be removed. Possibly we could tag them in git when merging
so it is not a lot more work for whoever does the release (namely
Cyril) to create a branch without them.

My main concern is this will throw up a barrier to motivated
contributors working on the cutting edge.

-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Cyril Hrubis]

Hi!
> >>> The issue is we may forget to merge patch sets for features which are
> >>> included (a far worse result). It's more stuff waiting around in the
> >>> queue. At the least we should have a procedure for tracking them (like
> >>> tagging github issues for review at each mainline release).
> >>>
> >>> If a test requires a kernel config which doesn't exist in mainline we
> >>> could also look for that automatically.
> >>
> >> The main issue is that if we happen to release LTP meanwhile with a test
> >> for a syscall that didn't get included in the mainline in the end we
> >> have released LTP that is supposed to be stable and the test will start
> >> to fail when the syscall number is allocated for something else which
> >> will happen sooner or later.
> > I know a example that is quotactl_path syscall.
> >>
> 
> If the real issue is LTP releases, then why not exclude tests for new
> features from them? I assume it's only a small number of commits which
> would need to be removed. Possibly we could tag them in git when merging
> so it is not a lot more work for whoever does the release (namely
> Cyril) to create a branch without them.

That sounds too complex for a test or two we are usually getting during
the release cycle.

Note that people who contribute the functionality to the kernel are used
to wait for next release window, kernel releases are aprox. twice as
fast as LTP.

> My main concern is this will throw up a barrier to motivated
> contributors working on the cutting edge.

So far really nobody complained, which may not be a good metric. But
still unless there is a evidence that this happens I wouldn't consider
spending effort on this.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Richard Palethorpe]

Hello,

Cyril Hrubis <chrubis@suse.cz> writes:

> Hi!
>> >>> The issue is we may forget to merge patch sets for features which are
>> >>> included (a far worse result). It's more stuff waiting around in the
>> >>> queue. At the least we should have a procedure for tracking them (like
>> >>> tagging github issues for review at each mainline release).
>> >>>
>> >>> If a test requires a kernel config which doesn't exist in mainline we
>> >>> could also look for that automatically.
>> >>
>> >> The main issue is that if we happen to release LTP meanwhile with a test
>> >> for a syscall that didn't get included in the mainline in the end we
>> >> have released LTP that is supposed to be stable and the test will start
>> >> to fail when the syscall number is allocated for something else which
>> >> will happen sooner or later.
>> > I know a example that is quotactl_path syscall.
>> >>
>> 
>> If the real issue is LTP releases, then why not exclude tests for new
>> features from them? I assume it's only a small number of commits which
>> would need to be removed. Possibly we could tag them in git when merging
>> so it is not a lot more work for whoever does the release (namely
>> Cyril) to create a branch without them.
>
> That sounds too complex for a test or two we are usually getting during
> the release cycle.
>
> Note that people who contribute the functionality to the kernel are used
> to wait for next release window, kernel releases are aprox. twice as
> fast as LTP.
>
>> My main concern is this will throw up a barrier to motivated
>> contributors working on the cutting edge.
>
> So far really nobody complained, which may not be a good metric. But
> still unless there is a evidence that this happens I wouldn't consider
> spending effort on this.

OK, well if it comes up again we can revisit it. However Petr please
could you add the reasoning about not adding unstable tests into
releases.

With that you can add

Acked-by: Richard Palethorpe <rpalethorpe@suse.com>


-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Petr Vorel]

Hi Richie, all,

> Hello,

> Cyril Hrubis <chrubis@suse.cz> writes:

> > Hi!
> >> >>> The issue is we may forget to merge patch sets for features which are
> >> >>> included (a far worse result). It's more stuff waiting around in the
> >> >>> queue. At the least we should have a procedure for tracking them (like
> >> >>> tagging github issues for review at each mainline release).

> >> >>> If a test requires a kernel config which doesn't exist in mainline we
> >> >>> could also look for that automatically.

> >> >> The main issue is that if we happen to release LTP meanwhile with a test
> >> >> for a syscall that didn't get included in the mainline in the end we
> >> >> have released LTP that is supposed to be stable and the test will start
> >> >> to fail when the syscall number is allocated for something else which
> >> >> will happen sooner or later.
> >> > I know a example that is quotactl_path syscall.


> >> If the real issue is LTP releases, then why not exclude tests for new
> >> features from them? I assume it's only a small number of commits which
> >> would need to be removed. Possibly we could tag them in git when merging
> >> so it is not a lot more work for whoever does the release (namely
> >> Cyril) to create a branch without them.

> > That sounds too complex for a test or two we are usually getting during
> > the release cycle.

> > Note that people who contribute the functionality to the kernel are used
> > to wait for next release window, kernel releases are aprox. twice as
> > fast as LTP.

> >> My main concern is this will throw up a barrier to motivated
> >> contributors working on the cutting edge.

> > So far really nobody complained, which may not be a good metric. But
> > still unless there is a evidence that this happens I wouldn't consider
> > spending effort on this.

> OK, well if it comes up again we can revisit it. However Petr please
> could you add the reasoning about not adding unstable tests into
> releases.
Isn't "... because it can be reverted in later rc if problematic" enough?
If not what would you add? Cyril posted test getting released in LTP and later
unstable. I could be more verbose, but not sure if that's better:

* Tests for new functionality in mainline kernel should be merged after final
  release of kernel which contains that functionality. It's not enough when the
  feature gets into rc1, because in later rc the functionality can be changed or
  even reverted if problematic. And this could lead to broken test.

> With that you can add

> Acked-by: Richard Palethorpe <rpalethorpe@suse.com>

Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Richard Palethorpe]

Hello Petr,

Petr Vorel <pvorel@suse.cz> writes:

> Hi Richie, all,
>
>> Hello,
>
>> Cyril Hrubis <chrubis@suse.cz> writes:
>
>> > Hi!
>> >> >>> The issue is we may forget to merge patch sets for features which are
>> >> >>> included (a far worse result). It's more stuff waiting around in the
>> >> >>> queue. At the least we should have a procedure for tracking them (like
>> >> >>> tagging github issues for review at each mainline release).
>
>> >> >>> If a test requires a kernel config which doesn't exist in mainline we
>> >> >>> could also look for that automatically.
>
>> >> >> The main issue is that if we happen to release LTP meanwhile with a test
>> >> >> for a syscall that didn't get included in the mainline in the end we
>> >> >> have released LTP that is supposed to be stable and the test will start
>> >> >> to fail when the syscall number is allocated for something else which
>> >> >> will happen sooner or later.
>> >> > I know a example that is quotactl_path syscall.
>
>
>> >> If the real issue is LTP releases, then why not exclude tests for new
>> >> features from them? I assume it's only a small number of commits which
>> >> would need to be removed. Possibly we could tag them in git when merging
>> >> so it is not a lot more work for whoever does the release (namely
>> >> Cyril) to create a branch without them.
>
>> > That sounds too complex for a test or two we are usually getting during
>> > the release cycle.
>
>> > Note that people who contribute the functionality to the kernel are used
>> > to wait for next release window, kernel releases are aprox. twice as
>> > fast as LTP.
>
>> >> My main concern is this will throw up a barrier to motivated
>> >> contributors working on the cutting edge.
>
>> > So far really nobody complained, which may not be a good metric. But
>> > still unless there is a evidence that this happens I wouldn't consider
>> > spending effort on this.
>
>> OK, well if it comes up again we can revisit it. However Petr please
>> could you add the reasoning about not adding unstable tests into
>> releases.
> Isn't "... because it can be reverted in later rc if problematic" enough?
> If not what would you add? Cyril posted test getting released in LTP and later
> unstable. I could be more verbose, but not sure if that's better:
>
> * Tests for new functionality in mainline kernel should be merged after final
>   release of kernel which contains that functionality. It's not enough when the
>   feature gets into rc1, because in later rc the functionality can be changed or
>   even reverted if problematic. And this could lead to broken test.

It's only a problem because of LTP releases. We can just fix or revert
tests otherwise. Especially if the test author is also developing the
kernel feature and updating the test in lockstep.

Without LTP releases then a test can be added for a feature only in RC
and removed again. Only people who test the RC will get a result other
than TCONF from the test before it disappears again or is updated to
support the final feature.

Assuming of course people update LTP from HEAD on a regular basis. Which
of course they won't or can't, so we have releases. We don't have a
separate unstable branch because no one has volunteered to create and
maintain it. If we did we could just add such tests to that.

>
>> With that you can add
>
>> Acked-by: Richard Palethorpe <rpalethorpe@suse.com>
>
> Kind regards,
> Petr


-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Petr Vorel]

Hi Richie, Cyril,

> Hello Petr,

> Petr Vorel <pvorel@suse.cz> writes:

> > Hi Richie, all,

> >> Hello,

> >> Cyril Hrubis <chrubis@suse.cz> writes:

> >> > Hi!
> >> >> >>> The issue is we may forget to merge patch sets for features which are
> >> >> >>> included (a far worse result). It's more stuff waiting around in the
> >> >> >>> queue. At the least we should have a procedure for tracking them (like
> >> >> >>> tagging github issues for review at each mainline release).

> >> >> >>> If a test requires a kernel config which doesn't exist in mainline we
> >> >> >>> could also look for that automatically.

> >> >> >> The main issue is that if we happen to release LTP meanwhile with a test
> >> >> >> for a syscall that didn't get included in the mainline in the end we
> >> >> >> have released LTP that is supposed to be stable and the test will start
> >> >> >> to fail when the syscall number is allocated for something else which
> >> >> >> will happen sooner or later.
> >> >> > I know a example that is quotactl_path syscall.


> >> >> If the real issue is LTP releases, then why not exclude tests for new
> >> >> features from them? I assume it's only a small number of commits which
> >> >> would need to be removed. Possibly we could tag them in git when merging
> >> >> so it is not a lot more work for whoever does the release (namely
> >> >> Cyril) to create a branch without them.

> >> > That sounds too complex for a test or two we are usually getting during
> >> > the release cycle.

> >> > Note that people who contribute the functionality to the kernel are used
> >> > to wait for next release window, kernel releases are aprox. twice as
> >> > fast as LTP.

> >> >> My main concern is this will throw up a barrier to motivated
> >> >> contributors working on the cutting edge.

> >> > So far really nobody complained, which may not be a good metric. But
> >> > still unless there is a evidence that this happens I wouldn't consider
> >> > spending effort on this.

> >> OK, well if it comes up again we can revisit it. However Petr please
> >> could you add the reasoning about not adding unstable tests into
> >> releases.
> > Isn't "... because it can be reverted in later rc if problematic" enough?
> > If not what would you add? Cyril posted test getting released in LTP and later
> > unstable. I could be more verbose, but not sure if that's better:

> > * Tests for new functionality in mainline kernel should be merged after final
> >   release of kernel which contains that functionality. It's not enough when the
> >   feature gets into rc1, because in later rc the functionality can be changed or
> >   even reverted if problematic. And this could lead to broken test.

> It's only a problem because of LTP releases. We can just fix or revert
> tests otherwise. Especially if the test author is also developing the
> kernel feature and updating the test in lockstep.

> Without LTP releases then a test can be added for a feature only in RC
> and removed again. Only people who test the RC will get a result other
> than TCONF from the test before it disappears again or is updated to
> support the final feature.

> Assuming of course people update LTP from HEAD on a regular basis. Which
> of course they won't or can't, so we have releases. We don't have a
> separate unstable branch because no one has volunteered to create and
> maintain it. If we did we could just add such tests to that.

Agree, but we *do* have releases and do *not* have unstable branch.

Thus, first, do you agree with current policy?
If yes, how would you phrase it (do you want to add / change anything)?

Or do you suggest to have policy when merged to rc1?
It would work for me, but we'd have to ask all maintainers
(I suggested that before, Cyril preferred kernel release).

Kind regards,
Petr

> >> With that you can add

> >> Acked-by: Richard Palethorpe <rpalethorpe@suse.com>

> > Kind regards,
> > Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Richard Palethorpe]

Hello Petr,

Petr Vorel <pvorel@suse.cz> writes:

> Hi Richie, Cyril,
>
>> Hello Petr,
>
>> Petr Vorel <pvorel@suse.cz> writes:
>
>> > Hi Richie, all,
>
>> >> Hello,
>
>> >> Cyril Hrubis <chrubis@suse.cz> writes:
>
>> >> > Hi!
>> >> >> >>> The issue is we may forget to merge patch sets for features which are
>> >> >> >>> included (a far worse result). It's more stuff waiting around in the
>> >> >> >>> queue. At the least we should have a procedure for tracking them (like
>> >> >> >>> tagging github issues for review at each mainline release).
>
>> >> >> >>> If a test requires a kernel config which doesn't exist in mainline we
>> >> >> >>> could also look for that automatically.
>
>> >> >> >> The main issue is that if we happen to release LTP meanwhile with a test
>> >> >> >> for a syscall that didn't get included in the mainline in the end we
>> >> >> >> have released LTP that is supposed to be stable and the test will start
>> >> >> >> to fail when the syscall number is allocated for something else which
>> >> >> >> will happen sooner or later.
>> >> >> > I know a example that is quotactl_path syscall.
>
>
>> >> >> If the real issue is LTP releases, then why not exclude tests for new
>> >> >> features from them? I assume it's only a small number of commits which
>> >> >> would need to be removed. Possibly we could tag them in git when merging
>> >> >> so it is not a lot more work for whoever does the release (namely
>> >> >> Cyril) to create a branch without them.
>
>> >> > That sounds too complex for a test or two we are usually getting during
>> >> > the release cycle.
>
>> >> > Note that people who contribute the functionality to the kernel are used
>> >> > to wait for next release window, kernel releases are aprox. twice as
>> >> > fast as LTP.
>
>> >> >> My main concern is this will throw up a barrier to motivated
>> >> >> contributors working on the cutting edge.
>
>> >> > So far really nobody complained, which may not be a good metric. But
>> >> > still unless there is a evidence that this happens I wouldn't consider
>> >> > spending effort on this.
>
>> >> OK, well if it comes up again we can revisit it. However Petr please
>> >> could you add the reasoning about not adding unstable tests into
>> >> releases.
>> > Isn't "... because it can be reverted in later rc if problematic" enough?
>> > If not what would you add? Cyril posted test getting released in LTP and later
>> > unstable. I could be more verbose, but not sure if that's better:
>
>> > * Tests for new functionality in mainline kernel should be merged after final
>> >   release of kernel which contains that functionality. It's not enough when the
>> >   feature gets into rc1, because in later rc the functionality can be changed or
>> >   even reverted if problematic. And this could lead to broken test.
>
>> It's only a problem because of LTP releases. We can just fix or revert
>> tests otherwise. Especially if the test author is also developing the
>> kernel feature and updating the test in lockstep.
>
>> Without LTP releases then a test can be added for a feature only in RC
>> and removed again. Only people who test the RC will get a result other
>> than TCONF from the test before it disappears again or is updated to
>> support the final feature.
>
>> Assuming of course people update LTP from HEAD on a regular basis. Which
>> of course they won't or can't, so we have releases. We don't have a
>> separate unstable branch because no one has volunteered to create and
>> maintain it. If we did we could just add such tests to that.
>
> Agree, but we *do* have releases and do *not* have unstable branch.
>
> Thus, first, do you agree with current policy?

Yes. Although we could add "next" and "rc" flags to tst_test (or
similar). Then require an environment variable to be set (or check the
kernel version) otherwise the test will return TCONF.

For LTP releases we just need to check if the flags are still needed or
if the feature has been merged. The metadata parser can generate a list
of tests to check.

This seems like quite little work to me. In fact we don't even have to
implement it until someone wants it. We can just add it to the policy.

> If yes, how would you phrase it (do you want to add / change
> anything)?

Something like:

"Tests for new functionality should only be added to the LTP once they
are part of the stable kernel ABI. This happens when a feature is
included in a final kernel release. Not during the RC phase where they
can still be removed and, for example, the syscall numbers reused.

This allows stable LTP releases to be taken directly from Git HEAD at
any time we are ready. Without needing to remove commits for unstable
feature tests or synchronise with the kernel releases.

Alternatively if someone is very motivated to add tests for linux-next
or the RC phase. We can add flags to tst_test which will prevent these
tests being run under normal circumstances. Meaning the releases are
unaffected."

>
> Or do you suggest to have policy when merged to rc1?
> It would work for me, but we'd have to ask all maintainers
> (I suggested that before, Cyril preferred kernel release).

I'd happily accept tests for things going into linux-next if there are
people willing to write them and they do not "throw them over the
wall and run".

>
> Kind regards,
> Petr
>
>> >> With that you can add
>
>> >> Acked-by: Richard Palethorpe <rpalethorpe@suse.com>
>
>> > Kind regards,
>> > Petr


-- 
Thank you,
Richard.

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Petr Vorel]

Hi Richie, Cyril,

<snip>
> > Thus, first, do you agree with current policy?

> Yes. Although we could add "next" and "rc" flags to tst_test (or
> similar). Then require an environment variable to be set (or check the
> kernel version) otherwise the test will return TCONF.

> For LTP releases we just need to check if the flags are still needed or
> if the feature has been merged. The metadata parser can generate a list
> of tests to check.

> This seems like quite little work to me. In fact we don't even have to
> implement it until someone wants it. We can just add it to the policy.

Yes, adding flag would work.

> > If yes, how would you phrase it (do you want to add / change
> > anything)?

> Something like:

> "Tests for new functionality should only be added to the LTP once they
> are part of the stable kernel ABI. This happens when a feature is
> included in a final kernel release. Not during the RC phase where they
> can still be removed and, for example, the syscall numbers reused.

> This allows stable LTP releases to be taken directly from Git HEAD at
> any time we are ready. Without needing to remove commits for unstable
> feature tests or synchronise with the kernel releases.

> Alternatively if someone is very motivated to add tests for linux-next
> or the RC phase. We can add flags to tst_test which will prevent these
> tests being run under normal circumstances. Meaning the releases are
> unaffected."

OK, we document our willingness to add infrastructure for merging in rc,
if desired. I'm ok with this, if Cyril agree, I can send v2 for others to
ack/review it.

> > Or do you suggest to have policy when merged to rc1?
> > It would work for me, but we'd have to ask all maintainers
> > (I suggested that before, Cyril preferred kernel release).

> I'd happily accept tests for things going into linux-next if there are
> people willing to write them and they do not "throw them over the
> wall and run".

IMHO it's up to us whether we want to implement. I know just about few tests in
last 2 years - fanotify and IMA subsystems which has brought tests and they just
accepted our decision when we're going to merge it. People are happy that we
help them in tests, nobody haven't felt being put off by merging later.


Kind regards,
Petr

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

Re: [LTP] [PATCH 1/1] doc/maintainer: Add policy for new functionality

[Cyril Hrubis]

Hi!
> Yes. Although we could add "next" and "rc" flags to tst_test (or
> similar). Then require an environment variable to be set (or check the
> kernel version) otherwise the test will return TCONF.
> 
> For LTP releases we just need to check if the flags are still needed or
> if the feature has been merged. The metadata parser can generate a list
> of tests to check.
> 
> This seems like quite little work to me. In fact we don't even have to
> implement it until someone wants it. We can just add it to the policy.

I was thinking of this and if we really want this feature it would make
sense to add "remove_before_use" flag to the test structure what would
render the test resultless. Ideally it would include the kernel version
the functionality is going to be included into, then we can
automatically check in the CI the test metadata against latest release
kernel version and either remove the flag or bump it in case it didn't
get in.

Maybe just .remove_after_release = "5.9"

@Ritchie feel free to go ahead if you want to implement and maintain
         something like this.

-- 
Cyril Hrubis
chrubis@suse.cz

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp