[PATCH] mm: vmscan: provide a change to the development-process group

[PATCH] mm: vmscan: provide a change to the development-process group

[Lukas Bulwahn]

I think this change is needed for safety, whatever that might mean to you.

I am unqualified to really make a change here, as I have no clue what this
code does, nor what my change does, but sure, the testing and verification
reference process can now point out the required next steps in the
reference process to test this code and code change.

Good luck :)

Not intended for distribution to the general kernel mailing lists.

Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
---
I would like to submit such a patch, what do I need to do according to
the expected testing and verification recommendations for safety-related
systems?

Please help me. What do I need to compile, what test do I need to run,
which verification tool do I need to employ for this change?

Or even more basic: where would I even find that information?

 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 980155e257bf..aef8060efa8f 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3893,7 +3893,7 @@ static int kswapd(void *p)
 					highest_zoneidx);
 
 		/* Read the new order and highest_zoneidx */
-		alloc_order = reclaim_order = READ_ONCE(pgdat->kswapd_order);
+		alloc_order = READ_ONCE(pgdat->kswapd_order);
 		highest_zoneidx = kswapd_highest_zoneidx(pgdat,
 							highest_zoneidx);
 		WRITE_ONCE(pgdat->kswapd_order, 0);
-- 
2.17.1

Re: [ELISA Development Process WG] [PATCH] mm: vmscan: provide a change to the development-process group

[Paoloni, Gabriele]

> -----Original Message-----
> From: development-process@lists.elisa.tech <development-
> process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> Sent: Thursday, September 17, 2020 10:44 AM
> To: linux-safety@lists.elisa.tech
> Cc: development-process@lists.elisa.tech; Lukas Bulwahn
> <lukas.bulwahn@gmail.com>
> Subject: [ELISA Development Process WG] [PATCH] mm: vmscan: provide a
> change to the development-process group
> 
> I think this change is needed for safety, whatever that might mean to you.
> 
> I am unqualified to really make a change here, as I have no clue what this
> code does, nor what my change does, but sure, the testing and verification
> reference process can now point out the required next steps in the
> reference process to test this code and code change.
> 
> Good luck :)
> 
> Not intended for distribution to the general kernel mailing lists.
> 
> Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> ---
> I would like to submit such a patch, what do I need to do according to
> the expected testing and verification recommendations for safety-related
> systems?

Probably the problem is not just limited to submitting patches, but I think it
is a good starting point.

> 
> Please help me. What do I need to compile, what test do I need to run,
> which verification tool do I need to employ for this change?

Right. I have tried to reformulate the problem as "ok I am looking at or I am
trying to change one or more code lines, so I would like to understand if such
code lines are tested today and how".
Right now if we had a s maintained structural coverage report we could tell
if the impacted code lines are covered and by which tests...I could not find it
so I guess that so far we are missing it....am I wrong?

As next step I went to look at the function wrapping the impacted code line
that in this case is kswapd. I could not find it tested in the Kunit framework.

As next steps then I would run the available tests in
https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests
in conjunction with GCOV trying to figure out if that line is covered already
somehow.

There are probably smarter, better, faster ways to elaborate on this...so
here I just put down what I would do in order to figure this out...

Gab


> 
> Or even more basic: where would I even find that information?
> 
>  mm/vmscan.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index 980155e257bf..aef8060efa8f 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -3893,7 +3893,7 @@ static int kswapd(void *p)
>  					highest_zoneidx);
> 
>  		/* Read the new order and highest_zoneidx */
> -		alloc_order = reclaim_order = READ_ONCE(pgdat-
> >kswapd_order);
> +		alloc_order = READ_ONCE(pgdat->kswapd_order);
>  		highest_zoneidx = kswapd_highest_zoneidx(pgdat,
>  							highest_zoneidx);
>  		WRITE_ONCE(pgdat->kswapd_order, 0);
> --
> 2.17.1
> 
> 
> 
> 
> 

---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4 
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale  04236760155
Repertorio Economico Amministrativo n. 997124 
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di 
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [linux-safety] [PATCH] mm: vmscan: provide a change to the development-process group

[Sudip Mukherjee]

On 17/09/2020 09:44, Lukas Bulwahn wrote:
> I think this change is needed for safety, whatever that might mean to you.
> 
> I am unqualified to really make a change here, as I have no clue what this
> code does, nor what my change does, but sure, the testing and verification
> reference process can now point out the required next steps in the
> reference process to test this code and code change.
> 
> Good luck :)
> 
> Not intended for distribution to the general kernel mailing lists.
> 
> Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> ---
> I would like to submit such a patch, what do I need to do according to
> the expected testing and verification recommendations for safety-related
> systems?
> 
> Please help me. What do I need to compile, what test do I need to run,
> which verification tool do I need to employ for this change?

The change looks valid, 'reclaim_order' has not been used anywhere after
READ_ONCE(), and its  So, it looks like a harmless change, you will only
need a good commit message detailing why its harmless.

So, from a safety pov, is it a requirement that every submitted patch
will need to be tested based on the safety tests and all the other
defined tests?



-- 
Regards
Sudip

Re: [linux-safety] [PATCH] mm: vmscan: provide a change to the development-process group

[Lukas Bulwahn]

On Thu, 17 Sep 2020, Sudip Mukherjee wrote:

> 
> 
> On 17/09/2020 09:44, Lukas Bulwahn wrote:
> > I think this change is needed for safety, whatever that might mean to you.
> > 
> > I am unqualified to really make a change here, as I have no clue what this
> > code does, nor what my change does, but sure, the testing and verification
> > reference process can now point out the required next steps in the
> > reference process to test this code and code change.
> > 
> > Good luck :)
> > 
> > Not intended for distribution to the general kernel mailing lists.
> > 
> > Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> > ---
> > I would like to submit such a patch, what do I need to do according to
> > the expected testing and verification recommendations for safety-related
> > systems?
> > 
> > Please help me. What do I need to compile, what test do I need to run,
> > which verification tool do I need to employ for this change?
> 
> The change looks valid, 'reclaim_order' has not been used anywhere after
> READ_ONCE(), and its  So, it looks like a harmless change, you will only
> need a good commit message detailing why its harmless.
> 

Thanks, Sudip. Yes, I also conclude it is harmless but I really cannot say 
as I did not even compile it :) and I guess you did not either :)

I would actually want to argue that I compiled it for all available (and 
relevant) kernel configurations and the binary is identical before and 
after the change.

It is a Dead Store, so I expect the compiler to detect that and just 
optimize that away...

> So, from a safety pov, is it a requirement that every submitted patch
> will need to be tested based on the safety tests and all the other
> defined tests?
>

Well, I do not know what Roberto thinks his reference process is good for, 
but I would like to know if Roberto thinks it can guide anyone on such a 
question or not?

It is really just some fun for the discussion in this group... there are 
thousands of commits travelling into the kernel... if we cannot provide 
an answer for a single one, how to do it for thousands?


Lukas

Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm: vmscan: provide a change to the development-process group

[Lukas Bulwahn]

On Thu, 17 Sep 2020, Paoloni, Gabriele wrote:

> > -----Original Message-----
> > From: development-process@lists.elisa.tech <development-
> > process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> > Sent: Thursday, September 17, 2020 10:44 AM
> > To: linux-safety@lists.elisa.tech
> > Cc: development-process@lists.elisa.tech; Lukas Bulwahn
> > <lukas.bulwahn@gmail.com>
> > Subject: [ELISA Development Process WG] [PATCH] mm: vmscan: provide a
> > change to the development-process group
> > 
> > I think this change is needed for safety, whatever that might mean to you.
> > 
> > I am unqualified to really make a change here, as I have no clue what this
> > code does, nor what my change does, but sure, the testing and verification
> > reference process can now point out the required next steps in the
> > reference process to test this code and code change.
> > 
> > Good luck :)
> > 
> > Not intended for distribution to the general kernel mailing lists.
> > 
> > Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> > ---
> > I would like to submit such a patch, what do I need to do according to
> > the expected testing and verification recommendations for safety-related
> > systems?
> 
> Probably the problem is not just limited to submitting patches, but I think it
> is a good starting point.
> 
> > 
> > Please help me. What do I need to compile, what test do I need to run,
> > which verification tool do I need to employ for this change?
> 
> Right. I have tried to reformulate the problem as "ok I am looking at or I am
> trying to change one or more code lines, so I would like to understand if such
> code lines are tested today and how".
> Right now if we had a s maintained structural coverage report we could tell
> if the impacted code lines are covered and by which tests...I could not find it
> so I guess that so far we are missing it....am I wrong?
> 
> As next step I went to look at the function wrapping the impacted code line
> that in this case is kswapd. I could not find it tested in the Kunit framework.
> 
> As next steps then I would run the available tests in
> https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests
> in conjunction with GCOV trying to figure out if that line is covered already
> somehow.
> 
> There are probably smarter, better, faster ways to elaborate on this...so
> here I just put down what I would do in order to figure this out...
>

Gab, these are GREAT ideas and approaches. So, do we intend to provide 
methods, tools and documentation to really do these things you suggest?

That makes the intensions more tangible to me; if we try to find out 
how we and anyone else can get that information in some 'easy' way.

You can imagine that for a single-line change, you are suggesting probably 
quite some few days of work of investigation to get this information, 
right?


Lukas

Re: [ELISA Development Process WG] [linux-safety] [PATCH] mm: vmscan: provide a change to the development-process group

[Sudip Mukherjee]

On 17/09/2020 16:29, Lukas Bulwahn wrote:
> 
> 
> On Thu, 17 Sep 2020, Sudip Mukherjee wrote:
> 
>>
>>
>> On 17/09/2020 09:44, Lukas Bulwahn wrote:
>>> I think this change is needed for safety, whatever that might mean to you.
>>>
>>> I am unqualified to really make a change here, as I have no clue what this
>>> code does, nor what my change does, but sure, the testing and verification
>>> reference process can now point out the required next steps in the
>>> reference process to test this code and code change.
>>>
>>> Good luck :)
>>>
>>> Not intended for distribution to the general kernel mailing lists.
>>>
>>> Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
>>> ---
>>> I would like to submit such a patch, what do I need to do according to
>>> the expected testing and verification recommendations for safety-related
>>> systems?
>>>
>>> Please help me. What do I need to compile, what test do I need to run,
>>> which verification tool do I need to employ for this change?
>>
>> The change looks valid, 'reclaim_order' has not been used anywhere after
>> READ_ONCE(), and its  So, it looks like a harmless change, you will only
>> need a good commit message detailing why its harmless.
>>
> 
> Thanks, Sudip. Yes, I also conclude it is harmless but I really cannot say 
> as I did not even compile it :) and I guess you did not either :)
> 
> I would actually want to argue that I compiled it for all available (and 
> relevant) kernel configurations and the binary is identical before and 
> after the change.

That, I dont think is possible. The maintainers will receive thousands
of patch in a day. If they have to compile each for all available
configuration (and arch), then they might spend the full day just
checking patches. Also, many upstream contributors contribute in their
personal spare time, so if building in all possible configuration
becomes a requirement then I think that is going to discourage many
contributors from contributing.

> 
> It is a Dead Store, so I expect the compiler to detect that and just 
> optimize that away...

Which is also something I always think, we are relying on the compiler
to produce the code that is actually executed on the hardware. There are
different compilers and each compiler has different versions, so that
means the generated code is going to be different. Even though we say
Linus Kernel meets the safety requirement, can we say that the kernel
that is executing on the hardware meets the safety requirement? This, I
think is completely off-topic for this WG, but just a thought.

> 
>> So, from a safety pov, is it a requirement that every submitted patch
>> will need to be tested based on the safety tests and all the other
>> defined tests?
>>
> 
> Well, I do not know what Roberto thinks his reference process is good for, 
> but I would like to know if Roberto thinks it can guide anyone on such a 
> question or not?
> 
> It is really just some fun for the discussion in this group... there are 
> thousands of commits travelling into the kernel... if we cannot provide 
> an answer for a single one, how to do it for thousands?

Lets have another example of a change.

diff --git a/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
b/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
index b78101afc93d..2b539335206a 100644
--- a/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
+++ b/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
@@ -367,7 +367,6 @@ uint oid_rt_get_scan_in_progress_hdl(struct
oid_par_priv *poid_par_priv)
        return RNDIS_STATUS_SUCCESS;
 }

-
 uint oid_rt_forced_data_rate_hdl(struct oid_par_priv *poid_par_priv)
 {
        return RNDIS_STATUS_SUCCESS;



Not a formal patch, just pasted the git diff. This fixes the checkpatch
warning of "Please don't use multiple blank lines". What tests are
needed on this patch to say that the kernel development meets the safety
requirement?



-- 
Regards
Sudip

Re: [ELISA Development Process WG] [linux-safety] [PATCH] mm: vmscan: provide a change to the development-process group

[Lukas Bulwahn]

On Thu, 17 Sep 2020, Sudip Mukherjee wrote:

> 
> 
> On 17/09/2020 16:29, Lukas Bulwahn wrote:
> > 
> > 
> > On Thu, 17 Sep 2020, Sudip Mukherjee wrote:
> > 
> >>
> >>
> >> On 17/09/2020 09:44, Lukas Bulwahn wrote:
> >>> I think this change is needed for safety, whatever that might mean to you.
> >>>
> >>> I am unqualified to really make a change here, as I have no clue what this
> >>> code does, nor what my change does, but sure, the testing and verification
> >>> reference process can now point out the required next steps in the
> >>> reference process to test this code and code change.
> >>>
> >>> Good luck :)
> >>>
> >>> Not intended for distribution to the general kernel mailing lists.
> >>>
> >>> Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> >>> ---
> >>> I would like to submit such a patch, what do I need to do according to
> >>> the expected testing and verification recommendations for safety-related
> >>> systems?
> >>>
> >>> Please help me. What do I need to compile, what test do I need to run,
> >>> which verification tool do I need to employ for this change?
> >>
> >> The change looks valid, 'reclaim_order' has not been used anywhere after
> >> READ_ONCE(), and its  So, it looks like a harmless change, you will only
> >> need a good commit message detailing why its harmless.
> >>
> > 
> > Thanks, Sudip. Yes, I also conclude it is harmless but I really cannot say 
> > as I did not even compile it :) and I guess you did not either :)
> > 
> > I would actually want to argue that I compiled it for all available (and 
> > relevant) kernel configurations and the binary is identical before and 
> > after the change.
> 
> That, I dont think is possible. The maintainers will receive thousands
> of patch in a day. If they have to compile each for all available
> configuration (and arch), then they might spend the full day just
> checking patches. Also, many upstream contributors contribute in their
> personal spare time, so if building in all possible configuration
> becomes a requirement then I think that is going to discourage many
> contributors from contributing.
>

I am not asking the maintainer, I would only waste my own energy bill on 
that :) if I would what to do...

But even finding out which configurations actually make a difference is an 
unsolved challenge, right? The experts know, but how would I find out?

> > 
> > It is a Dead Store, so I expect the compiler to detect that and just 
> > optimize that away...
> 
> Which is also something I always think, we are relying on the compiler
> to produce the code that is actually executed on the hardware. There are
> different compilers and each compiler has different versions, so that
> means the generated code is going to be different. Even though we say
> Linus Kernel meets the safety requirement, can we say that the kernel
> that is executing on the hardware meets the safety requirement? This, I
> think is completely off-topic for this WG, but just a thought.
>

Yes, let us keep it simple for now; but you are right. This whole story of 
'test and verification' fully independently quickly breaks... but let the
group figure that out.
  
> > 
> >> So, from a safety pov, is it a requirement that every submitted patch
> >> will need to be tested based on the safety tests and all the other
> >> defined tests?
> >>
> > 
> > Well, I do not know what Roberto thinks his reference process is good for, 
> > but I would like to know if Roberto thinks it can guide anyone on such a 
> > question or not?
> > 
> > It is really just some fun for the discussion in this group... there are 
> > thousands of commits travelling into the kernel... if we cannot provide 
> > an answer for a single one, how to do it for thousands?
> 
> Lets have another example of a change.
> 
> diff --git a/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
> b/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
> index b78101afc93d..2b539335206a 100644
> --- a/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
> +++ b/drivers/staging/rtl8712/rtl871x_ioctl_rtl.c
> @@ -367,7 +367,6 @@ uint oid_rt_get_scan_in_progress_hdl(struct
> oid_par_priv *poid_par_priv)
>         return RNDIS_STATUS_SUCCESS;
>  }
> 
> -
>  uint oid_rt_forced_data_rate_hdl(struct oid_par_priv *poid_par_priv)
>  {
>         return RNDIS_STATUS_SUCCESS;
> 
> 
> 
> Not a formal patch, just pasted the git diff. This fixes the checkpatch
> warning of "Please don't use multiple blank lines". What tests are
> needed on this patch to say that the kernel development meets the safety
> requirement?
> 
>

Nice :) Sudip, You are making the kernel safer, Yeah! ;)

Reference process, where are thou?

Lukas

Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm: vmscan: provide a change to the development-process group

[Paoloni, Gabriele]

> -----Original Message-----
> From: development-process@lists.elisa.tech <development-
> process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> Sent: Thursday, September 17, 2020 5:44 PM
> To: Paoloni, Gabriele <gabriele.paoloni@intel.com>
> Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com>; linux-
> safety@lists.elisa.tech; development-process@lists.elisa.tech
> Subject: Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm:
> vmscan: provide a change to the development-process group
> 
> 
> 
> On Thu, 17 Sep 2020, Paoloni, Gabriele wrote:
> 
> > > -----Original Message-----
> > > From: development-process@lists.elisa.tech <development-
> > > process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> > > Sent: Thursday, September 17, 2020 10:44 AM
> > > To: linux-safety@lists.elisa.tech
> > > Cc: development-process@lists.elisa.tech; Lukas Bulwahn
> > > <lukas.bulwahn@gmail.com>
> > > Subject: [ELISA Development Process WG] [PATCH] mm: vmscan: provide
> a
> > > change to the development-process group
> > >
> > > I think this change is needed for safety, whatever that might mean to
> you.
> > >
> > > I am unqualified to really make a change here, as I have no clue what this
> > > code does, nor what my change does, but sure, the testing and
> verification
> > > reference process can now point out the required next steps in the
> > > reference process to test this code and code change.
> > >
> > > Good luck :)
> > >
> > > Not intended for distribution to the general kernel mailing lists.
> > >
> > > Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> > > ---
> > > I would like to submit such a patch, what do I need to do according to
> > > the expected testing and verification recommendations for safety-
> related
> > > systems?
> >
> > Probably the problem is not just limited to submitting patches, but I think it
> > is a good starting point.
> >
> > >
> > > Please help me. What do I need to compile, what test do I need to run,
> > > which verification tool do I need to employ for this change?
> >
> > Right. I have tried to reformulate the problem as "ok I am looking at or I am
> > trying to change one or more code lines, so I would like to understand if
> such
> > code lines are tested today and how".
> > Right now if we had a s maintained structural coverage report we could tell
> > if the impacted code lines are covered and by which tests...I could not find
> it
> > so I guess that so far we are missing it....am I wrong?
> >
> > As next step I went to look at the function wrapping the impacted code line
> > that in this case is kswapd. I could not find it tested in the Kunit framework.
> >
> > As next steps then I would run the available tests in
> > https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests
> > in conjunction with GCOV trying to figure out if that line is covered already
> > somehow.
> >
> > There are probably smarter, better, faster ways to elaborate on this...so
> > here I just put down what I would do in order to figure this out...
> >
> 
> Gab, these are GREAT ideas and approaches. So, do we intend to provide
> methods, tools and documentation to really do these things you suggest?
> 
> That makes the intensions more tangible to me; if we try to find out
> how we and anyone else can get that information in some 'easy' way.
> 
> You can imagine that for a single-line change, you are suggesting probably
> quite some few days of work of investigation to get this information,
> right?

Unless together with the test frameworks in Linux we start maintaining a
corresponding structural coverage report that would instantly map the
affected code lines against the corresponding test.

Then reading the corresponding test we can figure out if the tests are still
valid (i.e. - black box - have I changed any behavior with my patch compared
to the expected result of the tests? or - white box - am I able to reach my
new change using the current test suites?)

My feeling is that we just need to get started putting together a more
structured test framework and report and them the problem as presented
by this patch would be very straightforward

Thanks
Gab

> 
> 
> Lukas
> 
> 
> 
> 

---------------------------------------------------------------------
INTEL CORPORATION ITALIA S.p.A. con unico socio
Sede: Milanofiori Palazzo E 4 
CAP 20094 Assago (MI)
Capitale Sociale Euro 104.000,00 interamente versato
Partita I.V.A. e Codice Fiscale  04236760155
Repertorio Economico Amministrativo n. 997124 
Registro delle Imprese di Milano nr. 183983/5281/33
Soggetta ad attivita' di direzione e coordinamento di 
INTEL CORPORATION, USA

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm: vmscan: provide a change to the development-process group

[Lukas Bulwahn]

On Thu, 17 Sep 2020, Paoloni, Gabriele wrote:

> 
> 
> > -----Original Message-----
> > From: development-process@lists.elisa.tech <development-
> > process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> > Sent: Thursday, September 17, 2020 5:44 PM
> > To: Paoloni, Gabriele <gabriele.paoloni@intel.com>
> > Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com>; linux-
> > safety@lists.elisa.tech; development-process@lists.elisa.tech
> > Subject: Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm:
> > vmscan: provide a change to the development-process group
> > 
> > 
> > 
> > On Thu, 17 Sep 2020, Paoloni, Gabriele wrote:
> > 
> > > > -----Original Message-----
> > > > From: development-process@lists.elisa.tech <development-
> > > > process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> > > > Sent: Thursday, September 17, 2020 10:44 AM
> > > > To: linux-safety@lists.elisa.tech
> > > > Cc: development-process@lists.elisa.tech; Lukas Bulwahn
> > > > <lukas.bulwahn@gmail.com>
> > > > Subject: [ELISA Development Process WG] [PATCH] mm: vmscan: provide
> > a
> > > > change to the development-process group
> > > >
> > > > I think this change is needed for safety, whatever that might mean to
> > you.
> > > >
> > > > I am unqualified to really make a change here, as I have no clue what this
> > > > code does, nor what my change does, but sure, the testing and
> > verification
> > > > reference process can now point out the required next steps in the
> > > > reference process to test this code and code change.
> > > >
> > > > Good luck :)
> > > >
> > > > Not intended for distribution to the general kernel mailing lists.
> > > >
> > > > Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> > > > ---
> > > > I would like to submit such a patch, what do I need to do according to
> > > > the expected testing and verification recommendations for safety-
> > related
> > > > systems?
> > >
> > > Probably the problem is not just limited to submitting patches, but I think it
> > > is a good starting point.
> > >
> > > >
> > > > Please help me. What do I need to compile, what test do I need to run,
> > > > which verification tool do I need to employ for this change?
> > >
> > > Right. I have tried to reformulate the problem as "ok I am looking at or I am
> > > trying to change one or more code lines, so I would like to understand if
> > such
> > > code lines are tested today and how".
> > > Right now if we had a s maintained structural coverage report we could tell
> > > if the impacted code lines are covered and by which tests...I could not find
> > it
> > > so I guess that so far we are missing it....am I wrong?
> > >
> > > As next step I went to look at the function wrapping the impacted code line
> > > that in this case is kswapd. I could not find it tested in the Kunit framework.
> > >
> > > As next steps then I would run the available tests in
> > > https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests
> > > in conjunction with GCOV trying to figure out if that line is covered already
> > > somehow.
> > >
> > > There are probably smarter, better, faster ways to elaborate on this...so
> > > here I just put down what I would do in order to figure this out...
> > >
> > 
> > Gab, these are GREAT ideas and approaches. So, do we intend to provide
> > methods, tools and documentation to really do these things you suggest?
> > 
> > That makes the intensions more tangible to me; if we try to find out
> > how we and anyone else can get that information in some 'easy' way.
> > 
> > You can imagine that for a single-line change, you are suggesting probably
> > quite some few days of work of investigation to get this information,
> > right?
> 
> Unless together with the test frameworks in Linux we start maintaining a
> corresponding structural coverage report that would instantly map the
> affected code lines against the corresponding test.
> 
> Then reading the corresponding test we can figure out if the tests are still
> valid (i.e. - black box - have I changed any behavior with my patch compared
> to the expected result of the tests? or - white box - am I able to reach my
> new change using the current test suites?)
> 
> My feeling is that we just need to get started putting together a more
> structured test framework and report and them the problem as presented
> by this patch would be very straightforward
> 

Show me and I will believe you, Gab :)

If we can pull this off, we could actually impress some people in the 
kernel community.

This sounds like a plan that could serve as reference for the next 
patch to come...

Lukas