From: "Lukas Bulwahn" <lukas.bulwahn@gmail.com>
To: "Paoloni, Gabriele" <gabriele.paoloni@intel.com>
Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com>,
"linux-safety@lists.elisa.tech" <linux-safety@lists.elisa.tech>,
"development-process@lists.elisa.tech"
<development-process@lists.elisa.tech>
Subject: Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm: vmscan: provide a change to the development-process group
Date: Thu, 17 Sep 2020 18:28:49 +0200 (CEST) [thread overview]
Message-ID: <alpine.DEB.2.21.2009171825170.9985@felia> (raw)
In-Reply-To: <BY5PR11MB41954AB0A7968933F7DA6314883E0@BY5PR11MB4195.namprd11.prod.outlook.com>
On Thu, 17 Sep 2020, Paoloni, Gabriele wrote:
>
>
> > -----Original Message-----
> > From: development-process@lists.elisa.tech <development-
> > process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> > Sent: Thursday, September 17, 2020 5:44 PM
> > To: Paoloni, Gabriele <gabriele.paoloni@intel.com>
> > Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com>; linux-
> > safety@lists.elisa.tech; development-process@lists.elisa.tech
> > Subject: Re: [linux-safety] [ELISA Development Process WG] [PATCH] mm:
> > vmscan: provide a change to the development-process group
> >
> >
> >
> > On Thu, 17 Sep 2020, Paoloni, Gabriele wrote:
> >
> > > > -----Original Message-----
> > > > From: development-process@lists.elisa.tech <development-
> > > > process@lists.elisa.tech> On Behalf Of Lukas Bulwahn
> > > > Sent: Thursday, September 17, 2020 10:44 AM
> > > > To: linux-safety@lists.elisa.tech
> > > > Cc: development-process@lists.elisa.tech; Lukas Bulwahn
> > > > <lukas.bulwahn@gmail.com>
> > > > Subject: [ELISA Development Process WG] [PATCH] mm: vmscan: provide
> > a
> > > > change to the development-process group
> > > >
> > > > I think this change is needed for safety, whatever that might mean to
> > you.
> > > >
> > > > I am unqualified to really make a change here, as I have no clue what this
> > > > code does, nor what my change does, but sure, the testing and
> > verification
> > > > reference process can now point out the required next steps in the
> > > > reference process to test this code and code change.
> > > >
> > > > Good luck :)
> > > >
> > > > Not intended for distribution to the general kernel mailing lists.
> > > >
> > > > Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com>
> > > > ---
> > > > I would like to submit such a patch, what do I need to do according to
> > > > the expected testing and verification recommendations for safety-
> > related
> > > > systems?
> > >
> > > Probably the problem is not just limited to submitting patches, but I think it
> > > is a good starting point.
> > >
> > > >
> > > > Please help me. What do I need to compile, what test do I need to run,
> > > > which verification tool do I need to employ for this change?
> > >
> > > Right. I have tried to reformulate the problem as "ok I am looking at or I am
> > > trying to change one or more code lines, so I would like to understand if
> > such
> > > code lines are tested today and how".
> > > Right now if we had a s maintained structural coverage report we could tell
> > > if the impacted code lines are covered and by which tests...I could not find
> > it
> > > so I guess that so far we are missing it....am I wrong?
> > >
> > > As next step I went to look at the function wrapping the impacted code line
> > > that in this case is kswapd. I could not find it tested in the Kunit framework.
> > >
> > > As next steps then I would run the available tests in
> > > https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests
> > > in conjunction with GCOV trying to figure out if that line is covered already
> > > somehow.
> > >
> > > There are probably smarter, better, faster ways to elaborate on this...so
> > > here I just put down what I would do in order to figure this out...
> > >
> >
> > Gab, these are GREAT ideas and approaches. So, do we intend to provide
> > methods, tools and documentation to really do these things you suggest?
> >
> > That makes the intensions more tangible to me; if we try to find out
> > how we and anyone else can get that information in some 'easy' way.
> >
> > You can imagine that for a single-line change, you are suggesting probably
> > quite some few days of work of investigation to get this information,
> > right?
>
> Unless together with the test frameworks in Linux we start maintaining a
> corresponding structural coverage report that would instantly map the
> affected code lines against the corresponding test.
>
> Then reading the corresponding test we can figure out if the tests are still
> valid (i.e. - black box - have I changed any behavior with my patch compared
> to the expected result of the tests? or - white box - am I able to reach my
> new change using the current test suites?)
>
> My feeling is that we just need to get started putting together a more
> structured test framework and report and them the problem as presented
> by this patch would be very straightforward
>
Show me and I will believe you, Gab :)
If we can pull this off, we could actually impress some people in the
kernel community.
This sounds like a plan that could serve as reference for the next
patch to come...
Lukas
next prev parent reply other threads:[~2020-09-17 16:28 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-17 8:44 [PATCH] mm: vmscan: provide a change to the development-process group Lukas Bulwahn
2020-09-17 12:59 ` [ELISA Development Process WG] " Paoloni, Gabriele
2020-09-17 15:44 ` [linux-safety] " Lukas Bulwahn
2020-09-17 16:23 ` Paoloni, Gabriele
2020-09-17 16:28 ` Lukas Bulwahn [this message]
2020-09-17 15:14 ` [linux-safety] " Sudip Mukherjee
2020-09-17 15:29 ` Lukas Bulwahn
2020-09-17 15:47 ` [ELISA Development Process WG] " Sudip Mukherjee
2020-09-17 16:02 ` Lukas Bulwahn
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.21.2009171825170.9985@felia \
--to=lukas.bulwahn@gmail.com \
--cc=development-process@lists.elisa.tech \
--cc=gabriele.paoloni@intel.com \
--cc=linux-safety@lists.elisa.tech \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).