All of lore.kernel.org
 help / color / mirror / Atom feed
* Wiki for automatic reports / fixes
@ 2015-10-02 19:19 Luis R. Rodriguez
  2015-10-05 16:56 ` Ian Jackson
  0 siblings, 1 reply; 6+ messages in thread
From: Luis R. Rodriguez @ 2015-10-02 19:19 UTC (permalink / raw)
  To: Fengguang Wu, Valentin Rothberg, Dan Carpenter, Mel Gorman,
	Steven Rostedt
  Cc: Johannes Thumshirn, Konstantin Ryabitsev, Jonathan Corbet,
	Lars Kurth, Ian Jackson, Stefano Stabellini, Vitaly Kuznetsov,
	Robert Schweikert, ms, Juergen Gross, linux-kernel, Joerg Roedel,
	linux-doc

Fengguang, Dan, Mel,

Valentin has some open source tools that uses some SAT solver the
backend to hunt for dead, the undertaker tool [0] inspects Kconfig,
and uses SAT solver to do this work. A while ago it was determined
that this code could not be merged upstream for a variety of reasons,
but mostly due to the fact that it was R&D work and it was under GPLv3
[1]. I know a few of you have other testing tools which helps find
other sorts of issues. To my knowledge automated reports for these may
be expected readily these days by developers from 0-day bot, and for
smatch issues, but its unclear what we should expect from the other
ones. There is also some pending ideas over a few of these other
testing frameworks, for instance a recent discussion was started to
evaluate the potential for integration of Xen into the 0-day bot
testing scheme. While discussing expectations and information about
reports over these with Valentin it occurred to me information about
all these may be scattered separately and some developers may be
surprised when they first get reports / fixes from these sorts of
testing systems and that perhaps it may be useful if we had a single
wiki entry point where we could refer folks to the different ongoing
testing infrastructures out there working upstream.

If we could piggy back off of an already existing wiki then great, but
if not I was thinking something off of wiki.kernel.org might be good.
How about tests.wiki.kernel.org ? If such projects don't have a wiki
they could perhaps use pages off of tests.wiki.kernel.org to elaborate
and set expectations straight. Thoughts?

[0] http://vamos.informatik.uni-erlangen.de/trac/undertake
[1] https://lwn.net/Articles/617383/

 Luis

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Wiki for automatic reports / fixes
  2015-10-02 19:19 Wiki for automatic reports / fixes Luis R. Rodriguez
@ 2015-10-05 16:56 ` Ian Jackson
  2015-10-05 17:03   ` Luis R. Rodriguez
  0 siblings, 1 reply; 6+ messages in thread
From: Ian Jackson @ 2015-10-05 16:56 UTC (permalink / raw)
  To: Luis R. Rodriguez
  Cc: Fengguang Wu, Valentin Rothberg, Dan Carpenter, Mel Gorman,
	Steven Rostedt, Johannes Thumshirn, Konstantin Ryabitsev,
	Jonathan Corbet, Lars Kurth, Stefano Stabellini,
	Vitaly Kuznetsov, Robert Schweikert, ms, Juergen Gross,
	linux-kernel, Joerg Roedel, linux-doc

Luis R. Rodriguez writes ("Wiki for automatic reports / fixes"):
[...]
>  While discussing expectations and information about
> reports over these with Valentin it occurred to me information about
> all these may be scattered separately and some developers may be
> surprised when they first get reports / fixes from these sorts of
> testing systems and that perhaps it may be useful if we had a single
> wiki entry point where we could refer folks to the different ongoing
> testing infrastructures out there working upstream.
> 
> If we could piggy back off of an already existing wiki then great, but
> if not I was thinking something off of wiki.kernel.org might be good.
> How about tests.wiki.kernel.org ? If such projects don't have a wiki
> they could perhaps use pages off of tests.wiki.kernel.org to elaborate
> and set expectations straight. Thoughts?

To clarify what I think you are suggesting, is to create a new wiki or
wiki page which gives information about automatic tests that are
performed on upstream (or going-upstream) Linux branches ?

I think this is a good idea.  I'm not sure how much information we
need for each tester, but a page for each would be about right.

Ian.

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Wiki for automatic reports / fixes
  2015-10-05 16:56 ` Ian Jackson
@ 2015-10-05 17:03   ` Luis R. Rodriguez
  2015-10-12 19:55     ` Luis R. Rodriguez
  0 siblings, 1 reply; 6+ messages in thread
From: Luis R. Rodriguez @ 2015-10-05 17:03 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Fengguang Wu, Valentin Rothberg, Dan Carpenter, Mel Gorman,
	Steven Rostedt, Johannes Thumshirn, Konstantin Ryabitsev,
	Jonathan Corbet, Lars Kurth, Stefano Stabellini,
	Vitaly Kuznetsov, Robert Schweikert, ms, Juergen Gross,
	linux-kernel, Joerg Roedel, linux-doc

On Mon, Oct 5, 2015 at 9:56 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
> Luis R. Rodriguez writes ("Wiki for automatic reports / fixes"):
> [...]
>>  While discussing expectations and information about
>> reports over these with Valentin it occurred to me information about
>> all these may be scattered separately and some developers may be
>> surprised when they first get reports / fixes from these sorts of
>> testing systems and that perhaps it may be useful if we had a single
>> wiki entry point where we could refer folks to the different ongoing
>> testing infrastructures out there working upstream.
>>
>> If we could piggy back off of an already existing wiki then great, but
>> if not I was thinking something off of wiki.kernel.org might be good.
>> How about tests.wiki.kernel.org ? If such projects don't have a wiki
>> they could perhaps use pages off of tests.wiki.kernel.org to elaborate
>> and set expectations straight. Thoughts?
>
> To clarify what I think you are suggesting, is to create a new wiki or
> wiki page which gives information about automatic tests that are
> performed on upstream (or going-upstream) Linux branches ?

That's right, as it stands we have a slew of folks doing a series of
battery of tests on either linux-next or other branches, and
developers / maintainers get e-mails about this. Typically one becomes
aware of these tests through experience and in dealing with reports
but other times one may not even be aware of ongoing effort on this
front, such was the case of Valentin's dead code analysis with
undertaker. Knowing what existing work is being done can and could be
used can also save people from re-inventing the wheel, but also and
most importantly collaborate.

> I think this is a good idea.  I'm not sure how much information we
> need for each tester, but a page for each would be about right.

Sure, I figure if each tester framework has its own dedicated page we
can at least refer to it, but a basic page which describes general
coverage / mailing lists / contact info / and what to expect might be
useful. As it stands most of this is just tribal knowledge.

 Luis

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Wiki for automatic reports / fixes
  2015-10-05 17:03   ` Luis R. Rodriguez
@ 2015-10-12 19:55     ` Luis R. Rodriguez
  2015-10-21 16:13         ` [Cocci] " Luis R. Rodriguez
  0 siblings, 1 reply; 6+ messages in thread
From: Luis R. Rodriguez @ 2015-10-12 19:55 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Fengguang Wu, Valentin Rothberg, Dan Carpenter, Mel Gorman,
	Steven Rostedt, Johannes Thumshirn, Konstantin Ryabitsev,
	Jonathan Corbet, Lars Kurth, Stefano Stabellini,
	Vitaly Kuznetsov, Robert Schweikert, ms, Juergen Gross,
	linux-kernel, Joerg Roedel, linux-doc

On Mon, Oct 5, 2015 at 10:03 AM, Luis R. Rodriguez
<mcgrof@do-not-panic.com> wrote:
> On Mon, Oct 5, 2015 at 9:56 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
>> Luis R. Rodriguez writes ("Wiki for automatic reports / fixes"):
>> [...]
>>>  While discussing expectations and information about
>>> reports over these with Valentin it occurred to me information about
>>> all these may be scattered separately and some developers may be
>>> surprised when they first get reports / fixes from these sorts of
>>> testing systems and that perhaps it may be useful if we had a single
>>> wiki entry point where we could refer folks to the different ongoing
>>> testing infrastructures out there working upstream.
>>>
>>> If we could piggy back off of an already existing wiki then great, but
>>> if not I was thinking something off of wiki.kernel.org might be good.
>>> How about tests.wiki.kernel.org ? If such projects don't have a wiki
>>> they could perhaps use pages off of tests.wiki.kernel.org to elaborate
>>> and set expectations straight. Thoughts?
>>
>> To clarify what I think you are suggesting, is to create a new wiki or
>> wiki page which gives information about automatic tests that are
>> performed on upstream (or going-upstream) Linux branches ?
>
> That's right, as it stands we have a slew of folks doing a series of
> battery of tests on either linux-next or other branches, and
> developers / maintainers get e-mails about this. Typically one becomes
> aware of these tests through experience and in dealing with reports
> but other times one may not even be aware of ongoing effort on this
> front, such was the case of Valentin's dead code analysis with
> undertaker. Knowing what existing work is being done can and could be
> used can also save people from re-inventing the wheel, but also and
> most importantly collaborate.
>
>> I think this is a good idea.  I'm not sure how much information we
>> need for each tester, but a page for each would be about right.
>
> Sure, I figure if each tester framework has its own dedicated page we
> can at least refer to it, but a basic page which describes general
> coverage / mailing lists / contact info / and what to expect might be
> useful. As it stands most of this is just tribal knowledge.

OK I'll poke and see if we can get this created.

 Luis

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Wiki for automatic reports / fixes
  2015-10-12 19:55     ` Luis R. Rodriguez
@ 2015-10-21 16:13         ` Luis R. Rodriguez
  0 siblings, 0 replies; 6+ messages in thread
From: Luis R. Rodriguez @ 2015-10-21 16:13 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Fengguang Wu, Valentin Rothberg, Dan Carpenter, Mel Gorman,
	Steven Rostedt, Johannes Thumshirn, Konstantin Ryabitsev,
	Jonathan Corbet, Lars Kurth, Stefano Stabellini,
	Vitaly Kuznetsov, Robert Schweikert, ms, Juergen Gross,
	linux-kernel, Joerg Roedel, linux-doc, Shuah Khan, Julia Lawall,
	xen-devel, cocci

On Mon, Oct 12, 2015 at 12:55 PM, Luis R. Rodriguez
<mcgrof@do-not-panic.com> wrote:
> On Mon, Oct 5, 2015 at 10:03 AM, Luis R. Rodriguez
> <mcgrof@do-not-panic.com> wrote:
>> On Mon, Oct 5, 2015 at 9:56 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
>>> Luis R. Rodriguez writes ("Wiki for automatic reports / fixes"):
>>> [...]
>>>>  While discussing expectations and information about
>>>> reports over these with Valentin it occurred to me information about
>>>> all these may be scattered separately and some developers may be
>>>> surprised when they first get reports / fixes from these sorts of
>>>> testing systems and that perhaps it may be useful if we had a single
>>>> wiki entry point where we could refer folks to the different ongoing
>>>> testing infrastructures out there working upstream.
>>>>
>>>> If we could piggy back off of an already existing wiki then great, but
>>>> if not I was thinking something off of wiki.kernel.org might be good.
>>>> How about tests.wiki.kernel.org ? If such projects don't have a wiki
>>>> they could perhaps use pages off of tests.wiki.kernel.org to elaborate
>>>> and set expectations straight. Thoughts?
>>>
>>> To clarify what I think you are suggesting, is to create a new wiki or
>>> wiki page which gives information about automatic tests that are
>>> performed on upstream (or going-upstream) Linux branches ?
>>
>> That's right, as it stands we have a slew of folks doing a series of
>> battery of tests on either linux-next or other branches, and
>> developers / maintainers get e-mails about this. Typically one becomes
>> aware of these tests through experience and in dealing with reports
>> but other times one may not even be aware of ongoing effort on this
>> front, such was the case of Valentin's dead code analysis with
>> undertaker. Knowing what existing work is being done can and could be
>> used can also save people from re-inventing the wheel, but also and
>> most importantly collaborate.
>>
>>> I think this is a good idea.  I'm not sure how much information we
>>> need for each tester, but a page for each would be about right.
>>
>> Sure, I figure if each tester framework has its own dedicated page we
>> can at least refer to it, but a basic page which describes general
>> coverage / mailing lists / contact info / and what to expect might be
>> useful. As it stands most of this is just tribal knowledge.
>
> OK I'll poke and see if we can get this created.

OK we have a page up now to help track and document (if no
documentation exists) or refer to existing testing efforts for Linux:

https://bottest.wiki.kernel.org/

I went ahead and provided references for those projects that I was
aware of that had a page, otherwise I added some boiler plate
documentation page for them with some initial documentation I just
wrote. Please feel free to use this for your own documentation of your
test suite or refer to your own project if you have one from the top
level page.

 Luis

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [Cocci] Wiki for automatic reports / fixes
@ 2015-10-21 16:13         ` Luis R. Rodriguez
  0 siblings, 0 replies; 6+ messages in thread
From: Luis R. Rodriguez @ 2015-10-21 16:13 UTC (permalink / raw)
  To: cocci

On Mon, Oct 12, 2015 at 12:55 PM, Luis R. Rodriguez
<mcgrof@do-not-panic.com> wrote:
> On Mon, Oct 5, 2015 at 10:03 AM, Luis R. Rodriguez
> <mcgrof@do-not-panic.com> wrote:
>> On Mon, Oct 5, 2015 at 9:56 AM, Ian Jackson <Ian.Jackson@eu.citrix.com> wrote:
>>> Luis R. Rodriguez writes ("Wiki for automatic reports / fixes"):
>>> [...]
>>>>  While discussing expectations and information about
>>>> reports over these with Valentin it occurred to me information about
>>>> all these may be scattered separately and some developers may be
>>>> surprised when they first get reports / fixes from these sorts of
>>>> testing systems and that perhaps it may be useful if we had a single
>>>> wiki entry point where we could refer folks to the different ongoing
>>>> testing infrastructures out there working upstream.
>>>>
>>>> If we could piggy back off of an already existing wiki then great, but
>>>> if not I was thinking something off of wiki.kernel.org might be good.
>>>> How about tests.wiki.kernel.org ? If such projects don't have a wiki
>>>> they could perhaps use pages off of tests.wiki.kernel.org to elaborate
>>>> and set expectations straight. Thoughts?
>>>
>>> To clarify what I think you are suggesting, is to create a new wiki or
>>> wiki page which gives information about automatic tests that are
>>> performed on upstream (or going-upstream) Linux branches ?
>>
>> That's right, as it stands we have a slew of folks doing a series of
>> battery of tests on either linux-next or other branches, and
>> developers / maintainers get e-mails about this. Typically one becomes
>> aware of these tests through experience and in dealing with reports
>> but other times one may not even be aware of ongoing effort on this
>> front, such was the case of Valentin's dead code analysis with
>> undertaker. Knowing what existing work is being done can and could be
>> used can also save people from re-inventing the wheel, but also and
>> most importantly collaborate.
>>
>>> I think this is a good idea.  I'm not sure how much information we
>>> need for each tester, but a page for each would be about right.
>>
>> Sure, I figure if each tester framework has its own dedicated page we
>> can at least refer to it, but a basic page which describes general
>> coverage / mailing lists / contact info / and what to expect might be
>> useful. As it stands most of this is just tribal knowledge.
>
> OK I'll poke and see if we can get this created.

OK we have a page up now to help track and document (if no
documentation exists) or refer to existing testing efforts for Linux:

https://bottest.wiki.kernel.org/

I went ahead and provided references for those projects that I was
aware of that had a page, otherwise I added some boiler plate
documentation page for them with some initial documentation I just
wrote. Please feel free to use this for your own documentation of your
test suite or refer to your own project if you have one from the top
level page.

 Luis

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2015-10-21 16:14 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-10-02 19:19 Wiki for automatic reports / fixes Luis R. Rodriguez
2015-10-05 16:56 ` Ian Jackson
2015-10-05 17:03   ` Luis R. Rodriguez
2015-10-12 19:55     ` Luis R. Rodriguez
2015-10-21 16:13       ` Luis R. Rodriguez
2015-10-21 16:13         ` [Cocci] " Luis R. Rodriguez

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.