* QA: Monitor Linux log messages as port of release (candidate) testing
@ 2021-09-07 8:40 Paul Menzel
2021-09-07 12:53 ` Guenter Roeck
0 siblings, 1 reply; 6+ messages in thread
From: Paul Menzel @ 2021-09-07 8:40 UTC (permalink / raw)
To: Guenter Roeck; +Cc: LKML
Dear Guenter,
Thank you for testing release candidates and releases [1]. Is your test
setup documented somewhere?
If not happening already, could the Linux messages (at least up to log
level warning) also be monitored? For example, in Linux 5.14, a new
warning snuck in by cefc7ca462 (ACPI: PRM: implement OperationRegion
handler for the PlatformRtMechanism subtype) [2], which could have been
caught early on, and fixed before the release.
The test summaries would then also notify about possible behavior change.
Kind regards,
Paul
[1]: https://lkml.org/lkml/2021/7/11/326
[2]:
https://lore.kernel.org/linux-acpi/54b6f8cb-4714-587c-b2d0-98134473293d@linux.intel.com/T/#m3f54733714381765c8bb5bfa5b2aa3969407931e
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: QA: Monitor Linux log messages as port of release (candidate) testing
2021-09-07 8:40 QA: Monitor Linux log messages as port of release (candidate) testing Paul Menzel
@ 2021-09-07 12:53 ` Guenter Roeck
2021-09-07 13:50 ` Paul Menzel
0 siblings, 1 reply; 6+ messages in thread
From: Guenter Roeck @ 2021-09-07 12:53 UTC (permalink / raw)
To: Paul Menzel; +Cc: LKML
On Tue, Sep 07, 2021 at 10:40:31AM +0200, Paul Menzel wrote:
> Dear Guenter,
>
>
> Thank you for testing release candidates and releases [1]. Is your test
> setup documented somewhere?
>
Not really, except its source is available at github:
https://github.com/groeck/linux-build-test
> If not happening already, could the Linux messages (at least up to log level
> warning) also be monitored? For example, in Linux 5.14, a new warning snuck
> in by cefc7ca462 (ACPI: PRM: implement OperationRegion handler for the
> PlatformRtMechanism subtype) [2], which could have been caught early on, and
> fixed before the release.
>
> The test summaries would then also notify about possible behavior change.
>
Logs are available and can be examined at kerneltests.org/builders.
Reports are generated manually, so it would be way too much effort to add
build warnings to those. Besides, logs are way too noisy to be useful in a
summary e-mail.
Also, Geert's build reports already provide build warnings and errors.
The same applies to reports sent by 0-day. Indeed, I do see at least
one 0-day report against commit cefc7ca46235. What would be the point
of adding yet another report of build warnings on top of that ?
Thanks,
Guenter
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: QA: Monitor Linux log messages as port of release (candidate) testing
2021-09-07 12:53 ` Guenter Roeck
@ 2021-09-07 13:50 ` Paul Menzel
2021-09-07 14:47 ` Guenter Roeck
0 siblings, 1 reply; 6+ messages in thread
From: Paul Menzel @ 2021-09-07 13:50 UTC (permalink / raw)
To: Guenter Roeck; +Cc: LKML
Dear Guenter,
Am 07.09.21 um 14:53 schrieb Guenter Roeck:
> On Tue, Sep 07, 2021 at 10:40:31AM +0200, Paul Menzel wrote:
>> Thank you for testing release candidates and releases [1]. Is your test
>> setup documented somewhere?
>>
> Not really, except its source is available at github:
> https://github.com/groeck/linux-build-test
Thank you.
>> If not happening already, could the Linux messages (at least up to log level
>> warning) also be monitored? For example, in Linux 5.14, a new warning snuck
>> in by cefc7ca462 (ACPI: PRM: implement OperationRegion handler for the
>> PlatformRtMechanism subtype) [2], which could have been caught early on, and
>> fixed before the release.
>>
>> The test summaries would then also notify about possible behavior change.
>
> Logs are available and can be examined at kerneltests.org/builders.
Sorry for being blind. Under *qemu-tests*, looking at build #1831 [1],
clicking on *stdio* [2] under *Steps and Logfiles*, I do not see any
Linux logs.
> Reports are generated manually, so it would be way too much effort to add
> build warnings to those. Besides, logs are way too noisy to be useful in a
> summary e-mail.
Just to avoid misunderstandings, it’s about the Linux run-time logs.
> Also, Geert's build reports already provide build warnings and errors.
> The same applies to reports sent by 0-day. Indeed, I do see at least
> one 0-day report against commit cefc7ca46235.
How can I find that report?
> What would be the point of adding yet another report of build
> warnings on top of that ?
If the functionality already exists, great. But to be clear, it’s about
the runtime logs.
Kind regards,
Paul
[1]: https://kerneltests.org/builders/hwmon-x86_64-master/builds/1831
[2]:
https://kerneltests.org/builders/qemu-x86_64-master/builds/1831/steps/qemubuildcommand/logs/stdio
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: QA: Monitor Linux log messages as port of release (candidate) testing
2021-09-07 13:50 ` Paul Menzel
@ 2021-09-07 14:47 ` Guenter Roeck
2021-09-07 15:18 ` Paul Menzel
0 siblings, 1 reply; 6+ messages in thread
From: Guenter Roeck @ 2021-09-07 14:47 UTC (permalink / raw)
To: Paul Menzel; +Cc: LKML
Hi Paul,
On Tue, Sep 07, 2021 at 03:50:39PM +0200, Paul Menzel wrote:
> Dear Guenter,
>
>
> Am 07.09.21 um 14:53 schrieb Guenter Roeck:
> > On Tue, Sep 07, 2021 at 10:40:31AM +0200, Paul Menzel wrote:
>
> > > Thank you for testing release candidates and releases [1]. Is your test
> > > setup documented somewhere?
> > >
> > Not really, except its source is available at github:
> > https://github.com/groeck/linux-build-test
>
> Thank you.
>
> > > If not happening already, could the Linux messages (at least up to log level
> > > warning) also be monitored? For example, in Linux 5.14, a new warning snuck
> > > in by cefc7ca462 (ACPI: PRM: implement OperationRegion handler for the
> > > PlatformRtMechanism subtype) [2], which could have been caught early on, and
> > > fixed before the release.
> > >
> > > The test summaries would then also notify about possible behavior change.
> >
> > Logs are available and can be examined at kerneltests.org/builders.
>
> Sorry for being blind. Under *qemu-tests*, looking at build #1831 [1],
> clicking on *stdio* [2] under *Steps and Logfiles*, I do not see any Linux
> logs.
>
> > Reports are generated manually, so it would be way too much effort to add
> > build warnings to those. Besides, logs are way too noisy to be useful in a
> > summary e-mail.
>
> Just to avoid misunderstandings, it’s about the Linux run-time logs.
>
Run-time logs are only provided if there are errors or runtime issues
(crashes, warning tracebacks, or test failures).
> > Also, Geert's build reports already provide build warnings and errors.
> > The same applies to reports sent by 0-day. Indeed, I do see at least
> > one 0-day report against commit cefc7ca46235.
>
> How can I find that report?
>
I just searched for the SHA.
https://www.spinics.net/lists/linux-acpi/msg101721.html
> > What would be the point of adding yet another report of build
> > warnings on top of that ?
> If the functionality already exists, great. But to be clear, it’s about the
> runtime logs.
>
If there are (new) runtime issues (crashes, warnings, or other test failures),
I usually analyze, bisect, and report the problem against the patch introducing
the problem unless it was already reported elsewhere.
For example, there is currently a backtrace in arm64 tests:
BUG: sleeping function called from invalid context at kernel/locking/semaphore.c:163
which is due to ACPI code being called from the wrong context.
Another example is the cirular locking problem reported in various
mips tests, from mtd code. Both problems have been fixed in -next,
and the fixes will hopefully be pushed upstream soon.
The qemu tests do not log build warnings. I used to do that, but it
was way too noisy (some builds used to produce hundreds of build
warnings). Also, there is no logging data if there are neither
crashes nor warnings or other test failures, for the same reason.
If you are looking for complete boot logs, I'd suggest looking at
test results from kernelci.org.
Thanks,
Guenter
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: QA: Monitor Linux log messages as port of release (candidate) testing
2021-09-07 14:47 ` Guenter Roeck
@ 2021-09-07 15:18 ` Paul Menzel
2021-09-07 15:28 ` Guenter Roeck
0 siblings, 1 reply; 6+ messages in thread
From: Paul Menzel @ 2021-09-07 15:18 UTC (permalink / raw)
To: Guenter Roeck; +Cc: LKML
Dear Guenter,
Am 07.09.21 um 16:47 schrieb Guenter Roeck:
> On Tue, Sep 07, 2021 at 03:50:39PM +0200, Paul Menzel wrote:
>> Am 07.09.21 um 14:53 schrieb Guenter Roeck:
>>> On Tue, Sep 07, 2021 at 10:40:31AM +0200, Paul Menzel wrote:
>>
>>>> Thank you for testing release candidates and releases [1]. Is your test
>>>> setup documented somewhere?
>>>>
>>> Not really, except its source is available at github:
>>> https://github.com/groeck/linux-build-test
>>
>> Thank you.
>>
>>>> If not happening already, could the Linux messages (at least up to log level
>>>> warning) also be monitored? For example, in Linux 5.14, a new warning snuck
>>>> in by cefc7ca462 (ACPI: PRM: implement OperationRegion handler for the
>>>> PlatformRtMechanism subtype) [2], which could have been caught early on, and
>>>> fixed before the release.
>>>>
>>>> The test summaries would then also notify about possible behavior change.
>>>
>>> Logs are available and can be examined at kerneltests.org/builders.
>>
>> Sorry for being blind. Under *qemu-tests*, looking at build #1831 [1],
>> clicking on *stdio* [2] under *Steps and Logfiles*, I do not see any Linux
>> logs.
>>
>>> Reports are generated manually, so it would be way too much effort to add
>>> build warnings to those. Besides, logs are way too noisy to be useful in a
>>> summary e-mail.
>>
>> Just to avoid misunderstandings, it’s about the Linux run-time logs.
>
> Run-time logs are only provided if there are errors or runtime issues
> (crashes, warning tracebacks, or test failures).
Could this be changed to always publish them? Or is that too demanding
on storage?
>>> Also, Geert's build reports already provide build warnings and errors.
>>> The same applies to reports sent by 0-day. Indeed, I do see at least
>>> one 0-day report against commit cefc7ca46235.
>>
>> How can I find that report?
>>
> I just searched for the SHA.
>
> https://www.spinics.net/lists/linux-acpi/msg101721.html
Alright, that is a build time thing. I am looking for runtime things.
>>> What would be the point of adding yet another report of build
>>> warnings on top of that ?
>> If the functionality already exists, great. But to be clear, it’s about the
>> runtime logs.
>
> If there are (new) runtime issues (crashes, warnings, or other test failures),
> I usually analyze, bisect, and report the problem against the patch introducing
> the problem unless it was already reported elsewhere.
>
> For example, there is currently a backtrace in arm64 tests:
>
> BUG: sleeping function called from invalid context at kernel/locking/semaphore.c:163
>
> which is due to ACPI code being called from the wrong context.
> Another example is the cirular locking problem reported in various
> mips tests, from mtd code. Both problems have been fixed in -next,
> and the fixes will hopefully be pushed upstream soon.
>
> The qemu tests do not log build warnings. I used to do that, but it
> was way too noisy (some builds used to produce hundreds of build
> warnings). Also, there is no logging data if there are neither
> crashes nor warnings or other test failures, for the same reason.
Thank you for the clarification and your QA work. Sorry about the
misunderstanding about compile/build logs and runtime logs.
> If you are looking for complete boot logs, I'd suggest looking at
> test results from kernelci.org.
Thank you. I am going to do that then.
Kind regards,
Paul
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: QA: Monitor Linux log messages as port of release (candidate) testing
2021-09-07 15:18 ` Paul Menzel
@ 2021-09-07 15:28 ` Guenter Roeck
0 siblings, 0 replies; 6+ messages in thread
From: Guenter Roeck @ 2021-09-07 15:28 UTC (permalink / raw)
To: Paul Menzel; +Cc: LKML
Hi Paul,
On 9/7/21 8:18 AM, Paul Menzel wrote:
> Dear Guenter,
>
>
> Am 07.09.21 um 16:47 schrieb Guenter Roeck:
>
>> On Tue, Sep 07, 2021 at 03:50:39PM +0200, Paul Menzel wrote:
>
>>> Am 07.09.21 um 14:53 schrieb Guenter Roeck:
>>>> On Tue, Sep 07, 2021 at 10:40:31AM +0200, Paul Menzel wrote:
>>>
>>>>> Thank you for testing release candidates and releases [1]. Is your test
>>>>> setup documented somewhere?
>>>>>
>>>> Not really, except its source is available at github:
>>>> https://github.com/groeck/linux-build-test
>>>
>>> Thank you.
>>>
>>>>> If not happening already, could the Linux messages (at least up to log level
>>>>> warning) also be monitored? For example, in Linux 5.14, a new warning snuck
>>>>> in by cefc7ca462 (ACPI: PRM: implement OperationRegion handler for the
>>>>> PlatformRtMechanism subtype) [2], which could have been caught early on, and
>>>>> fixed before the release.
>>>>>
>>>>> The test summaries would then also notify about possible behavior change.
>>>>
>>>> Logs are available and can be examined at kerneltests.org/builders.
>>>
>>> Sorry for being blind. Under *qemu-tests*, looking at build #1831 [1],
>>> clicking on *stdio* [2] under *Steps and Logfiles*, I do not see any Linux
>>> logs.
>>>
>>>> Reports are generated manually, so it would be way too much effort to add
>>>> build warnings to those. Besides, logs are way too noisy to be useful in a
>>>> summary e-mail.
>>>
>>> Just to avoid misunderstandings, it’s about the Linux run-time logs.
>>
>> Run-time logs are only provided if there are errors or runtime issues
>> (crashes, warning tracebacks, or test failures).
>
> Could this be changed to always publish them? Or is that too demanding on storage?
>
It isn't a matter of storage, but of noise. It is mostly me looking at the data,
and I really don't want to see logs if nothing interesting is there. With 450+
builds per release I'd drown in noise otherwise. Sure, a large part of that
is a UI issue, but that is where systems like kernelci come in. If I ever
find the time to do it, I might report build and runtime results/logs to kernelci.
But that is a big IF.
>>>> Also, Geert's build reports already provide build warnings and errors.
>>>> The same applies to reports sent by 0-day. Indeed, I do see at least
>>>> one 0-day report against commit cefc7ca46235.
>>>
>>> How can I find that report?
>>>
>> I just searched for the SHA.
>>
>> https://www.spinics.net/lists/linux-acpi/msg101721.html
>
> Alright, that is a build time thing. I am looking for runtime things.
>
If there is nothing in my reports, you'd probably not find what you
are looking for (it would be reported if it results in a backtrace).
Guenter
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-09-07 15:28 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-07 8:40 QA: Monitor Linux log messages as port of release (candidate) testing Paul Menzel
2021-09-07 12:53 ` Guenter Roeck
2021-09-07 13:50 ` Paul Menzel
2021-09-07 14:47 ` Guenter Roeck
2021-09-07 15:18 ` Paul Menzel
2021-09-07 15:28 ` Guenter Roeck
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.