* [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
@ 2022-03-26 0:33 Daniel Latypov
2022-03-26 2:56 ` David Gow
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Daniel Latypov @ 2022-03-26 0:33 UTC (permalink / raw)
To: brendanhiggins, davidgow, keescook
Cc: linux-kernel, kunit-dev, linux-kselftest, skhan, maxime, Daniel Latypov
Recent changes have made it so the current set is not sufficient.
Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
Specifying a version of the debug info fixes this.
Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's
hopefully less fragile (esp. given we're tied to GCC 6 and lower).
Signed-off-by: Daniel Latypov <dlatypov@google.com>
---
Documentation/dev-tools/kunit/running_tips.rst | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst
index 7b6d26a25959..c36f6760087d 100644
--- a/Documentation/dev-tools/kunit/running_tips.rst
+++ b/Documentation/dev-tools/kunit/running_tips.rst
@@ -114,6 +114,7 @@ Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options:
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_INFO=y
+ CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
CONFIG_GCOV=y
@@ -122,7 +123,7 @@ Putting it together into a copy-pastable sequence of commands:
.. code-block:: bash
# Append coverage options to the current config
- $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
+ $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
$ ./tools/testing/kunit/kunit.py run
# Extract the coverage information from the build dir (.kunit/)
$ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/
base-commit: b14ffae378aa1db993e62b01392e70d1e585fb23
--
2.35.1.1021.g381101b075-goog
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
2022-03-26 0:33 [PATCH] Documentation: kunit: update kconfig options needed for UML coverage Daniel Latypov
@ 2022-03-26 2:56 ` David Gow
2022-03-28 16:35 ` Daniel Latypov
2022-03-28 7:58 ` Maxime Ripard
2022-03-28 16:27 ` Brendan Higgins
2 siblings, 1 reply; 8+ messages in thread
From: David Gow @ 2022-03-26 2:56 UTC (permalink / raw)
To: Daniel Latypov
Cc: Brendan Higgins, Kees Cook, Linux Kernel Mailing List,
KUnit Development, open list:KERNEL SELFTEST FRAMEWORK,
Shuah Khan, maxime
On Sat, Mar 26, 2022 at 8:34 AM Daniel Latypov <dlatypov@google.com> wrote:
>
> Recent changes have made it so the current set is not sufficient.
> Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
>
> Specifying a version of the debug info fixes this.
> Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's
> hopefully less fragile (esp. given we're tied to GCC 6 and lower).
>
> Signed-off-by: Daniel Latypov <dlatypov@google.com>
> ---
Looks good to me. One idea below, but it's a bit subjective and I'm
fine with this going in as-is if you prefer.
Reviewed-by: David Gow <davidgow@google.com>
Cheers,
-- Daivd
> Documentation/dev-tools/kunit/running_tips.rst | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst
> index 7b6d26a25959..c36f6760087d 100644
> --- a/Documentation/dev-tools/kunit/running_tips.rst
> +++ b/Documentation/dev-tools/kunit/running_tips.rst
> @@ -114,6 +114,7 @@ Instead of enabling ``CONFIG_GCOV_KERNEL=y``, we can set these options:
>
> CONFIG_DEBUG_KERNEL=y
> CONFIG_DEBUG_INFO=y
> + CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
> CONFIG_GCOV=y
>
>
> @@ -122,7 +123,7 @@ Putting it together into a copy-pastable sequence of commands:
> .. code-block:: bash
>
> # Append coverage options to the current config
> - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> $ ./tools/testing/kunit/kunit.py run
Would we want to instead use a chain of --kconfig_add arguments? (I
think there are advantages either way...)
> # Extract the coverage information from the build dir (.kunit/)
> $ lcov -t "my_kunit_tests" -o coverage.info -c -d .kunit/
>
> base-commit: b14ffae378aa1db993e62b01392e70d1e585fb23
> --
> 2.35.1.1021.g381101b075-goog
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
2022-03-26 0:33 [PATCH] Documentation: kunit: update kconfig options needed for UML coverage Daniel Latypov
2022-03-26 2:56 ` David Gow
@ 2022-03-28 7:58 ` Maxime Ripard
2022-03-28 16:27 ` Brendan Higgins
2 siblings, 0 replies; 8+ messages in thread
From: Maxime Ripard @ 2022-03-28 7:58 UTC (permalink / raw)
To: Daniel Latypov
Cc: brendanhiggins, davidgow, keescook, linux-kernel, kunit-dev,
linux-kselftest, skhan
[-- Attachment #1: Type: text/plain, Size: 532 bytes --]
On Fri, Mar 25, 2022 at 05:33:56PM -0700, Daniel Latypov wrote:
> Recent changes have made it so the current set is not sufficient.
> Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
>
> Specifying a version of the debug info fixes this.
> Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's
> hopefully less fragile (esp. given we're tied to GCC 6 and lower).
>
> Signed-off-by: Daniel Latypov <dlatypov@google.com>
Tested-by: Maxime Ripard <maxime@cerno.tech>
Thanks!
Maxime
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
2022-03-26 0:33 [PATCH] Documentation: kunit: update kconfig options needed for UML coverage Daniel Latypov
2022-03-26 2:56 ` David Gow
2022-03-28 7:58 ` Maxime Ripard
@ 2022-03-28 16:27 ` Brendan Higgins
2 siblings, 0 replies; 8+ messages in thread
From: Brendan Higgins @ 2022-03-28 16:27 UTC (permalink / raw)
To: Daniel Latypov
Cc: davidgow, keescook, linux-kernel, kunit-dev, linux-kselftest,
skhan, maxime
On Fri, Mar 25, 2022 at 8:34 PM Daniel Latypov <dlatypov@google.com> wrote:
>
> Recent changes have made it so the current set is not sufficient.
> Namely, CONFIG_DEBUG_INFO is not being set even when explicitly asked.
>
> Specifying a version of the debug info fixes this.
> Pick CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT as an option that's
> hopefully less fragile (esp. given we're tied to GCC 6 and lower).
>
> Signed-off-by: Daniel Latypov <dlatypov@google.com>
I agree with David's comment. Otherwise,
Reviewed-by: Brendan Higgins <brendanhiggins@google.com>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
2022-03-26 2:56 ` David Gow
@ 2022-03-28 16:35 ` Daniel Latypov
2022-03-28 16:54 ` Brendan Higgins
0 siblings, 1 reply; 8+ messages in thread
From: Daniel Latypov @ 2022-03-28 16:35 UTC (permalink / raw)
To: David Gow
Cc: Brendan Higgins, Kees Cook, Linux Kernel Mailing List,
KUnit Development, open list:KERNEL SELFTEST FRAMEWORK,
Shuah Khan, maxime
On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote:
>
<snip>
> > # Append coverage options to the current config
> > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > $ ./tools/testing/kunit/kunit.py run
>
> Would we want to instead use a chain of --kconfig_add arguments? (I
> think there are advantages either way...)
I've been considering this ever since the --kconfig_add patch was accepted.
It's more compatible w/ commands using --kunitconfig, but it also
looks very verbose.
E.g. it looks like
$ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
--kconfig_add=CONFIG_DEBUG_INFO=y
--kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
--kconfig_add=CONFIG_GCOV=y
Neither looks very appealing to me, so I've just kept it as-is for now.
Maybe there's something we can do to make this easier (e.g. allowing
--kunitconfig to be repeated and mergable)?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
2022-03-28 16:35 ` Daniel Latypov
@ 2022-03-28 16:54 ` Brendan Higgins
2022-03-28 18:58 ` Daniel Latypov
0 siblings, 1 reply; 8+ messages in thread
From: Brendan Higgins @ 2022-03-28 16:54 UTC (permalink / raw)
To: Daniel Latypov
Cc: David Gow, Kees Cook, Linux Kernel Mailing List,
KUnit Development, open list:KERNEL SELFTEST FRAMEWORK,
Shuah Khan, maxime
On Mon, Mar 28, 2022 at 12:35 PM Daniel Latypov <dlatypov@google.com> wrote:
>
> On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote:
> >
>
> <snip>
>
> > > # Append coverage options to the current config
> > > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > > $ ./tools/testing/kunit/kunit.py run
> >
> > Would we want to instead use a chain of --kconfig_add arguments? (I
> > think there are advantages either way...)
>
> I've been considering this ever since the --kconfig_add patch was accepted.
> It's more compatible w/ commands using --kunitconfig, but it also
> looks very verbose.
> E.g. it looks like
>
> $ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
> --kconfig_add=CONFIG_DEBUG_INFO=y
> --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
> --kconfig_add=CONFIG_GCOV=y
I don't think it's *that* much more verbose, but I see your point. I
personally prefer this, but not enough to argue about it.
> Neither looks very appealing to me, so I've just kept it as-is for now.
>
> Maybe there's something we can do to make this easier (e.g. allowing
> --kunitconfig to be repeated and mergable)?
I would like --kunitconfig to be repeadable and mergable.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
2022-03-28 16:54 ` Brendan Higgins
@ 2022-03-28 18:58 ` Daniel Latypov
2022-03-28 19:36 ` Brendan Higgins
0 siblings, 1 reply; 8+ messages in thread
From: Daniel Latypov @ 2022-03-28 18:58 UTC (permalink / raw)
To: Brendan Higgins
Cc: David Gow, Kees Cook, Linux Kernel Mailing List,
KUnit Development, open list:KERNEL SELFTEST FRAMEWORK,
Shuah Khan, maxime
On Mon, Mar 28, 2022 at 11:54 AM 'Brendan Higgins' via KUnit
Development <kunit-dev@googlegroups.com> wrote:
>
> On Mon, Mar 28, 2022 at 12:35 PM Daniel Latypov <dlatypov@google.com> wrote:
> >
> > On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote:
> > >
> >
> > <snip>
> >
> > > > # Append coverage options to the current config
> > > > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > > > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > > > $ ./tools/testing/kunit/kunit.py run
> > >
> > > Would we want to instead use a chain of --kconfig_add arguments? (I
> > > think there are advantages either way...)
> >
> > I've been considering this ever since the --kconfig_add patch was accepted.
> > It's more compatible w/ commands using --kunitconfig, but it also
> > looks very verbose.
> > E.g. it looks like
> >
> > $ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
> > --kconfig_add=CONFIG_DEBUG_INFO=y
> > --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
> > --kconfig_add=CONFIG_GCOV=y
>
> I don't think it's *that* much more verbose, but I see your point. I
> personally prefer this, but not enough to argue about it.
I personally prefer it too, but I'm biased as the person who added
--kconfig_add.
They're both ugly enough I'd figured I'd save the bikeshedding for
another patch.
>
> > Neither looks very appealing to me, so I've just kept it as-is for now.
> >
> > Maybe there's something we can do to make this easier (e.g. allowing
> > --kunitconfig to be repeated and mergable)?
>
> I would like --kunitconfig to be repeadable and mergable.
Ack.
There's some things to consider first.
1. This will conflict w/
https://patchwork.kernel.org/project/linux-kselftest/patch/20220226212325.2984807-1-dlatypov@google.com/,
so I'm going to wait until that gets merged first.
2. some kconfigs can be incompatible (e.g. these options only work on
UML, can't combine w/ a non-UML compatible file)
How do we make this less of a footgun?
We'd talked about how it'd be nice if kconfig/"make olddefconfig"
could print out *why* options get dropped (either they're not visible,
have unmet deps, etc.). If we had that, I'd feel more comfortable w/
repeatable kunitconfig.
3. People have the ability to do this already if they're really sure it's safe
$ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin
4. are we committed to supporting a "uml_coverage.kunitconfig" file?
As shown by the existence of this patch, we've let it get broken for a
bit, at least against linux-next (afaik, it was working on
torvalds/master up until the 5.18 window opened and we had some
patches reworking CONFIG_DEBUG_INFO).
These instructions exist so others don't have to try and re-figure out
the steps/workarounds.
But they're not more formally "part of KUnit" since no one has had the
expertise to maintain it (and fix issues like the reliance on gcc-6),
etc.
Creating a kunitconfig file for this will further imply ownership.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH] Documentation: kunit: update kconfig options needed for UML coverage
2022-03-28 18:58 ` Daniel Latypov
@ 2022-03-28 19:36 ` Brendan Higgins
0 siblings, 0 replies; 8+ messages in thread
From: Brendan Higgins @ 2022-03-28 19:36 UTC (permalink / raw)
To: Daniel Latypov
Cc: David Gow, Kees Cook, Linux Kernel Mailing List,
KUnit Development, open list:KERNEL SELFTEST FRAMEWORK,
Shuah Khan, maxime
On Mon, Mar 28, 2022 at 2:58 PM 'Daniel Latypov' via KUnit Development
<kunit-dev@googlegroups.com> wrote:
>
> On Mon, Mar 28, 2022 at 11:54 AM 'Brendan Higgins' via KUnit
> Development <kunit-dev@googlegroups.com> wrote:
> >
> > On Mon, Mar 28, 2022 at 12:35 PM Daniel Latypov <dlatypov@google.com> wrote:
> > >
> > > On Fri, Mar 25, 2022 at 9:56 PM David Gow <davidgow@google.com> wrote:
> > > >
> > >
> > > <snip>
> > >
> > > > > # Append coverage options to the current config
> > > > > - $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > > > > + $ echo -e "CONFIG_DEBUG_KERNEL=y\nCONFIG_DEBUG_INFO=y\nCONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y\nCONFIG_GCOV=y" >> .kunit/.kunitconfig
> > > > > $ ./tools/testing/kunit/kunit.py run
> > > >
> > > > Would we want to instead use a chain of --kconfig_add arguments? (I
> > > > think there are advantages either way...)
> > >
> > > I've been considering this ever since the --kconfig_add patch was accepted.
> > > It's more compatible w/ commands using --kunitconfig, but it also
> > > looks very verbose.
> > > E.g. it looks like
> > >
> > > $ tools/testing/kunit/kunit.py run --make_options=CC=/usr/bin/gcc-6
> > > --kconfig_add=CONFIG_DEBUG_INFO=y
> > > --kconfig_add=CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y
> > > --kconfig_add=CONFIG_GCOV=y
> >
> > I don't think it's *that* much more verbose, but I see your point. I
> > personally prefer this, but not enough to argue about it.
>
> I personally prefer it too, but I'm biased as the person who added
> --kconfig_add.
> They're both ugly enough I'd figured I'd save the bikeshedding for
> another patch.
>
> >
> > > Neither looks very appealing to me, so I've just kept it as-is for now.
> > >
> > > Maybe there's something we can do to make this easier (e.g. allowing
> > > --kunitconfig to be repeated and mergable)?
> >
> > I would like --kunitconfig to be repeadable and mergable.
>
> Ack.
> There's some things to consider first.
I wasn't saying I want you to do it now. I just like the idea.
> 1. This will conflict w/
> https://patchwork.kernel.org/project/linux-kselftest/patch/20220226212325.2984807-1-dlatypov@google.com/,
> so I'm going to wait until that gets merged first.
Agreed.
> 2. some kconfigs can be incompatible (e.g. these options only work on
> UML, can't combine w/ a non-UML compatible file)
> How do we make this less of a footgun?
> We'd talked about how it'd be nice if kconfig/"make olddefconfig"
> could print out *why* options get dropped (either they're not visible,
> have unmet deps, etc.). If we had that, I'd feel more comfortable w/
> repeatable kunitconfig.
Good point. Something to think about.
> 3. People have the ability to do this already if they're really sure it's safe
> $ cat <files...> | ./tools/testing/kunit/kunit.py run --kunitconfig=/dev/stdin
Sure, but I still think multiple --kunitconfig s would be a good
feature to add. I think it makes it a bit easier to think about mixing
and matching kunitconfigs.
> 4. are we committed to supporting a "uml_coverage.kunitconfig" file?
> As shown by the existence of this patch, we've let it get broken for a
> bit, at least against linux-next (afaik, it was working on
> torvalds/master up until the 5.18 window opened and we had some
> patches reworking CONFIG_DEBUG_INFO).
Good point. I don't think we want to get in that business. UML
Coverage needs a lot of work, and I don't think we have the time or
resources to own that work by ourselves.
If someone else wants to add - and then own - such a kunitconfig, I
would fully support them and maybe even help them a bit, but I don't
want to own such a file until UML coverage gets a bit more stable.
> These instructions exist so others don't have to try and re-figure out
> the steps/workarounds.
> But they're not more formally "part of KUnit" since no one has had the
> expertise to maintain it (and fix issues like the reliance on gcc-6),
> etc.
>
> Creating a kunitconfig file for this will further imply ownership.
Agreed.
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-03-28 19:36 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-26 0:33 [PATCH] Documentation: kunit: update kconfig options needed for UML coverage Daniel Latypov
2022-03-26 2:56 ` David Gow
2022-03-28 16:35 ` Daniel Latypov
2022-03-28 16:54 ` Brendan Higgins
2022-03-28 18:58 ` Daniel Latypov
2022-03-28 19:36 ` Brendan Higgins
2022-03-28 7:58 ` Maxime Ripard
2022-03-28 16:27 ` Brendan Higgins
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).