[dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Yann Dirson]

From: Yann Dirson <yann@blade-group.com>

The warning was not issued because of a filename error (looking for a
gatesgath-era file in older tree).

Before gatesgarth we have two files (one with short list and one with
more details), whereas starting with gatesgarth we only have a single
file with all details.  Nevertheless, for consistency with the
presentation of "invalid.cfg" we'll just print the contents of the
short "mismatch.cfg" and point the user to the more detailed
information.

Signed-off-by: Yann Dirson <yann@blade-group.com>
---
 meta/classes/kernel-yocto.bbclass | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

This is not applicable to master, where the "mismatch.txt" file exists
and shows even greater details.  Looks like the change was introduced
for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
mismatch with yocto-kernel-tools version ?

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index cc8bcb909a..771b4d8654 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -431,12 +431,13 @@ python do_kernel_configcheck() {
     bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
 
     # if config check visibility is non-zero, report dropped configuration values
-    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
+    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
     if os.path.exists(mismatch_file):
         if config_check_visibility:
+            mismatch_details_file = d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
             with open (mismatch_file, "r") as myfile:
                 results = myfile.read()
-                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration:\n\n%s" % results)
+                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration (see details in %s):\n\n%s" % (mismatch_details_file, results))
 
     if bsp_check_visibility:
         invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
-- 
2.28.0

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Bruce Ashfield]

On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>
> From: Yann Dirson <yann@blade-group.com>
>
> The warning was not issued because of a filename error (looking for a
> gatesgath-era file in older tree).
>
> Before gatesgarth we have two files (one with short list and one with
> more details), whereas starting with gatesgarth we only have a single
> file with all details.  Nevertheless, for consistency with the
> presentation of "invalid.cfg" we'll just print the contents of the
> short "mismatch.cfg" and point the user to the more detailed
> information.
>
> Signed-off-by: Yann Dirson <yann@blade-group.com>
> ---
>  meta/classes/kernel-yocto.bbclass | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> This is not applicable to master, where the "mismatch.txt" file exists
> and shows even greater details.  Looks like the change was introduced
> for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
> mismatch with yocto-kernel-tools version ?
>
> diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
> index cc8bcb909a..771b4d8654 100644
> --- a/meta/classes/kernel-yocto.bbclass
> +++ b/meta/classes/kernel-yocto.bbclass
> @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
>      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
>
>      # if config check visibility is non-zero, report dropped configuration values
> -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
> +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)

The use of mismatch.txt is completely intentional, and it is populated
by hardware tagged options that are detected as being mistmatches.

So no, we don't want to directly expose mismatch.cfg. it shows
unclassified options.

Cheers,

Bruce

>      if os.path.exists(mismatch_file):
>          if config_check_visibility:
> +            mismatch_details_file = d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
>              with open (mismatch_file, "r") as myfile:
>                  results = myfile.read()
> -                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration:\n\n%s" % results)
> +                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration (see details in %s):\n\n%s" % (mismatch_details_file, results))
>
>      if bsp_check_visibility:
>          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
> --
> 2.28.0
>
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Yann Dirson]

[-- Attachment #1: Type: text/plain, Size: 3469 bytes --]

Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <bruce.ashfield@gmail.com> a
écrit :

> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <yann.dirson@blade-group.com>
> wrote:
> >
> > From: Yann Dirson <yann@blade-group.com>
> >
> > The warning was not issued because of a filename error (looking for a
> > gatesgath-era file in older tree).
> >
> > Before gatesgarth we have two files (one with short list and one with
> > more details), whereas starting with gatesgarth we only have a single
> > file with all details.  Nevertheless, for consistency with the
> > presentation of "invalid.cfg" we'll just print the contents of the
> > short "mismatch.cfg" and point the user to the more detailed
> > information.
> >
> > Signed-off-by: Yann Dirson <yann@blade-group.com>
> > ---
> >  meta/classes/kernel-yocto.bbclass | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > This is not applicable to master, where the "mismatch.txt" file exists
> > and shows even greater details.  Looks like the change was introduced
> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
> > mismatch with yocto-kernel-tools version ?
> >
> > diff --git a/meta/classes/kernel-yocto.bbclass
> b/meta/classes/kernel-yocto.bbclass
> > index cc8bcb909a..771b4d8654 100644
> > --- a/meta/classes/kernel-yocto.bbclass
> > +++ b/meta/classes/kernel-yocto.bbclass
> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
> >
> >      # if config check visibility is non-zero, report dropped
> configuration values
> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
>
> The use of mismatch.txt is completely intentional, and it is populated
> by hardware tagged options that are detected as being mistmatches.
>
> So no, we don't want to directly expose mismatch.cfg. it shows
> unclassified options.
>

OK, so let's get back to the original problem:

- I used to have kernel_configcheck report those problems enumerated in
mismatch.cfg / mismatch-all.txt,
but for some time I'm not getting them, and this could be since migration
to sumo

- those files are generated, and seem to contain real issues, whereas there
is no mismatch.txt

What do you see going wrong and how do we move forward ?


> Cheers,
>
> Bruce
>
> >      if os.path.exists(mismatch_file):
> >          if config_check_visibility:
> > +            mismatch_details_file =
> d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
> >              with open (mismatch_file, "r") as myfile:
> >                  results = myfile.read()
> > -                bb.warn( "[kernel config]: specified values did not
> make it into the kernel's final configuration:\n\n%s" % results)
> > +                bb.warn( "[kernel config]: specified values did not
> make it into the kernel's final configuration (see details in %s):\n\n%s" %
> (mismatch_details_file, results))
> >
> >      if bsp_check_visibility:
> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
> > --
> > 2.28.0
> >
> >
> > 
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
Yann Dirson <yann@blade-group.com>
Blade / Shadow -- http://shadow.tech

[-- Attachment #2: Type: text/html, Size: 4879 bytes --]

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Bruce Ashfield]

On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>
>
>
> Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>
>> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>> >
>> > From: Yann Dirson <yann@blade-group.com>
>> >
>> > The warning was not issued because of a filename error (looking for a
>> > gatesgath-era file in older tree).
>> >
>> > Before gatesgarth we have two files (one with short list and one with
>> > more details), whereas starting with gatesgarth we only have a single
>> > file with all details.  Nevertheless, for consistency with the
>> > presentation of "invalid.cfg" we'll just print the contents of the
>> > short "mismatch.cfg" and point the user to the more detailed
>> > information.
>> >
>> > Signed-off-by: Yann Dirson <yann@blade-group.com>
>> > ---
>> >  meta/classes/kernel-yocto.bbclass | 5 +++--
>> >  1 file changed, 3 insertions(+), 2 deletions(-)
>> >
>> > This is not applicable to master, where the "mismatch.txt" file exists
>> > and shows even greater details.  Looks like the change was introduced
>> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
>> > mismatch with yocto-kernel-tools version ?
>> >
>> > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
>> > index cc8bcb909a..771b4d8654 100644
>> > --- a/meta/classes/kernel-yocto.bbclass
>> > +++ b/meta/classes/kernel-yocto.bbclass
>> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
>> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
>> >
>> >      # if config check visibility is non-zero, report dropped configuration values
>> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
>> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
>>
>> The use of mismatch.txt is completely intentional, and it is populated
>> by hardware tagged options that are detected as being mistmatches.
>>
>> So no, we don't want to directly expose mismatch.cfg. it shows
>> unclassified options.
>
>
> OK, so let's get back to the original problem:
>
> - I used to have kernel_configcheck report those problems enumerated in mismatch.cfg / mismatch-all.txt,
> but for some time I'm not getting them, and this could be since migration to sumo
>
> - those files are generated, and seem to contain real issues, whereas there is no mismatch.txt
>
> What do you see going wrong and how do we move forward ?

Is your configuration something that I can set up and reproduce the
problem ? I'm seeing what I expect out of the kconfig processing, but
there very well could be a bug that isn't showing in the machines that
I normally build.

The visibility of the warnings has changed over time (I expose them,
there are complaints, I make them less visible, and then the process
repeats ..), so there could be something there as well.

And as you noticed, I totally rewrote things in master, since
maintaining the existing script was just too painful.

Cheers,

Bruce

>
>>
>> Cheers,
>>
>> Bruce
>>
>> >      if os.path.exists(mismatch_file):
>> >          if config_check_visibility:
>> > +            mismatch_details_file = d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
>> >              with open (mismatch_file, "r") as myfile:
>> >                  results = myfile.read()
>> > -                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration:\n\n%s" % results)
>> > +                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration (see details in %s):\n\n%s" % (mismatch_details_file, results))
>> >
>> >      if bsp_check_visibility:
>> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
>> > --
>> > 2.28.0
>> >
>> >
>> > 
>> >
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Yann Dirson <yann@blade-group.com>
> Blade / Shadow -- http://shadow.tech
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Yann Dirson]

[-- Attachment #1: Type: text/plain, Size: 5616 bytes --]

Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <bruce.ashfield@gmail.com> a
écrit :

> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <yann.dirson@blade-group.com>
> wrote:
> >
> >
> >
> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <bruce.ashfield@gmail.com>
> a écrit :
> >>
> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <
> yann.dirson@blade-group.com> wrote:
> >> >
> >> > From: Yann Dirson <yann@blade-group.com>
> >> >
> >> > The warning was not issued because of a filename error (looking for a
> >> > gatesgath-era file in older tree).
> >> >
> >> > Before gatesgarth we have two files (one with short list and one with
> >> > more details), whereas starting with gatesgarth we only have a single
> >> > file with all details.  Nevertheless, for consistency with the
> >> > presentation of "invalid.cfg" we'll just print the contents of the
> >> > short "mismatch.cfg" and point the user to the more detailed
> >> > information.
> >> >
> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
> >> > ---
> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >> >
> >> > This is not applicable to master, where the "mismatch.txt" file exists
> >> > and shows even greater details.  Looks like the change was introduced
> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
> >> > mismatch with yocto-kernel-tools version ?
> >> >
> >> > diff --git a/meta/classes/kernel-yocto.bbclass
> b/meta/classes/kernel-yocto.bbclass
> >> > index cc8bcb909a..771b4d8654 100644
> >> > --- a/meta/classes/kernel-yocto.bbclass
> >> > +++ b/meta/classes/kernel-yocto.bbclass
> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
> >> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or
> 0)
> >> >
> >> >      # if config check visibility is non-zero, report dropped
> configuration values
> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
> >>
> >> The use of mismatch.txt is completely intentional, and it is populated
> >> by hardware tagged options that are detected as being mistmatches.
> >>
> >> So no, we don't want to directly expose mismatch.cfg. it shows
> >> unclassified options.
> >
> >
> > OK, so let's get back to the original problem:
> >
> > - I used to have kernel_configcheck report those problems enumerated in
> mismatch.cfg / mismatch-all.txt,
> > but for some time I'm not getting them, and this could be since
> migration to sumo
> >
> > - those files are generated, and seem to contain real issues, whereas
> there is no mismatch.txt
> >
> > What do you see going wrong and how do we move forward ?
>
> Is your configuration something that I can set up and reproduce the
> problem ? I'm seeing what I expect out of the kconfig processing, but
> there very well could be a bug that isn't showing in the machines that
> I normally build.
>

The problem can be triggered with the default qemu_x86-64 configuration.
I'm just adding this in local.conf to inject a few options:

SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"

with this line in stuff.cfg:

CONFIG_EEEPC_LAPTOP=y

On master I see:

WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0
do_kernel_configcheck: [kernel config]: specified values did not make it
into the kernel's final configuration:

    [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do not
match
    [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353
:configs/v5.8/standard/./hibern.cfg (y)

But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed in
mismatch*, though it is just
one of 127 issues, most of which looking pretty much valid at first
glance.  Are they not ?


> The visibility of the warnings has changed over time (I expose them,
> there are complaints, I make them less visible, and then the process
> repeats ..), so there could be something there as well.
>
> And as you noticed, I totally rewrote things in master, since
> maintaining the existing script was just too painful.
>
> Cheers,
>
> Bruce
>
> >
> >>
> >> Cheers,
> >>
> >> Bruce
> >>
> >> >      if os.path.exists(mismatch_file):
> >> >          if config_check_visibility:
> >> > +            mismatch_details_file =
> d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
> >> >              with open (mismatch_file, "r") as myfile:
> >> >                  results = myfile.read()
> >> > -                bb.warn( "[kernel config]: specified values did not
> make it into the kernel's final configuration:\n\n%s" % results)
> >> > +                bb.warn( "[kernel config]: specified values did not
> make it into the kernel's final configuration (see details in %s):\n\n%s" %
> (mismatch_details_file, results))
> >> >
> >> >      if bsp_check_visibility:
> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
> >> > --
> >> > 2.28.0
> >> >
> >> >
> >> > 
> >> >
> >>
> >>
> >> --
> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> thee at its end
> >> - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > Yann Dirson <yann@blade-group.com>
> > Blade / Shadow -- http://shadow.tech
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
Yann Dirson <yann@blade-group.com>
Blade / Shadow -- http://shadow.tech

[-- Attachment #2: Type: text/html, Size: 7986 bytes --]

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Bruce Ashfield]

On Tue, Oct 13, 2020 at 10:46 AM Yann Dirson
<yann.dirson@blade-group.com> wrote:
>
>
>
> Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>
>> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>> >
>> >
>> >
>> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>> >>
>> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>> >> >
>> >> > From: Yann Dirson <yann@blade-group.com>
>> >> >
>> >> > The warning was not issued because of a filename error (looking for a
>> >> > gatesgath-era file in older tree).
>> >> >
>> >> > Before gatesgarth we have two files (one with short list and one with
>> >> > more details), whereas starting with gatesgarth we only have a single
>> >> > file with all details.  Nevertheless, for consistency with the
>> >> > presentation of "invalid.cfg" we'll just print the contents of the
>> >> > short "mismatch.cfg" and point the user to the more detailed
>> >> > information.
>> >> >
>> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
>> >> > ---
>> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
>> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
>> >> >
>> >> > This is not applicable to master, where the "mismatch.txt" file exists
>> >> > and shows even greater details.  Looks like the change was introduced
>> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
>> >> > mismatch with yocto-kernel-tools version ?
>> >> >
>> >> > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
>> >> > index cc8bcb909a..771b4d8654 100644
>> >> > --- a/meta/classes/kernel-yocto.bbclass
>> >> > +++ b/meta/classes/kernel-yocto.bbclass
>> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
>> >> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
>> >> >
>> >> >      # if config check visibility is non-zero, report dropped configuration values
>> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
>> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
>> >>
>> >> The use of mismatch.txt is completely intentional, and it is populated
>> >> by hardware tagged options that are detected as being mistmatches.
>> >>
>> >> So no, we don't want to directly expose mismatch.cfg. it shows
>> >> unclassified options.
>> >
>> >
>> > OK, so let's get back to the original problem:
>> >
>> > - I used to have kernel_configcheck report those problems enumerated in mismatch.cfg / mismatch-all.txt,
>> > but for some time I'm not getting them, and this could be since migration to sumo
>> >
>> > - those files are generated, and seem to contain real issues, whereas there is no mismatch.txt
>> >
>> > What do you see going wrong and how do we move forward ?
>>
>> Is your configuration something that I can set up and reproduce the
>> problem ? I'm seeing what I expect out of the kconfig processing, but
>> there very well could be a bug that isn't showing in the machines that
>> I normally build.
>
>
> The problem can be triggered with the default qemu_x86-64 configuration.
> I'm just adding this in local.conf to inject a few options:
>
> SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"
>
> with this line in stuff.cfg:
>
> CONFIG_EEEPC_LAPTOP=y
>
> On master I see:
>
> WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:
>
>     [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do not match
>     [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353 :configs/v5.8/standard/./hibern.cfg (y)
>
> But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed in mismatch*, though it is just
> one of 127 issues, most of which looking pretty much valid at first glance.  Are they not ?

Only if something is tagged as "hardware" is it reported. The
classification name is historical, so you can read "hardware" as
"required / important / needed for boot". The organization of the
fragments (and a push and pull between stand alone fragments,
different configurations, software stacks, etc) means that there are
many fragments that have some options listed, that are disabled due to
competing interests. Those are for the most part valid (to be
different), and only differing values for the ones tagged as hardware
are reported.

So no, we really don't want to report all those options. That's the
wobbling between reporting and non-reporting that I mentioned before.

In non-master, there's a complex way to tag options as hardware (and
hopefully an easier way in master), so if there's a mismatch you'll
see it.

The delta between the two could be that I re-did the default
classification buckets for the new kernel and that's why it shows in
one, and not the other.

 I can fire up a build on dunfell and see if I can see where the
classification differs.

Bruce

>
>>
>> The visibility of the warnings has changed over time (I expose them,
>> there are complaints, I make them less visible, and then the process
>> repeats ..), so there could be something there as well.
>>
>> And as you noticed, I totally rewrote things in master, since
>> maintaining the existing script was just too painful.
>>
>> Cheers,
>>
>> Bruce
>>
>> >
>> >>
>> >> Cheers,
>> >>
>> >> Bruce
>> >>
>> >> >      if os.path.exists(mismatch_file):
>> >> >          if config_check_visibility:
>> >> > +            mismatch_details_file = d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
>> >> >              with open (mismatch_file, "r") as myfile:
>> >> >                  results = myfile.read()
>> >> > -                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration:\n\n%s" % results)
>> >> > +                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration (see details in %s):\n\n%s" % (mismatch_details_file, results))
>> >> >
>> >> >      if bsp_check_visibility:
>> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
>> >> > --
>> >> > 2.28.0
>> >> >
>> >> >
>> >> > 
>> >> >
>> >>
>> >>
>> >> --
>> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> thee at its end
>> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >
>> >
>> >
>> > --
>> > Yann Dirson <yann@blade-group.com>
>> > Blade / Shadow -- http://shadow.tech
>> >
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Yann Dirson <yann@blade-group.com>
> Blade / Shadow -- http://shadow.tech
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Yann Dirson]

[-- Attachment #1: Type: text/plain, Size: 7992 bytes --]

Le mar. 13 oct. 2020 à 17:04, Bruce Ashfield <bruce.ashfield@gmail.com> a
écrit :

> On Tue, Oct 13, 2020 at 10:46 AM Yann Dirson
> <yann.dirson@blade-group.com> wrote:
> >
> >
> >
> > Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <bruce.ashfield@gmail.com>
> a écrit :
> >>
> >> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <
> yann.dirson@blade-group.com> wrote:
> >> >
> >> >
> >> >
> >> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <
> bruce.ashfield@gmail.com> a écrit :
> >> >>
> >> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <
> yann.dirson@blade-group.com> wrote:
> >> >> >
> >> >> > From: Yann Dirson <yann@blade-group.com>
> >> >> >
> >> >> > The warning was not issued because of a filename error (looking
> for a
> >> >> > gatesgath-era file in older tree).
> >> >> >
> >> >> > Before gatesgarth we have two files (one with short list and one
> with
> >> >> > more details), whereas starting with gatesgarth we only have a
> single
> >> >> > file with all details.  Nevertheless, for consistency with the
> >> >> > presentation of "invalid.cfg" we'll just print the contents of the
> >> >> > short "mismatch.cfg" and point the user to the more detailed
> >> >> > information.
> >> >> >
> >> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
> >> >> > ---
> >> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
> >> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >> >> >
> >> >> > This is not applicable to master, where the "mismatch.txt" file
> exists
> >> >> > and shows even greater details.  Looks like the change was
> introduced
> >> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that
> be a
> >> >> > mismatch with yocto-kernel-tools version ?
> >> >> >
> >> >> > diff --git a/meta/classes/kernel-yocto.bbclass
> b/meta/classes/kernel-yocto.bbclass
> >> >> > index cc8bcb909a..771b4d8654 100644
> >> >> > --- a/meta/classes/kernel-yocto.bbclass
> >> >> > +++ b/meta/classes/kernel-yocto.bbclass
> >> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
> >> >> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL")
> or 0)
> >> >> >
> >> >> >      # if config check visibility is non-zero, report dropped
> configuration values
> >> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
> >> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
> >> >>
> >> >> The use of mismatch.txt is completely intentional, and it is
> populated
> >> >> by hardware tagged options that are detected as being mistmatches.
> >> >>
> >> >> So no, we don't want to directly expose mismatch.cfg. it shows
> >> >> unclassified options.
> >> >
> >> >
> >> > OK, so let's get back to the original problem:
> >> >
> >> > - I used to have kernel_configcheck report those problems enumerated
> in mismatch.cfg / mismatch-all.txt,
> >> > but for some time I'm not getting them, and this could be since
> migration to sumo
> >> >
> >> > - those files are generated, and seem to contain real issues, whereas
> there is no mismatch.txt
> >> >
> >> > What do you see going wrong and how do we move forward ?
> >>
> >> Is your configuration something that I can set up and reproduce the
> >> problem ? I'm seeing what I expect out of the kconfig processing, but
> >> there very well could be a bug that isn't showing in the machines that
> >> I normally build.
> >
> >
> > The problem can be triggered with the default qemu_x86-64 configuration.
> > I'm just adding this in local.conf to inject a few options:
> >
> > SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"
> >
> > with this line in stuff.cfg:
> >
> > CONFIG_EEEPC_LAPTOP=y
> >
> > On master I see:
> >
> > WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0
> do_kernel_configcheck: [kernel config]: specified values did not make it
> into the kernel's final configuration:
> >
> >     [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do
> not match
> >     [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353
> :configs/v5.8/standard/./hibern.cfg (y)
> >
> > But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed in
> mismatch*, though it is just
> > one of 127 issues, most of which looking pretty much valid at first
> glance.  Are they not ?
>
> Only if something is tagged as "hardware" is it reported. The
> classification name is historical, so you can read "hardware" as
> "required / important / needed for boot". The organization of the
> fragments (and a push and pull between stand alone fragments,
> different configurations, software stacks, etc) means that there are
> many fragments that have some options listed, that are disabled due to
> competing interests. Those are for the most part valid (to be
> different), and only differing values for the ones tagged as hardware
> are reported.
>
> So no, we really don't want to report all those options. That's the
> wobbling between reporting and non-reporting that I mentioned before.
>

Hm.  In our case, I've tried to write carefully-non-overlapping snippets,
and I'd like to get a warning on *any* mismatch because they would point
to a bug in my files.  Should I write an additional configcheck stage that
would check for mismatch.cfg by itself ?  If so, what about gatesgarth ?


> In non-master, there's a complex way to tag options as hardware (and
> hopefully an easier way in master), so if there's a mismatch you'll
> see it.
>
> The delta between the two could be that I re-did the default
> classification buckets for the new kernel and that's why it shows in
> one, and not the other.
>
>  I can fire up a build on dunfell and see if I can see where the
> classification differs.
>
> Bruce
>
> >
> >>
> >> The visibility of the warnings has changed over time (I expose them,
> >> there are complaints, I make them less visible, and then the process
> >> repeats ..), so there could be something there as well.
> >>
> >> And as you noticed, I totally rewrote things in master, since
> >> maintaining the existing script was just too painful.
> >>
> >> Cheers,
> >>
> >> Bruce
> >>
> >> >
> >> >>
> >> >> Cheers,
> >> >>
> >> >> Bruce
> >> >>
> >> >> >      if os.path.exists(mismatch_file):
> >> >> >          if config_check_visibility:
> >> >> > +            mismatch_details_file =
> d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
> >> >> >              with open (mismatch_file, "r") as myfile:
> >> >> >                  results = myfile.read()
> >> >> > -                bb.warn( "[kernel config]: specified values did
> not make it into the kernel's final configuration:\n\n%s" % results)
> >> >> > +                bb.warn( "[kernel config]: specified values did
> not make it into the kernel's final configuration (see details in
> %s):\n\n%s" % (mismatch_details_file, results))
> >> >> >
> >> >> >      if bsp_check_visibility:
> >> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
> >> >> > --
> >> >> > 2.28.0
> >> >> >
> >> >> >
> >> >> > 
> >> >> >
> >> >>
> >> >>
> >> >> --
> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> >> thee at its end
> >> >> - "Use the force Harry" - Gandalf, Star Trek II
> >> >
> >> >
> >> >
> >> > --
> >> > Yann Dirson <yann@blade-group.com>
> >> > Blade / Shadow -- http://shadow.tech
> >> >
> >>
> >>
> >> --
> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> thee at its end
> >> - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > Yann Dirson <yann@blade-group.com>
> > Blade / Shadow -- http://shadow.tech
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
Yann Dirson <yann@blade-group.com>
Blade / Shadow -- http://shadow.tech

[-- Attachment #2: Type: text/html, Size: 11460 bytes --]

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Bruce Ashfield]

On Tue, Oct 13, 2020 at 11:12 AM Yann Dirson
<yann.dirson@blade-group.com> wrote:
>
>
>
> Le mar. 13 oct. 2020 à 17:04, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>
>> On Tue, Oct 13, 2020 at 10:46 AM Yann Dirson
>> <yann.dirson@blade-group.com> wrote:
>> >
>> >
>> >
>> > Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>> >>
>> >> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>> >> >
>> >> >
>> >> >
>> >> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>> >> >>
>> >> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>> >> >> >
>> >> >> > From: Yann Dirson <yann@blade-group.com>
>> >> >> >
>> >> >> > The warning was not issued because of a filename error (looking for a
>> >> >> > gatesgath-era file in older tree).
>> >> >> >
>> >> >> > Before gatesgarth we have two files (one with short list and one with
>> >> >> > more details), whereas starting with gatesgarth we only have a single
>> >> >> > file with all details.  Nevertheless, for consistency with the
>> >> >> > presentation of "invalid.cfg" we'll just print the contents of the
>> >> >> > short "mismatch.cfg" and point the user to the more detailed
>> >> >> > information.
>> >> >> >
>> >> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
>> >> >> > ---
>> >> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
>> >> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
>> >> >> >
>> >> >> > This is not applicable to master, where the "mismatch.txt" file exists
>> >> >> > and shows even greater details.  Looks like the change was introduced
>> >> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
>> >> >> > mismatch with yocto-kernel-tools version ?
>> >> >> >
>> >> >> > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
>> >> >> > index cc8bcb909a..771b4d8654 100644
>> >> >> > --- a/meta/classes/kernel-yocto.bbclass
>> >> >> > +++ b/meta/classes/kernel-yocto.bbclass
>> >> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
>> >> >> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
>> >> >> >
>> >> >> >      # if config check visibility is non-zero, report dropped configuration values
>> >> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
>> >> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
>> >> >>
>> >> >> The use of mismatch.txt is completely intentional, and it is populated
>> >> >> by hardware tagged options that are detected as being mistmatches.
>> >> >>
>> >> >> So no, we don't want to directly expose mismatch.cfg. it shows
>> >> >> unclassified options.
>> >> >
>> >> >
>> >> > OK, so let's get back to the original problem:
>> >> >
>> >> > - I used to have kernel_configcheck report those problems enumerated in mismatch.cfg / mismatch-all.txt,
>> >> > but for some time I'm not getting them, and this could be since migration to sumo
>> >> >
>> >> > - those files are generated, and seem to contain real issues, whereas there is no mismatch.txt
>> >> >
>> >> > What do you see going wrong and how do we move forward ?
>> >>
>> >> Is your configuration something that I can set up and reproduce the
>> >> problem ? I'm seeing what I expect out of the kconfig processing, but
>> >> there very well could be a bug that isn't showing in the machines that
>> >> I normally build.
>> >
>> >
>> > The problem can be triggered with the default qemu_x86-64 configuration.
>> > I'm just adding this in local.conf to inject a few options:
>> >
>> > SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"
>> >
>> > with this line in stuff.cfg:
>> >
>> > CONFIG_EEEPC_LAPTOP=y
>> >
>> > On master I see:
>> >
>> > WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:
>> >
>> >     [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do not match
>> >     [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353 :configs/v5.8/standard/./hibern.cfg (y)
>> >
>> > But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed in mismatch*, though it is just
>> > one of 127 issues, most of which looking pretty much valid at first glance.  Are they not ?
>>
>> Only if something is tagged as "hardware" is it reported. The
>> classification name is historical, so you can read "hardware" as
>> "required / important / needed for boot". The organization of the
>> fragments (and a push and pull between stand alone fragments,
>> different configurations, software stacks, etc) means that there are
>> many fragments that have some options listed, that are disabled due to
>> competing interests. Those are for the most part valid (to be
>> different), and only differing values for the ones tagged as hardware
>> are reported.
>>
>> So no, we really don't want to report all those options. That's the
>> wobbling between reporting and non-reporting that I mentioned before.
>
>
> Hm.  In our case, I've tried to write carefully-non-overlapping snippets,
> and I'd like to get a warning on *any* mismatch because they would point
> to a bug in my files.  Should I write an additional configcheck stage that
> would check for mismatch.cfg by itself ?  If so, what about gatesgarth ?

That's what the KCONF_BSP_AUDIT_LEVEL variable is for. It is used
right now with:

    if bsp_check_visibility > 2:
        redefinition_file = d.expand("${S}/%s/cfg/redefinition.txt" % kmeta)

In the kconfig audit reporting phase.  That dumps out a huge amount of
info and will show you any duplicated/differing options through the fragment
stack. In your own layer you could add the dumping of mismatch.cfg to that
step if you wanted (or invent a new, higher level of logging).

Master already has a better way to deal with this, via the 'config blame'
task.

Bruce

>
>>
>> In non-master, there's a complex way to tag options as hardware (and
>> hopefully an easier way in master), so if there's a mismatch you'll
>> see it.
>>
>> The delta between the two could be that I re-did the default
>> classification buckets for the new kernel and that's why it shows in
>> one, and not the other.
>>
>>  I can fire up a build on dunfell and see if I can see where the
>> classification differs.
>>
>> Bruce
>>
>> >
>> >>
>> >> The visibility of the warnings has changed over time (I expose them,
>> >> there are complaints, I make them less visible, and then the process
>> >> repeats ..), so there could be something there as well.
>> >>
>> >> And as you noticed, I totally rewrote things in master, since
>> >> maintaining the existing script was just too painful.
>> >>
>> >> Cheers,
>> >>
>> >> Bruce
>> >>
>> >> >
>> >> >>
>> >> >> Cheers,
>> >> >>
>> >> >> Bruce
>> >> >>
>> >> >> >      if os.path.exists(mismatch_file):
>> >> >> >          if config_check_visibility:
>> >> >> > +            mismatch_details_file = d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
>> >> >> >              with open (mismatch_file, "r") as myfile:
>> >> >> >                  results = myfile.read()
>> >> >> > -                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration:\n\n%s" % results)
>> >> >> > +                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration (see details in %s):\n\n%s" % (mismatch_details_file, results))
>> >> >> >
>> >> >> >      if bsp_check_visibility:
>> >> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
>> >> >> > --
>> >> >> > 2.28.0
>> >> >> >
>> >> >> >
>> >> >> > 
>> >> >> >
>> >> >>
>> >> >>
>> >> >> --
>> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> >> thee at its end
>> >> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Yann Dirson <yann@blade-group.com>
>> >> > Blade / Shadow -- http://shadow.tech
>> >> >
>> >>
>> >>
>> >> --
>> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> thee at its end
>> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >
>> >
>> >
>> > --
>> > Yann Dirson <yann@blade-group.com>
>> > Blade / Shadow -- http://shadow.tech
>> >
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Yann Dirson <yann@blade-group.com>
> Blade / Shadow -- http://shadow.tech
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Yann Dirson]

[-- Attachment #1: Type: text/plain, Size: 10058 bytes --]

Le mar. 13 oct. 2020 à 17:33, Bruce Ashfield <bruce.ashfield@gmail.com> a
écrit :

> On Tue, Oct 13, 2020 at 11:12 AM Yann Dirson
> <yann.dirson@blade-group.com> wrote:
> >
> >
> >
> > Le mar. 13 oct. 2020 à 17:04, Bruce Ashfield <bruce.ashfield@gmail.com>
> a écrit :
> >>
> >> On Tue, Oct 13, 2020 at 10:46 AM Yann Dirson
> >> <yann.dirson@blade-group.com> wrote:
> >> >
> >> >
> >> >
> >> > Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <
> bruce.ashfield@gmail.com> a écrit :
> >> >>
> >> >> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <
> yann.dirson@blade-group.com> wrote:
> >> >> >
> >> >> >
> >> >> >
> >> >> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <
> bruce.ashfield@gmail.com> a écrit :
> >> >> >>
> >> >> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <
> yann.dirson@blade-group.com> wrote:
> >> >> >> >
> >> >> >> > From: Yann Dirson <yann@blade-group.com>
> >> >> >> >
> >> >> >> > The warning was not issued because of a filename error (looking
> for a
> >> >> >> > gatesgath-era file in older tree).
> >> >> >> >
> >> >> >> > Before gatesgarth we have two files (one with short list and
> one with
> >> >> >> > more details), whereas starting with gatesgarth we only have a
> single
> >> >> >> > file with all details.  Nevertheless, for consistency with the
> >> >> >> > presentation of "invalid.cfg" we'll just print the contents of
> the
> >> >> >> > short "mismatch.cfg" and point the user to the more detailed
> >> >> >> > information.
> >> >> >> >
> >> >> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
> >> >> >> > ---
> >> >> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
> >> >> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >> >> >> >
> >> >> >> > This is not applicable to master, where the "mismatch.txt" file
> exists
> >> >> >> > and shows even greater details.  Looks like the change was
> introduced
> >> >> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would
> that be a
> >> >> >> > mismatch with yocto-kernel-tools version ?
> >> >> >> >
> >> >> >> > diff --git a/meta/classes/kernel-yocto.bbclass
> b/meta/classes/kernel-yocto.bbclass
> >> >> >> > index cc8bcb909a..771b4d8654 100644
> >> >> >> > --- a/meta/classes/kernel-yocto.bbclass
> >> >> >> > +++ b/meta/classes/kernel-yocto.bbclass
> >> >> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
> >> >> >> >      bsp_check_visibility =
> int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
> >> >> >> >
> >> >> >> >      # if config check visibility is non-zero, report dropped
> configuration values
> >> >> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" %
> kmeta)
> >> >> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" %
> kmeta)
> >> >> >>
> >> >> >> The use of mismatch.txt is completely intentional, and it is
> populated
> >> >> >> by hardware tagged options that are detected as being mistmatches.
> >> >> >>
> >> >> >> So no, we don't want to directly expose mismatch.cfg. it shows
> >> >> >> unclassified options.
> >> >> >
> >> >> >
> >> >> > OK, so let's get back to the original problem:
> >> >> >
> >> >> > - I used to have kernel_configcheck report those problems
> enumerated in mismatch.cfg / mismatch-all.txt,
> >> >> > but for some time I'm not getting them, and this could be since
> migration to sumo
> >> >> >
> >> >> > - those files are generated, and seem to contain real issues,
> whereas there is no mismatch.txt
> >> >> >
> >> >> > What do you see going wrong and how do we move forward ?
> >> >>
> >> >> Is your configuration something that I can set up and reproduce the
> >> >> problem ? I'm seeing what I expect out of the kconfig processing, but
> >> >> there very well could be a bug that isn't showing in the machines
> that
> >> >> I normally build.
> >> >
> >> >
> >> > The problem can be triggered with the default qemu_x86-64
> configuration.
> >> > I'm just adding this in local.conf to inject a few options:
> >> >
> >> > SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"
> >> >
> >> > with this line in stuff.cfg:
> >> >
> >> > CONFIG_EEEPC_LAPTOP=y
> >> >
> >> > On master I see:
> >> >
> >> > WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0
> do_kernel_configcheck: [kernel config]: specified values did not make it
> into the kernel's final configuration:
> >> >
> >> >     [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do
> not match
> >> >     [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353
> :configs/v5.8/standard/./hibern.cfg (y)
> >> >
> >> > But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed
> in mismatch*, though it is just
> >> > one of 127 issues, most of which looking pretty much valid at first
> glance.  Are they not ?
> >>
> >> Only if something is tagged as "hardware" is it reported. The
> >> classification name is historical, so you can read "hardware" as
> >> "required / important / needed for boot". The organization of the
> >> fragments (and a push and pull between stand alone fragments,
> >> different configurations, software stacks, etc) means that there are
> >> many fragments that have some options listed, that are disabled due to
> >> competing interests. Those are for the most part valid (to be
> >> different), and only differing values for the ones tagged as hardware
> >> are reported.
> >>
> >> So no, we really don't want to report all those options. That's the
> >> wobbling between reporting and non-reporting that I mentioned before.
> >
> >
> > Hm.  In our case, I've tried to write carefully-non-overlapping snippets,
> > and I'd like to get a warning on *any* mismatch because they would point
> > to a bug in my files.  Should I write an additional configcheck stage
> that
> > would check for mismatch.cfg by itself ?  If so, what about gatesgarth ?
>
> That's what the KCONF_BSP_AUDIT_LEVEL variable is for. It is used
> right now with:
>
>     if bsp_check_visibility > 2:
>         redefinition_file = d.expand("${S}/%s/cfg/redefinition.txt" %
> kmeta)
>
> In the kconfig audit reporting phase.  That dumps out a huge amount of
> info and will show you any duplicated/differing options through the
> fragment
> stack. In your own layer you could add the dumping of mismatch.cfg to that
> step if you wanted (or invent a new, higher level of logging).
>

Oh I should have read the code more carefully, I would have noticed that,
even
as the doc does not mention it (in fact it seems that in dunfell  "2" is
not doing anything
more than "1", right ?):

# LEVELS:
#   0: no reporting
#   1: report options that are specified, but not in the final config
#   2: report options that are not hardware related, but set by a BSP


> Master already has a better way to deal with this, via the 'config blame'
> task.
>
> Bruce
>
> >
> >>
> >> In non-master, there's a complex way to tag options as hardware (and
> >> hopefully an easier way in master), so if there's a mismatch you'll
> >> see it.
> >>
> >> The delta between the two could be that I re-did the default
> >> classification buckets for the new kernel and that's why it shows in
> >> one, and not the other.
> >>
> >>  I can fire up a build on dunfell and see if I can see where the
> >> classification differs.
> >>
> >> Bruce
> >>
> >> >
> >> >>
> >> >> The visibility of the warnings has changed over time (I expose them,
> >> >> there are complaints, I make them less visible, and then the process
> >> >> repeats ..), so there could be something there as well.
> >> >>
> >> >> And as you noticed, I totally rewrote things in master, since
> >> >> maintaining the existing script was just too painful.
> >> >>
> >> >> Cheers,
> >> >>
> >> >> Bruce
> >> >>
> >> >> >
> >> >> >>
> >> >> >> Cheers,
> >> >> >>
> >> >> >> Bruce
> >> >> >>
> >> >> >> >      if os.path.exists(mismatch_file):
> >> >> >> >          if config_check_visibility:
> >> >> >> > +            mismatch_details_file =
> d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
> >> >> >> >              with open (mismatch_file, "r") as myfile:
> >> >> >> >                  results = myfile.read()
> >> >> >> > -                bb.warn( "[kernel config]: specified values
> did not make it into the kernel's final configuration:\n\n%s" % results)
> >> >> >> > +                bb.warn( "[kernel config]: specified values
> did not make it into the kernel's final configuration (see details in
> %s):\n\n%s" % (mismatch_details_file, results))
> >> >> >> >
> >> >> >> >      if bsp_check_visibility:
> >> >> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" %
> kmeta)
> >> >> >> > --
> >> >> >> > 2.28.0
> >> >> >> >
> >> >> >> >
> >> >> >> > 
> >> >> >> >
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness
> await
> >> >> >> thee at its end
> >> >> >> - "Use the force Harry" - Gandalf, Star Trek II
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Yann Dirson <yann@blade-group.com>
> >> >> > Blade / Shadow -- http://shadow.tech
> >> >> >
> >> >>
> >> >>
> >> >> --
> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> >> thee at its end
> >> >> - "Use the force Harry" - Gandalf, Star Trek II
> >> >
> >> >
> >> >
> >> > --
> >> > Yann Dirson <yann@blade-group.com>
> >> > Blade / Shadow -- http://shadow.tech
> >> >
> >>
> >>
> >> --
> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
> >> thee at its end
> >> - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > Yann Dirson <yann@blade-group.com>
> > Blade / Shadow -- http://shadow.tech
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>


-- 
Yann Dirson <yann@blade-group.com>
Blade / Shadow -- http://shadow.tech

[-- Attachment #2: Type: text/html, Size: 15084 bytes --]

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Bruce Ashfield]

On Tue, Oct 13, 2020 at 11:47 AM Yann Dirson
<yann.dirson@blade-group.com> wrote:
>
>
>
> Le mar. 13 oct. 2020 à 17:33, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>
>> On Tue, Oct 13, 2020 at 11:12 AM Yann Dirson
>> <yann.dirson@blade-group.com> wrote:
>> >
>> >
>> >
>> > Le mar. 13 oct. 2020 à 17:04, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>> >>
>> >> On Tue, Oct 13, 2020 at 10:46 AM Yann Dirson
>> >> <yann.dirson@blade-group.com> wrote:
>> >> >
>> >> >
>> >> >
>> >> > Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>> >> >>
>> >> >> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>> >> >> >>
>> >> >> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>> >> >> >> >
>> >> >> >> > From: Yann Dirson <yann@blade-group.com>
>> >> >> >> >
>> >> >> >> > The warning was not issued because of a filename error (looking for a
>> >> >> >> > gatesgath-era file in older tree).
>> >> >> >> >
>> >> >> >> > Before gatesgarth we have two files (one with short list and one with
>> >> >> >> > more details), whereas starting with gatesgarth we only have a single
>> >> >> >> > file with all details.  Nevertheless, for consistency with the
>> >> >> >> > presentation of "invalid.cfg" we'll just print the contents of the
>> >> >> >> > short "mismatch.cfg" and point the user to the more detailed
>> >> >> >> > information.
>> >> >> >> >
>> >> >> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
>> >> >> >> > ---
>> >> >> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
>> >> >> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
>> >> >> >> >
>> >> >> >> > This is not applicable to master, where the "mismatch.txt" file exists
>> >> >> >> > and shows even greater details.  Looks like the change was introduced
>> >> >> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
>> >> >> >> > mismatch with yocto-kernel-tools version ?
>> >> >> >> >
>> >> >> >> > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
>> >> >> >> > index cc8bcb909a..771b4d8654 100644
>> >> >> >> > --- a/meta/classes/kernel-yocto.bbclass
>> >> >> >> > +++ b/meta/classes/kernel-yocto.bbclass
>> >> >> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
>> >> >> >> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
>> >> >> >> >
>> >> >> >> >      # if config check visibility is non-zero, report dropped configuration values
>> >> >> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
>> >> >> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
>> >> >> >>
>> >> >> >> The use of mismatch.txt is completely intentional, and it is populated
>> >> >> >> by hardware tagged options that are detected as being mistmatches.
>> >> >> >>
>> >> >> >> So no, we don't want to directly expose mismatch.cfg. it shows
>> >> >> >> unclassified options.
>> >> >> >
>> >> >> >
>> >> >> > OK, so let's get back to the original problem:
>> >> >> >
>> >> >> > - I used to have kernel_configcheck report those problems enumerated in mismatch.cfg / mismatch-all.txt,
>> >> >> > but for some time I'm not getting them, and this could be since migration to sumo
>> >> >> >
>> >> >> > - those files are generated, and seem to contain real issues, whereas there is no mismatch.txt
>> >> >> >
>> >> >> > What do you see going wrong and how do we move forward ?
>> >> >>
>> >> >> Is your configuration something that I can set up and reproduce the
>> >> >> problem ? I'm seeing what I expect out of the kconfig processing, but
>> >> >> there very well could be a bug that isn't showing in the machines that
>> >> >> I normally build.
>> >> >
>> >> >
>> >> > The problem can be triggered with the default qemu_x86-64 configuration.
>> >> > I'm just adding this in local.conf to inject a few options:
>> >> >
>> >> > SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"
>> >> >
>> >> > with this line in stuff.cfg:
>> >> >
>> >> > CONFIG_EEEPC_LAPTOP=y
>> >> >
>> >> > On master I see:
>> >> >
>> >> > WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:
>> >> >
>> >> >     [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do not match
>> >> >     [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353 :configs/v5.8/standard/./hibern.cfg (y)
>> >> >
>> >> > But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed in mismatch*, though it is just
>> >> > one of 127 issues, most of which looking pretty much valid at first glance.  Are they not ?
>> >>
>> >> Only if something is tagged as "hardware" is it reported. The
>> >> classification name is historical, so you can read "hardware" as
>> >> "required / important / needed for boot". The organization of the
>> >> fragments (and a push and pull between stand alone fragments,
>> >> different configurations, software stacks, etc) means that there are
>> >> many fragments that have some options listed, that are disabled due to
>> >> competing interests. Those are for the most part valid (to be
>> >> different), and only differing values for the ones tagged as hardware
>> >> are reported.
>> >>
>> >> So no, we really don't want to report all those options. That's the
>> >> wobbling between reporting and non-reporting that I mentioned before.
>> >
>> >
>> > Hm.  In our case, I've tried to write carefully-non-overlapping snippets,
>> > and I'd like to get a warning on *any* mismatch because they would point
>> > to a bug in my files.  Should I write an additional configcheck stage that
>> > would check for mismatch.cfg by itself ?  If so, what about gatesgarth ?
>>
>> That's what the KCONF_BSP_AUDIT_LEVEL variable is for. It is used
>> right now with:
>>
>>     if bsp_check_visibility > 2:
>>         redefinition_file = d.expand("${S}/%s/cfg/redefinition.txt" % kmeta)
>>
>> In the kconfig audit reporting phase.  That dumps out a huge amount of
>> info and will show you any duplicated/differing options through the fragment
>> stack. In your own layer you could add the dumping of mismatch.cfg to that
>> step if you wanted (or invent a new, higher level of logging).
>
>
> Oh I should have read the code more carefully, I would have noticed that, even
> as the doc does not mention it (in fact it seems that in dunfell  "2" is not doing anything
> more than "1", right ?):
>
> # LEVELS:
> #   0: no reporting
> #   1: report options that are specified, but not in the final config
> #   2: report options that are not hardware related, but set by a BSP
>

Correct.

That's again the impact of the changing opinion on what should or
shouldn't be reported. I've kept everything around (including that out
of date comment), since I just wait a few months for the next time I
need it :D

Cheers,

Bruce


>>
>> Master already has a better way to deal with this, via the 'config blame'
>> task.
>>
>> Bruce
>>
>> >
>> >>
>> >> In non-master, there's a complex way to tag options as hardware (and
>> >> hopefully an easier way in master), so if there's a mismatch you'll
>> >> see it.
>> >>
>> >> The delta between the two could be that I re-did the default
>> >> classification buckets for the new kernel and that's why it shows in
>> >> one, and not the other.
>> >>
>> >>  I can fire up a build on dunfell and see if I can see where the
>> >> classification differs.
>> >>
>> >> Bruce
>> >>
>> >> >
>> >> >>
>> >> >> The visibility of the warnings has changed over time (I expose them,
>> >> >> there are complaints, I make them less visible, and then the process
>> >> >> repeats ..), so there could be something there as well.
>> >> >>
>> >> >> And as you noticed, I totally rewrote things in master, since
>> >> >> maintaining the existing script was just too painful.
>> >> >>
>> >> >> Cheers,
>> >> >>
>> >> >> Bruce
>> >> >>
>> >> >> >
>> >> >> >>
>> >> >> >> Cheers,
>> >> >> >>
>> >> >> >> Bruce
>> >> >> >>
>> >> >> >> >      if os.path.exists(mismatch_file):
>> >> >> >> >          if config_check_visibility:
>> >> >> >> > +            mismatch_details_file = d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
>> >> >> >> >              with open (mismatch_file, "r") as myfile:
>> >> >> >> >                  results = myfile.read()
>> >> >> >> > -                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration:\n\n%s" % results)
>> >> >> >> > +                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration (see details in %s):\n\n%s" % (mismatch_details_file, results))
>> >> >> >> >
>> >> >> >> >      if bsp_check_visibility:
>> >> >> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
>> >> >> >> > --
>> >> >> >> > 2.28.0
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > 
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> >> >> thee at its end
>> >> >> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Yann Dirson <yann@blade-group.com>
>> >> >> > Blade / Shadow -- http://shadow.tech
>> >> >> >
>> >> >>
>> >> >>
>> >> >> --
>> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> >> thee at its end
>> >> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Yann Dirson <yann@blade-group.com>
>> >> > Blade / Shadow -- http://shadow.tech
>> >> >
>> >>
>> >>
>> >> --
>> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> thee at its end
>> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >
>> >
>> >
>> > --
>> > Yann Dirson <yann@blade-group.com>
>> > Blade / Shadow -- http://shadow.tech
>> >
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Yann Dirson <yann@blade-group.com>
> Blade / Shadow -- http://shadow.tech
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Yann Dirson]

[-- Attachment #1: Type: text/plain, Size: 9063 bytes --]

Le mar. 13 oct. 2020 à 17:12, Yann Dirson via lists.openembedded.org
<yann.dirson=blade-group.com@lists.openembedded.org> a écrit :

>
>
> Le mar. 13 oct. 2020 à 17:04, Bruce Ashfield <bruce.ashfield@gmail.com> a
> écrit :
>
>> On Tue, Oct 13, 2020 at 10:46 AM Yann Dirson
>> <yann.dirson@blade-group.com> wrote:
>> >
>> >
>> >
>> > Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <bruce.ashfield@gmail.com>
>> a écrit :
>> >>
>> >> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <
>> yann.dirson@blade-group.com> wrote:
>> >> >
>> >> >
>> >> >
>> >> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <
>> bruce.ashfield@gmail.com> a écrit :
>> >> >>
>> >> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <
>> yann.dirson@blade-group.com> wrote:
>> >> >> >
>> >> >> > From: Yann Dirson <yann@blade-group.com>
>> >> >> >
>> >> >> > The warning was not issued because of a filename error (looking
>> for a
>> >> >> > gatesgath-era file in older tree).
>> >> >> >
>> >> >> > Before gatesgarth we have two files (one with short list and one
>> with
>> >> >> > more details), whereas starting with gatesgarth we only have a
>> single
>> >> >> > file with all details.  Nevertheless, for consistency with the
>> >> >> > presentation of "invalid.cfg" we'll just print the contents of the
>> >> >> > short "mismatch.cfg" and point the user to the more detailed
>> >> >> > information.
>> >> >> >
>> >> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
>> >> >> > ---
>> >> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
>> >> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
>> >> >> >
>> >> >> > This is not applicable to master, where the "mismatch.txt" file
>> exists
>> >> >> > and shows even greater details.  Looks like the change was
>> introduced
>> >> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that
>> be a
>> >> >> > mismatch with yocto-kernel-tools version ?
>> >> >> >
>> >> >> > diff --git a/meta/classes/kernel-yocto.bbclass
>> b/meta/classes/kernel-yocto.bbclass
>> >> >> > index cc8bcb909a..771b4d8654 100644
>> >> >> > --- a/meta/classes/kernel-yocto.bbclass
>> >> >> > +++ b/meta/classes/kernel-yocto.bbclass
>> >> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
>> >> >> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL")
>> or 0)
>> >> >> >
>> >> >> >      # if config check visibility is non-zero, report dropped
>> configuration values
>> >> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
>> >> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
>> >> >>
>> >> >> The use of mismatch.txt is completely intentional, and it is
>> populated
>> >> >> by hardware tagged options that are detected as being mistmatches.
>> >> >>
>> >> >> So no, we don't want to directly expose mismatch.cfg. it shows
>> >> >> unclassified options.
>> >> >
>> >> >
>> >> > OK, so let's get back to the original problem:
>> >> >
>> >> > - I used to have kernel_configcheck report those problems enumerated
>> in mismatch.cfg / mismatch-all.txt,
>> >> > but for some time I'm not getting them, and this could be since
>> migration to sumo
>> >> >
>> >> > - those files are generated, and seem to contain real issues,
>> whereas there is no mismatch.txt
>> >> >
>> >> > What do you see going wrong and how do we move forward ?
>> >>
>> >> Is your configuration something that I can set up and reproduce the
>> >> problem ? I'm seeing what I expect out of the kconfig processing, but
>> >> there very well could be a bug that isn't showing in the machines that
>> >> I normally build.
>> >
>> >
>> > The problem can be triggered with the default qemu_x86-64 configuration.
>> > I'm just adding this in local.conf to inject a few options:
>> >
>> > SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"
>> >
>> > with this line in stuff.cfg:
>> >
>> > CONFIG_EEEPC_LAPTOP=y
>> >
>> > On master I see:
>> >
>> > WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0
>> do_kernel_configcheck: [kernel config]: specified values did not make it
>> into the kernel's final configuration:
>> >
>> >     [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do
>> not match
>> >     [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353
>> :configs/v5.8/standard/./hibern.cfg (y)
>> >
>> > But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed in
>> mismatch*, though it is just
>> > one of 127 issues, most of which looking pretty much valid at first
>> glance.  Are they not ?
>>
>> Only if something is tagged as "hardware" is it reported. The
>> classification name is historical, so you can read "hardware" as
>> "required / important / needed for boot". The organization of the
>> fragments (and a push and pull between stand alone fragments,
>> different configurations, software stacks, etc) means that there are
>> many fragments that have some options listed, that are disabled due to
>> competing interests. Those are for the most part valid (to be
>> different), and only differing values for the ones tagged as hardware
>> are reported.
>>
>> So no, we really don't want to report all those options. That's the
>> wobbling between reporting and non-reporting that I mentioned before.
>>
>
> Hm.  In our case, I've tried to write carefully-non-overlapping snippets,
> and I'd like to get a warning on *any* mismatch because they would point
> to a bug in my files.  Should I write an additional configcheck stage that
> would check for mismatch.cfg by itself ?  If so, what about gatesgarth ?
>

Double-checking the state of things against what I wanted to achieve
reveals that
it's not that easy in the end, esp. for our old arch based on x86-64, for
which I felt it
was not a good idea to write a defconfig and update it for every kernel
release.

For that particular (and probably common) case, it does make sense for .cfg
snippets to override stuff set by defconfig, and we would want to hide
those,
while still exposing all others.  It seems to be a bit much to ask from the
dunfell configcheck, is it something that would be easily achievable in
gatesgarth ?



>
>> In non-master, there's a complex way to tag options as hardware (and
>> hopefully an easier way in master), so if there's a mismatch you'll
>> see it.
>>
>> The delta between the two could be that I re-did the default
>> classification buckets for the new kernel and that's why it shows in
>> one, and not the other.
>>
>>  I can fire up a build on dunfell and see if I can see where the
>> classification differs.
>>
>> Bruce
>>
>> >
>> >>
>> >> The visibility of the warnings has changed over time (I expose them,
>> >> there are complaints, I make them less visible, and then the process
>> >> repeats ..), so there could be something there as well.
>> >>
>> >> And as you noticed, I totally rewrote things in master, since
>> >> maintaining the existing script was just too painful.
>> >>
>> >> Cheers,
>> >>
>> >> Bruce
>> >>
>> >> >
>> >> >>
>> >> >> Cheers,
>> >> >>
>> >> >> Bruce
>> >> >>
>> >> >> >      if os.path.exists(mismatch_file):
>> >> >> >          if config_check_visibility:
>> >> >> > +            mismatch_details_file =
>> d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
>> >> >> >              with open (mismatch_file, "r") as myfile:
>> >> >> >                  results = myfile.read()
>> >> >> > -                bb.warn( "[kernel config]: specified values did
>> not make it into the kernel's final configuration:\n\n%s" % results)
>> >> >> > +                bb.warn( "[kernel config]: specified values did
>> not make it into the kernel's final configuration (see details in
>> %s):\n\n%s" % (mismatch_details_file, results))
>> >> >> >
>> >> >> >      if bsp_check_visibility:
>> >> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" %
>> kmeta)
>> >> >> > --
>> >> >> > 2.28.0
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >>
>> >> >>
>> >> >> --
>> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness
>> await
>> >> >> thee at its end
>> >> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Yann Dirson <yann@blade-group.com>
>> >> > Blade / Shadow -- http://shadow.tech
>> >> >
>> >>
>> >>
>> >> --
>> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> >> thee at its end
>> >> - "Use the force Harry" - Gandalf, Star Trek II
>> >
>> >
>> >
>> > --
>> > Yann Dirson <yann@blade-group.com>
>> > Blade / Shadow -- http://shadow.tech
>> >
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>>
>
>
> --
> Yann Dirson <yann@blade-group.com>
> Blade / Shadow -- http://shadow.tech
>
>
> 
>
>

-- 
Yann Dirson <yann@blade-group.com>
Blade / Shadow -- http://shadow.tech

[-- Attachment #2: Type: text/html, Size: 13148 bytes --]

Re: [OE-core] [dunfell][PATCH] kernel-yocto.bbclass: fix reporting of config mismatches

[Bruce Ashfield]

On Tue, Oct 13, 2020 at 12:14 PM Yann Dirson
<yann.dirson@blade-group.com> wrote:

>
>
>
> Le mar. 13 oct. 2020 à 17:12, Yann Dirson via lists.openembedded.org <yann.dirson=blade-group.com@lists.openembedded.org> a écrit :
>>
>>
>>
>> Le mar. 13 oct. 2020 à 17:04, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>>
>>> On Tue, Oct 13, 2020 at 10:46 AM Yann Dirson
>>> <yann.dirson@blade-group.com> wrote:
>>> >
>>> >
>>> >
>>> > Le mar. 13 oct. 2020 à 16:10, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>> >>
>>> >> On Tue, Oct 13, 2020 at 9:57 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>>> >> >
>>> >> >
>>> >> >
>>> >> > Le mar. 13 oct. 2020 à 15:34, Bruce Ashfield <bruce.ashfield@gmail.com> a écrit :
>>> >> >>
>>> >> >> On Tue, Oct 13, 2020 at 9:21 AM Yann Dirson <yann.dirson@blade-group.com> wrote:
>>> >> >> >
>>> >> >> > From: Yann Dirson <yann@blade-group.com>
>>> >> >> >
>>> >> >> > The warning was not issued because of a filename error (looking for a
>>> >> >> > gatesgath-era file in older tree).
>>> >> >> >
>>> >> >> > Before gatesgarth we have two files (one with short list and one with
>>> >> >> > more details), whereas starting with gatesgarth we only have a single
>>> >> >> > file with all details.  Nevertheless, for consistency with the
>>> >> >> > presentation of "invalid.cfg" we'll just print the contents of the
>>> >> >> > short "mismatch.cfg" and point the user to the more detailed
>>> >> >> > information.
>>> >> >> >
>>> >> >> > Signed-off-by: Yann Dirson <yann@blade-group.com>
>>> >> >> > ---
>>> >> >> >  meta/classes/kernel-yocto.bbclass | 5 +++--
>>> >> >> >  1 file changed, 3 insertions(+), 2 deletions(-)
>>> >> >> >
>>> >> >> > This is not applicable to master, where the "mismatch.txt" file exists
>>> >> >> > and shows even greater details.  Looks like the change was introduced
>>> >> >> > for sumo in 5ef0620b18606fd5749b6e7811d337f26075b86b - would that be a
>>> >> >> > mismatch with yocto-kernel-tools version ?
>>> >> >> >
>>> >> >> > diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
>>> >> >> > index cc8bcb909a..771b4d8654 100644
>>> >> >> > --- a/meta/classes/kernel-yocto.bbclass
>>> >> >> > +++ b/meta/classes/kernel-yocto.bbclass
>>> >> >> > @@ -431,12 +431,13 @@ python do_kernel_configcheck() {
>>> >> >> >      bsp_check_visibility = int(d.getVar("KCONF_BSP_AUDIT_LEVEL") or 0)
>>> >> >> >
>>> >> >> >      # if config check visibility is non-zero, report dropped configuration values
>>> >> >> > -    mismatch_file = d.expand("${S}/%s/cfg/mismatch.txt" % kmeta)
>>> >> >> > +    mismatch_file = d.expand("${S}/%s/cfg/mismatch.cfg" % kmeta)
>>> >> >>
>>> >> >> The use of mismatch.txt is completely intentional, and it is populated
>>> >> >> by hardware tagged options that are detected as being mistmatches.
>>> >> >>
>>> >> >> So no, we don't want to directly expose mismatch.cfg. it shows
>>> >> >> unclassified options.
>>> >> >
>>> >> >
>>> >> > OK, so let's get back to the original problem:
>>> >> >
>>> >> > - I used to have kernel_configcheck report those problems enumerated in mismatch.cfg / mismatch-all.txt,
>>> >> > but for some time I'm not getting them, and this could be since migration to sumo
>>> >> >
>>> >> > - those files are generated, and seem to contain real issues, whereas there is no mismatch.txt
>>> >> >
>>> >> > What do you see going wrong and how do we move forward ?
>>> >>
>>> >> Is your configuration something that I can set up and reproduce the
>>> >> problem ? I'm seeing what I expect out of the kconfig processing, but
>>> >> there very well could be a bug that isn't showing in the machines that
>>> >> I normally build.
>>> >
>>> >
>>> > The problem can be triggered with the default qemu_x86-64 configuration.
>>> > I'm just adding this in local.conf to inject a few options:
>>> >
>>> > SRC_URI_append_pn-linux-yocto += " file://stuff.cfg"
>>> >
>>> > with this line in stuff.cfg:
>>> >
>>> > CONFIG_EEEPC_LAPTOP=y
>>> >
>>> > On master I see:
>>> >
>>> > WARNING: linux-yocto-5.8.9+gitAUTOINC+ffbfe61a19_31fafe701e-r0 do_kernel_configcheck: [kernel config]: specified values did not make it into the kernel's final configuration:
>>> >
>>> >     [NOTE]: 'CONFIG_EEEPC_LAPTOP' last val (y) and .config val (m) do not match
>>> >     [INFO]: CONFIG_EEEPC_LAPTOP : m ## .config: 4353 :configs/v5.8/standard/./hibern.cfg (y)
>>> >
>>> > But on dunfell I don't see anything.  CONFIG_EEEPC_LAPTOP is listed in mismatch*, though it is just
>>> > one of 127 issues, most of which looking pretty much valid at first glance.  Are they not ?
>>>
>>> Only if something is tagged as "hardware" is it reported. The
>>> classification name is historical, so you can read "hardware" as
>>> "required / important / needed for boot". The organization of the
>>> fragments (and a push and pull between stand alone fragments,
>>> different configurations, software stacks, etc) means that there are
>>> many fragments that have some options listed, that are disabled due to
>>> competing interests. Those are for the most part valid (to be
>>> different), and only differing values for the ones tagged as hardware
>>> are reported.
>>>
>>> So no, we really don't want to report all those options. That's the
>>> wobbling between reporting and non-reporting that I mentioned before.
>>
>>
>> Hm.  In our case, I've tried to write carefully-non-overlapping snippets,
>> and I'd like to get a warning on *any* mismatch because they would point
>> to a bug in my files.  Should I write an additional configcheck stage that
>> would check for mismatch.cfg by itself ?  If so, what about gatesgarth ?
>
>
> Double-checking the state of things against what I wanted to achieve reveals that
> it's not that easy in the end, esp. for our old arch based on x86-64, for which I felt it
> was not a good idea to write a defconfig and update it for every kernel release.
>
> For that particular (and probably common) case, it does make sense for .cfg
> snippets to override stuff set by defconfig, and we would want to hide those,
> while still exposing all others.  It seems to be a bit much to ask from the
> dunfell configcheck, is it something that would be easily achievable in gatesgarth ?
>

It is a common scenario. You have a baseline config (whether it be a defconfig,
or another "base.cfg" fragment) and then features / optional fragments
are applied
and then BSP options (or whatever order you want). The fragments very often do
override / change the values, or you'd be spending a lot of time locking in a
lowest common denominator base configuration (which is doable, but it tends to
be fairly small).

It would be easier to add that level of granularity into the check in
master (I used
the reverse of it to mask some warnings that were spurious), I'm not
sure it would
work out of the box, but with relatively little tweaking, it would be doable.

Bruce


>
>>
>>>
>>> In non-master, there's a complex way to tag options as hardware (and
>>> hopefully an easier way in master), so if there's a mismatch you'll
>>> see it.
>>>
>>> The delta between the two could be that I re-did the default
>>> classification buckets for the new kernel and that's why it shows in
>>> one, and not the other.
>>>
>>>  I can fire up a build on dunfell and see if I can see where the
>>> classification differs.
>>>
>>> Bruce
>>>
>>> >
>>> >>
>>> >> The visibility of the warnings has changed over time (I expose them,
>>> >> there are complaints, I make them less visible, and then the process
>>> >> repeats ..), so there could be something there as well.
>>> >>
>>> >> And as you noticed, I totally rewrote things in master, since
>>> >> maintaining the existing script was just too painful.
>>> >>
>>> >> Cheers,
>>> >>
>>> >> Bruce
>>> >>
>>> >> >
>>> >> >>
>>> >> >> Cheers,
>>> >> >>
>>> >> >> Bruce
>>> >> >>
>>> >> >> >      if os.path.exists(mismatch_file):
>>> >> >> >          if config_check_visibility:
>>> >> >> > +            mismatch_details_file = d.expand("${S}/%s/cfg/mismatch-all.txt" % kmeta)
>>> >> >> >              with open (mismatch_file, "r") as myfile:
>>> >> >> >                  results = myfile.read()
>>> >> >> > -                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration:\n\n%s" % results)
>>> >> >> > +                bb.warn( "[kernel config]: specified values did not make it into the kernel's final configuration (see details in %s):\n\n%s" % (mismatch_details_file, results))
>>> >> >> >
>>> >> >> >      if bsp_check_visibility:
>>> >> >> >          invalid_file = d.expand("${S}/%s/cfg/invalid.cfg" % kmeta)
>>> >> >> > --
>>> >> >> > 2.28.0
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>>> >> >> thee at its end
>>> >> >> - "Use the force Harry" - Gandalf, Star Trek II
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Yann Dirson <yann@blade-group.com>
>>> >> > Blade / Shadow -- http://shadow.tech
>>> >> >
>>> >>
>>> >>
>>> >> --
>>> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>>> >> thee at its end
>>> >> - "Use the force Harry" - Gandalf, Star Trek II
>>> >
>>> >
>>> >
>>> > --
>>> > Yann Dirson <yann@blade-group.com>
>>> > Blade / Shadow -- http://shadow.tech
>>> >
>>>
>>>
>>> --
>>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>>> thee at its end
>>> - "Use the force Harry" - Gandalf, Star Trek II
>>
>>
>>
>> --
>> Yann Dirson <yann@blade-group.com>
>> Blade / Shadow -- http://shadow.tech
>>
>>
>> 
>>
>
>
> --
> Yann Dirson <yann@blade-group.com>
> Blade / Shadow -- http://shadow.tech
>


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II