[oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Max Krummenacher]

I am trying out a kernel recipe using linux master, 5.16-rc2. I found
that perf fails to build in do_compile with such a setup.
Root cause is that the kernel did add 'static' and 'const' to the
arrays of structs which sort-pmuevents does sort.
This is addressed in the second commit.

While at it I noticed that there are some struct arrays which neither
have a name or cpuid element with 1 array element. That 1 element was
dropped. I did not verify if that causes any issues at runtime, however
that behaviour was improved with the first commit.

Note that the kernel itself doesn't build either as after 5.15 it now
does try to run the dt_binding_check for which the kernel recipe does
not provide whatever stuff is needed.
I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
DT schema checks for %.dtb targets") to get over that.

Max Krummenacher (2):
  perf: sort-pmuevents: don't drop elements
  perf: sort-pmuevents: allow for additional type qualifiers and storage
    class

 .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
 1 file changed, 16 insertions(+), 12 deletions(-)

-- 
2.20.1

[oe][OE-core][Patch 1/2] perf: sort-pmuevents: don't drop elements

[Max Krummenacher]

If a struct element neither has an element cpuid or name it gets silenty
dropped.
Kernel 5.15 for some ARCHs have at least one array of structs matching
this.
e.g. for arm pmu-events.c:

|#include "pmu-events/pmu-events.h"
struct pmu_events_map pmu_events_map[] = {
{
	.cpuid = 0,
	.version = 0,
	.type = 0,
	.table = 0,
},
};

struct pmu_sys_events pmu_sys_event_tables[] = {
	{
		.table = 0
	},
};

Before this patch the second struct is translated to an empty array::

struct pmu_sys_events pmu_sys_event_tables[] = {
};

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
---
 meta/recipes-kernel/perf/perf/sort-pmuevents.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
index 5ddf0f144f..4f841eb822 100755
--- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
+++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
@@ -61,6 +61,8 @@ for struct in re.findall( struct_block_regex, data ):
             #print( "    name found: %s" % name.group(1) )
             entry_dict[struct[1]]['fields'][name.group(1)] = entry
         
+        if not entry_dict[struct[1]]['fields']:
+            entry_dict[struct[1]]['fields']['0'] = entry
 
 # created ordered dictionaries from the captured values. These are ordered by
 # a sorted() iteration of the keys. We don't care about the order we read
-- 
2.20.1

[oe][OE-core][Patch 2/2] perf: sort-pmuevents: allow for additional type qualifiers and storage class

[Max Krummenacher]

With kernel 5.16 some structs in pmu-events do get a const qualifier, some
a static const storage class and qualifier.

The current sort-pmuevents cannot cope with that and drops all struct
arrays with such additional elements. This then leads to compiler errors.

Allow '^struct', '^const struct', '^static struct', '^static const struct'.

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
---
 .../perf/perf/sort-pmuevents.py               | 30 ++++++++++---------
 1 file changed, 16 insertions(+), 14 deletions(-)

diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
index 4f841eb822..09ba3328a7 100755
--- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
+++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
@@ -33,10 +33,10 @@ if os.path.exists(outfile):
 with open(infile, 'r') as file:
     data = file.read()
 
-preamble_regex = re.compile( '^(.*?)^struct', re.MULTILINE | re.DOTALL )
+preamble_regex = re.compile( '^(.*?)^(struct|const struct|static struct|static const struct)', re.MULTILINE | re.DOTALL )
 
 preamble = re.search( preamble_regex, data )
-struct_block_regex = re.compile( '^struct.*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
+struct_block_regex = re.compile( '^(struct|const struct|static struct|static const struct).*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
 field_regex =  re.compile( '{.*?},', re.MULTILINE | re.DOTALL )
 cpuid_regex = re.compile( '\.cpuid = (.*?),', re.MULTILINE | re.DOTALL )
 name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
@@ -45,24 +45,25 @@ name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
 # types and then their fields.
 entry_dict = {}
 for struct in re.findall( struct_block_regex, data ):
-    # print( "struct: %s %s" % (struct[0],struct[1]) )
-    entry_dict[struct[1]] = {}
-    entry_dict[struct[1]]['type'] = struct[0]
-    entry_dict[struct[1]]['fields'] = {}
-    for entry in re.findall( field_regex, struct[2] ):
+    # print( "struct: %s %s %s" % (struct[0],struct[1],struct[2]) )
+    entry_dict[struct[2]] = {}
+    entry_dict[struct[2]]['type_prefix'] = struct[0]
+    entry_dict[struct[2]]['type'] = struct[1]
+    entry_dict[struct[2]]['fields'] = {}
+    for entry in re.findall( field_regex, struct[3] ):
         #print( "    entry: %s" % entry )
         cpuid = re.search( cpuid_regex, entry )
         if cpuid:
             #print( "    cpuid found: %s" % cpuid.group(1) )
-            entry_dict[struct[1]]['fields'][cpuid.group(1)] = entry
-            
+            entry_dict[struct[2]]['fields'][cpuid.group(1)] = entry
+
         name = re.search( name_regex, entry )
         if name:
             #print( "    name found: %s" % name.group(1) )
-            entry_dict[struct[1]]['fields'][name.group(1)] = entry
-        
-        if not entry_dict[struct[1]]['fields']:
-            entry_dict[struct[1]]['fields']['0'] = entry
+            entry_dict[struct[2]]['fields'][name.group(1)] = entry
+
+        if not entry_dict[struct[2]]['fields']:
+            entry_dict[struct[2]]['fields']['0'] = entry
 
 # created ordered dictionaries from the captured values. These are ordered by
 # a sorted() iteration of the keys. We don't care about the order we read
@@ -74,6 +75,7 @@ for struct in re.findall( struct_block_regex, data ):
 entry_dict_sorted = OrderedDict()
 for i in sorted(entry_dict.keys()):
     entry_dict_sorted[i] = {}
+    entry_dict_sorted[i]['type_prefix'] = entry_dict[i]['type_prefix']
     entry_dict_sorted[i]['type'] = entry_dict[i]['type']
     entry_dict_sorted[i]['fields'] = {}
     for f in sorted(entry_dict[i]['fields'].keys()):
@@ -85,7 +87,7 @@ outf = open( outfile, 'w' )
 print( preamble.group(1) )
 outf.write( preamble.group(1) )
 for d in entry_dict_sorted:
-    outf.write( "struct %s %s[] = {\n" % (entry_dict_sorted[d]['type'],d) )
+    outf.write( "%s %s %s[] = {\n" % (entry_dict_sorted[d]['type_prefix'], entry_dict_sorted[d]['type'],d) )
     for f in entry_dict_sorted[d]['fields']:
         outf.write( entry_dict_sorted[d]['fields'][f] + '\n' )
 
-- 
2.20.1

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Bruce Ashfield]

On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
>
> I am trying out a kernel recipe using linux master, 5.16-rc2. I found
> that perf fails to build in do_compile with such a setup.
> Root cause is that the kernel did add 'static' and 'const' to the
> arrays of structs which sort-pmuevents does sort.
> This is addressed in the second commit.
>
> While at it I noticed that there are some struct arrays which neither
> have a name or cpuid element with 1 array element. That 1 element was
> dropped. I did not verify if that causes any issues at runtime, however
> that behaviour was improved with the first commit.
>
> Note that the kernel itself doesn't build either as after 5.15 it now
> does try to run the dt_binding_check for which the kernel recipe does
> not provide whatever stuff is needed.
> I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> DT schema checks for %.dtb targets") to get over that.

I have a fixefor that particular problem (the bindings),  so you can
safely continue to ignore it .. but I hadn't gotten around to fixing
perf yet.

The changes look fine to me for perf. The sorting of events is
hopefully something we can drop soon, as there have been other
reproducibility changes to perf upstream, that will make it
unnecessary.

Bruce

>
> Max Krummenacher (2):
>   perf: sort-pmuevents: don't drop elements
>   perf: sort-pmuevents: allow for additional type qualifiers and storage
>     class
>
>  .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
>  1 file changed, 16 insertions(+), 12 deletions(-)
>
> --
> 2.20.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Daniel Díaz]

Hello!

On Mon, 22 Nov 2021 at 10:51, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
[...]
> > Note that the kernel itself doesn't build either as after 5.15 it now
> > does try to run the dt_binding_check for which the kernel recipe does
> > not provide whatever stuff is needed.
> > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > DT schema checks for %.dtb targets") to get over that.
>
> I have a fixefor that particular problem (the bindings),  so you can
> safely continue to ignore it .. but I hadn't gotten around to fixing
> perf yet.
[...]

I don't mean to hijack the subject, but do you have some quick
pointers on that? We're seeing the same on linux-next and
linux-mainline since that patch landed in those trees, and have
soft-reverted the commit for now. This (incomplete) recipe has not
helped us get through that:

  https://github.com/mrchapp/meta-lkft/blob/d/add-dtschema/recipes-devtools/python/dtschema_2021.10.bb

Thanks and greetings!

Daniel Díaz
daniel.diaz@linaro.org

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Bruce Ashfield]

On Mon, Nov 22, 2021 at 12:23 PM Daniel Díaz <daniel.diaz@linaro.org> wrote:
>
> Hello!
>
> On Mon, 22 Nov 2021 at 10:51, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> [...]
> > > Note that the kernel itself doesn't build either as after 5.15 it now
> > > does try to run the dt_binding_check for which the kernel recipe does
> > > not provide whatever stuff is needed.
> > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > > DT schema checks for %.dtb targets") to get over that.
> >
> > I have a fixefor that particular problem (the bindings),  so you can
> > safely continue to ignore it .. but I hadn't gotten around to fixing
> > perf yet.
> [...]
>
> I don't mean to hijack the subject, but do you have some quick
> pointers on that? We're seeing the same on linux-next and
> linux-mainline since that patch landed in those trees, and have

I'm trying to come up with a way to fix this without needing a kernel patch.

Sounds like you are already aware of the pkgconfig issue, which is
what I'm attempting to fix via environment manipulation versus a
patch.

We do need python dtschema as well, as a new native dependency of the
bindings check, which unfortunately pulls in ruamel and jsonschema.
Those are both in meta-python, so they need to migrate to core to
support the new checking.

There's also a problem with nproc being used, and it was giving me an
error on one of my servers, I haven't looked into it more yet.

I have a quick recipe here as well. And with those two pieces in
place, I'm up and running again.

The right thing for the short term, is that revert. There's quite a
bit of sorting out to do, to get this in decent shape for core ...
personally, I'd like to make the checking optional for those that
don't want it .. but that goes against what Rob is trying to do in the
kernel :D

If those issues sound familiar, and there's interest, I can drop my
WIP commits into a branch and share.

Bruce

> soft-reverted the commit for now. This (incomplete) recipe has not
> helped us get through that:
>
>   https://github.com/mrchapp/meta-lkft/blob/d/add-dtschema/recipes-devtools/python/dtschema_2021.10.bb
>
> Thanks and greetings!
>
> Daniel Díaz
> daniel.diaz@linaro.org


--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Bruce Ashfield]

On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
>
> I am trying out a kernel recipe using linux master, 5.16-rc2. I found
> that perf fails to build in do_compile with such a setup.
> Root cause is that the kernel did add 'static' and 'const' to the
> arrays of structs which sort-pmuevents does sort.
> This is addressed in the second commit.
>
> While at it I noticed that there are some struct arrays which neither
> have a name or cpuid element with 1 array element. That 1 element was
> dropped. I did not verify if that causes any issues at runtime, however
> that behaviour was improved with the first commit.

What MACHINE were you testing against ? When I build perf against my
-rc2 dev kernel, I still run into the build issues with these commits
in place.

I just want to make sure that I'm not seeing something different .. so
I can do some runs against a similar config as you are using.

Bruce

>
> Note that the kernel itself doesn't build either as after 5.15 it now
> does try to run the dt_binding_check for which the kernel recipe does
> not provide whatever stuff is needed.
> I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> DT schema checks for %.dtb targets") to get over that.
>
> Max Krummenacher (2):
>   perf: sort-pmuevents: don't drop elements
>   perf: sort-pmuevents: allow for additional type qualifiers and storage
>     class
>
>  .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
>  1 file changed, 16 insertions(+), 12 deletions(-)
>
> --
> 2.20.1
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Bruce Ashfield]

On Mon, Nov 22, 2021 at 8:39 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Mon, Nov 22, 2021 at 12:23 PM Daniel Díaz <daniel.diaz@linaro.org> wrote:
> >
> > Hello!
> >
> > On Mon, 22 Nov 2021 at 10:51, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> > [...]
> > > > Note that the kernel itself doesn't build either as after 5.15 it now
> > > > does try to run the dt_binding_check for which the kernel recipe does
> > > > not provide whatever stuff is needed.
> > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > > > DT schema checks for %.dtb targets") to get over that.
> > >
> > > I have a fixefor that particular problem (the bindings),  so you can
> > > safely continue to ignore it .. but I hadn't gotten around to fixing
> > > perf yet.
> > [...]
> >
> > I don't mean to hijack the subject, but do you have some quick
> > pointers on that? We're seeing the same on linux-next and
> > linux-mainline since that patch landed in those trees, and have
>
> I'm trying to come up with a way to fix this without needing a kernel patch.
>

FYI: I have this sorted out now.

The commits to get 5.16 building with device tree validation are here:

https://github.com/zeddii/openembedded-core/tree/zedd/kernel

(I would have pushed it to my poky-contrib zedd/kernel branch, but
there's something broken on the machine tonight).

You'll need to add the DEPENDS to your kernel recipe, like I did for
linux-yocto-dev, but outside of that the pkg-config tweaking that I
did avoided the need for kernel patches. The dtschema recipe + the
imported meta-python recipes should "just work". I haven't done a
second pass to make sure I found all the dependencies, but it is close
.. and if you have meta-python in your layers, you won't have any
issues.

For now, the DEPENDS are recipe specific, since they are fairly heavy,
and the mapping of PV -> ARCH -> conditional dependencies is still a
WIP (it gets brittle/complex very quickly) .. but moving the DEPENDS
into kernel.bbclass is something that I'm working on as well.

I still need more cleanup, but I expect that I can submit the queue by
the end of the week.

.. and the proof:

qemuarm64 login: root
uroot@qemuarm64:~# uname -a
Linux qemuarm64 5.16.0-rc2-yoctodev-standard #1 SMP PREEMPT Sun Nov 21
21:47:39 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
root@qemuarm64:~#

Cheers,

Bruce


> Sounds like you are already aware of the pkgconfig issue, which is
> what I'm attempting to fix via environment manipulation versus a
> patch.
>
> We do need python dtschema as well, as a new native dependency of the
> bindings check, which unfortunately pulls in ruamel and jsonschema.
> Those are both in meta-python, so they need to migrate to core to
> support the new checking.
>
> There's also a problem with nproc being used, and it was giving me an
> error on one of my servers, I haven't looked into it more yet.
>
> I have a quick recipe here as well. And with those two pieces in
> place, I'm up and running again.
>
> The right thing for the short term, is that revert. There's quite a
> bit of sorting out to do, to get this in decent shape for core ...
> personally, I'd like to make the checking optional for those that
> don't want it .. but that goes against what Rob is trying to do in the
> kernel :D
>
> If those issues sound familiar, and there's interest, I can drop my
> WIP commits into a branch and share.
>
> Bruce
>
> > soft-reverted the commit for now. This (incomplete) recipe has not
> > helped us get through that:
> >
> >   https://github.com/mrchapp/meta-lkft/blob/d/add-dtschema/recipes-devtools/python/dtschema_2021.10.bb
> >
> > Thanks and greetings!
> >
> > Daniel Díaz
> > daniel.diaz@linaro.org
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II



--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Max Krummenacher]

Hi Bruce,

On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> >
> > I am trying out a kernel recipe using linux master, 5.16-rc2. I found
> > that perf fails to build in do_compile with such a setup.
> > Root cause is that the kernel did add 'static' and 'const' to the
> > arrays of structs which sort-pmuevents does sort.
> > This is addressed in the second commit.
> >
> > While at it I noticed that there are some struct arrays which neither
> > have a name or cpuid element with 1 array element. That 1 element was
> > dropped. I did not verify if that causes any issues at runtime, however
> > that behaviour was improved with the first commit.
>
> What MACHINE were you testing against ? When I build perf against my
> -rc2 dev kernel, I still run into the build issues with these commits
> in place.
>
> I just want to make sure that I'm not seeing something different .. so
> I can do some runs against a similar config as you are using.
>

I did the inital work against a Toradex machine, apalis-imx6.
However I built before sending the patches with qemuarm64 qemuarm qemux86-64
with the following modification to use my 'master' kernel recipe:
+PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline"
+KBUILD_DEFCONFIG:arm = "tegra_defconfig"
+KBUILD_DEFCONFIG:aarch64 = "defconfig"
+KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig"

I tried again. With the patches perf builds for all three machines,
reverting the patches and it fails again.
The x86 compiles forever, for the two ARM machines I get:
...
Traceback (most recent call last):
  File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py",
line 83, in <module>
    print( preamble.group(1) )
AttributeError: 'NoneType' object has no attribute 'group'
...

Max

>
> >
> > Note that the kernel itself doesn't build either as after 5.15 it now
> > does try to run the dt_binding_check for which the kernel recipe does
> > not provide whatever stuff is needed.
> > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > DT schema checks for %.dtb targets") to get over that.
> >
> > Max Krummenacher (2):
> >   perf: sort-pmuevents: don't drop elements
> >   perf: sort-pmuevents: allow for additional type qualifiers and storage
> >     class
> >
> >  .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
> >  1 file changed, 16 insertions(+), 12 deletions(-)
> >
> > --
> > 2.20.1
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Bruce Ashfield]

On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
>
> Hi Bruce,
>
> On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> > >
> > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found
> > > that perf fails to build in do_compile with such a setup.
> > > Root cause is that the kernel did add 'static' and 'const' to the
> > > arrays of structs which sort-pmuevents does sort.
> > > This is addressed in the second commit.
> > >
> > > While at it I noticed that there are some struct arrays which neither
> > > have a name or cpuid element with 1 array element. That 1 element was
> > > dropped. I did not verify if that causes any issues at runtime, however
> > > that behaviour was improved with the first commit.
> >
> > What MACHINE were you testing against ? When I build perf against my
> > -rc2 dev kernel, I still run into the build issues with these commits
> > in place.
> >
> > I just want to make sure that I'm not seeing something different .. so
> > I can do some runs against a similar config as you are using.
> >
>
> I did the inital work against a Toradex machine, apalis-imx6.
> However I built before sending the patches with qemuarm64 qemuarm qemux86-64
> with the following modification to use my 'master' kernel recipe:
> +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline"
> +KBUILD_DEFCONFIG:arm = "tegra_defconfig"
> +KBUILD_DEFCONFIG:aarch64 = "defconfig"
> +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig"
>
> I tried again. With the patches perf builds for all three machines,
> reverting the patches and it fails again.
> The x86 compiles forever, for the two ARM machines I get:
> ...
> Traceback (most recent call last):
>   File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py",
> line 83, in <module>
>     print( preamble.group(1) )
> AttributeError: 'NoneType' object has no attribute 'group'

Definitely still fails here for some of my configurations. I'll look
into it more today, and see if there's something in my reference
kernel configurations that are causing it.

I need to sort this out, and can put it in a queue with my -dev
upgrades to 5.16, since without those version bumps, we have nothing
in core that exercises the change.

Cheers,

Bruce

> ...
>
> Max
>
> >
> > >
> > > Note that the kernel itself doesn't build either as after 5.15 it now
> > > does try to run the dt_binding_check for which the kernel recipe does
> > > not provide whatever stuff is needed.
> > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > > DT schema checks for %.dtb targets") to get over that.
> > >
> > > Max Krummenacher (2):
> > >   perf: sort-pmuevents: don't drop elements
> > >   perf: sort-pmuevents: allow for additional type qualifiers and storage
> > >     class
> > >
> > >  .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
> > >  1 file changed, 16 insertions(+), 12 deletions(-)
> > >
> > > --
> > > 2.20.1
> > >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Max Krummenacher]

Let me know if there is anything I can do/provide from my side.
I know you already did, but does 'bitbake perf -c cleansstate' help?

Max

On Tue, Nov 23, 2021 at 2:55 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
> On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> >
> > Hi Bruce,
> >
> > On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > >
> > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> > > >
> > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found
> > > > that perf fails to build in do_compile with such a setup.
> > > > Root cause is that the kernel did add 'static' and 'const' to the
> > > > arrays of structs which sort-pmuevents does sort.
> > > > This is addressed in the second commit.
> > > >
> > > > While at it I noticed that there are some struct arrays which neither
> > > > have a name or cpuid element with 1 array element. That 1 element was
> > > > dropped. I did not verify if that causes any issues at runtime, however
> > > > that behaviour was improved with the first commit.
> > >
> > > What MACHINE were you testing against ? When I build perf against my
> > > -rc2 dev kernel, I still run into the build issues with these commits
> > > in place.
> > >
> > > I just want to make sure that I'm not seeing something different .. so
> > > I can do some runs against a similar config as you are using.
> > >
> >
> > I did the inital work against a Toradex machine, apalis-imx6.
> > However I built before sending the patches with qemuarm64 qemuarm qemux86-64
> > with the following modification to use my 'master' kernel recipe:
> > +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline"
> > +KBUILD_DEFCONFIG:arm = "tegra_defconfig"
> > +KBUILD_DEFCONFIG:aarch64 = "defconfig"
> > +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig"
> >
> > I tried again. With the patches perf builds for all three machines,
> > reverting the patches and it fails again.
> > The x86 compiles forever, for the two ARM machines I get:
> > ...
> > Traceback (most recent call last):
> >   File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py",
> > line 83, in <module>
> >     print( preamble.group(1) )
> > AttributeError: 'NoneType' object has no attribute 'group'
>
> Definitely still fails here for some of my configurations. I'll look
> into it more today, and see if there's something in my reference
> kernel configurations that are causing it.
>
> I need to sort this out, and can put it in a queue with my -dev
> upgrades to 5.16, since without those version bumps, we have nothing
> in core that exercises the change.
>
> Cheers,
>
> Bruce
>
> > ...
> >
> > Max
> >
> > >
> > > >
> > > > Note that the kernel itself doesn't build either as after 5.15 it now
> > > > does try to run the dt_binding_check for which the kernel recipe does
> > > > not provide whatever stuff is needed.
> > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > > > DT schema checks for %.dtb targets") to get over that.
> > > >
> > > > Max Krummenacher (2):
> > > >   perf: sort-pmuevents: don't drop elements
> > > >   perf: sort-pmuevents: allow for additional type qualifiers and storage
> > > >     class
> > > >
> > > >  .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
> > > >  1 file changed, 16 insertions(+), 12 deletions(-)
> > > >
> > > > --
> > > > 2.20.1
> > > >
> > >
> > >
> > > --
> > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > thee at its end
> > > - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Bruce Ashfield]

On Tue, Nov 23, 2021 at 8:58 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
>
> Let me know if there is anything I can do/provide from my side.
> I know you already did, but does 'bitbake perf -c cleansstate' help?

The builds were clean, but to rule out old artifacts, I just started a
new build on a separate server, without my normal sstate sharing
configuration. Fingers crossed :D

Bruce

>
> Max
>
> On Tue, Nov 23, 2021 at 2:55 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> > On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> > >
> > > Hi Bruce,
> > >
> > > On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > > >
> > > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> > > > >
> > > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found
> > > > > that perf fails to build in do_compile with such a setup.
> > > > > Root cause is that the kernel did add 'static' and 'const' to the
> > > > > arrays of structs which sort-pmuevents does sort.
> > > > > This is addressed in the second commit.
> > > > >
> > > > > While at it I noticed that there are some struct arrays which neither
> > > > > have a name or cpuid element with 1 array element. That 1 element was
> > > > > dropped. I did not verify if that causes any issues at runtime, however
> > > > > that behaviour was improved with the first commit.
> > > >
> > > > What MACHINE were you testing against ? When I build perf against my
> > > > -rc2 dev kernel, I still run into the build issues with these commits
> > > > in place.
> > > >
> > > > I just want to make sure that I'm not seeing something different .. so
> > > > I can do some runs against a similar config as you are using.
> > > >
> > >
> > > I did the inital work against a Toradex machine, apalis-imx6.
> > > However I built before sending the patches with qemuarm64 qemuarm qemux86-64
> > > with the following modification to use my 'master' kernel recipe:
> > > +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline"
> > > +KBUILD_DEFCONFIG:arm = "tegra_defconfig"
> > > +KBUILD_DEFCONFIG:aarch64 = "defconfig"
> > > +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig"
> > >
> > > I tried again. With the patches perf builds for all three machines,
> > > reverting the patches and it fails again.
> > > The x86 compiles forever, for the two ARM machines I get:
> > > ...
> > > Traceback (most recent call last):
> > >   File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py",
> > > line 83, in <module>
> > >     print( preamble.group(1) )
> > > AttributeError: 'NoneType' object has no attribute 'group'
> >
> > Definitely still fails here for some of my configurations. I'll look
> > into it more today, and see if there's something in my reference
> > kernel configurations that are causing it.
> >
> > I need to sort this out, and can put it in a queue with my -dev
> > upgrades to 5.16, since without those version bumps, we have nothing
> > in core that exercises the change.
> >
> > Cheers,
> >
> > Bruce
> >
> > > ...
> > >
> > > Max
> > >
> > > >
> > > > >
> > > > > Note that the kernel itself doesn't build either as after 5.15 it now
> > > > > does try to run the dt_binding_check for which the kernel recipe does
> > > > > not provide whatever stuff is needed.
> > > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > > > > DT schema checks for %.dtb targets") to get over that.
> > > > >
> > > > > Max Krummenacher (2):
> > > > >   perf: sort-pmuevents: don't drop elements
> > > > >   perf: sort-pmuevents: allow for additional type qualifiers and storage
> > > > >     class
> > > > >
> > > > >  .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
> > > > >  1 file changed, 16 insertions(+), 12 deletions(-)
> > > > >
> > > > > --
> > > > > 2.20.1
> > > > >
> > > >
> > > >
> > > > --
> > > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > > thee at its end
> > > > - "Use the force Harry" - Gandalf, Star Trek II
> >
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes

[Bruce Ashfield]

On Tue, Nov 23, 2021 at 9:05 AM Bruce Ashfield via
lists.openembedded.org
<bruce.ashfield=gmail.com@lists.openembedded.org> wrote:
>
> On Tue, Nov 23, 2021 at 8:58 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> >
> > Let me know if there is anything I can do/provide from my side.
> > I know you already did, but does 'bitbake perf -c cleansstate' help?
>
> The builds were clean, but to rule out old artifacts, I just started a
> new build on a separate server, without my normal sstate sharing
> configuration. Fingers crossed :D

It is building on my completely fresh box. Which does imply I was
getting the old script before .. but I can't say for sure why.

Regardless, I'm sending a consolidated pull request shortly with some
v5.16 fixes, and have these stacked on top, just to confirm they are
working in my setup and that I've tested them.

Bruce

>
> Bruce
>
> >
> > Max
> >
> > On Tue, Nov 23, 2021 at 2:55 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > >
> > > On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> > > >
> > > > Hi Bruce,
> > > >
> > > > On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> > > > >
> > > > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote:
> > > > > >
> > > > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found
> > > > > > that perf fails to build in do_compile with such a setup.
> > > > > > Root cause is that the kernel did add 'static' and 'const' to the
> > > > > > arrays of structs which sort-pmuevents does sort.
> > > > > > This is addressed in the second commit.
> > > > > >
> > > > > > While at it I noticed that there are some struct arrays which neither
> > > > > > have a name or cpuid element with 1 array element. That 1 element was
> > > > > > dropped. I did not verify if that causes any issues at runtime, however
> > > > > > that behaviour was improved with the first commit.
> > > > >
> > > > > What MACHINE were you testing against ? When I build perf against my
> > > > > -rc2 dev kernel, I still run into the build issues with these commits
> > > > > in place.
> > > > >
> > > > > I just want to make sure that I'm not seeing something different .. so
> > > > > I can do some runs against a similar config as you are using.
> > > > >
> > > >
> > > > I did the inital work against a Toradex machine, apalis-imx6.
> > > > However I built before sending the patches with qemuarm64 qemuarm qemux86-64
> > > > with the following modification to use my 'master' kernel recipe:
> > > > +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline"
> > > > +KBUILD_DEFCONFIG:arm = "tegra_defconfig"
> > > > +KBUILD_DEFCONFIG:aarch64 = "defconfig"
> > > > +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig"
> > > >
> > > > I tried again. With the patches perf builds for all three machines,
> > > > reverting the patches and it fails again.
> > > > The x86 compiles forever, for the two ARM machines I get:
> > > > ...
> > > > Traceback (most recent call last):
> > > >   File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py",
> > > > line 83, in <module>
> > > >     print( preamble.group(1) )
> > > > AttributeError: 'NoneType' object has no attribute 'group'
> > >
> > > Definitely still fails here for some of my configurations. I'll look
> > > into it more today, and see if there's something in my reference
> > > kernel configurations that are causing it.
> > >
> > > I need to sort this out, and can put it in a queue with my -dev
> > > upgrades to 5.16, since without those version bumps, we have nothing
> > > in core that exercises the change.
> > >
> > > Cheers,
> > >
> > > Bruce
> > >
> > > > ...
> > > >
> > > > Max
> > > >
> > > > >
> > > > > >
> > > > > > Note that the kernel itself doesn't build either as after 5.15 it now
> > > > > > does try to run the dt_binding_check for which the kernel recipe does
> > > > > > not provide whatever stuff is needed.
> > > > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable
> > > > > > DT schema checks for %.dtb targets") to get over that.
> > > > > >
> > > > > > Max Krummenacher (2):
> > > > > >   perf: sort-pmuevents: don't drop elements
> > > > > >   perf: sort-pmuevents: allow for additional type qualifiers and storage
> > > > > >     class
> > > > > >
> > > > > >  .../perf/perf/sort-pmuevents.py               | 28 +++++++++++--------
> > > > > >  1 file changed, 16 insertions(+), 12 deletions(-)
> > > > > >
> > > > > > --
> > > > > > 2.20.1
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > > > thee at its end
> > > > > - "Use the force Harry" - Gandalf, Star Trek II
> > >
> > >
> > >
> > > --
> > > - Thou shalt not follow the NULL pointer, for chaos and madness await
> > > thee at its end
> > > - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#158604): https://lists.openembedded.org/g/openembedded-core/message/158604
> Mute This Topic: https://lists.openembedded.org/mt/87238903/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II