* [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes @ 2021-11-22 16:34 Max Krummenacher 2021-11-22 16:34 ` [oe][OE-core][Patch 1/2] perf: sort-pmuevents: don't drop elements Max Krummenacher ` (3 more replies) 0 siblings, 4 replies; 13+ messages in thread From: Max Krummenacher @ 2021-11-22 16:34 UTC (permalink / raw) To: openembedded-core; +Cc: bruce.ashfield, Max Krummenacher I am trying out a kernel recipe using linux master, 5.16-rc2. I found that perf fails to build in do_compile with such a setup. Root cause is that the kernel did add 'static' and 'const' to the arrays of structs which sort-pmuevents does sort. This is addressed in the second commit. While at it I noticed that there are some struct arrays which neither have a name or cpuid element with 1 array element. That 1 element was dropped. I did not verify if that causes any issues at runtime, however that behaviour was improved with the first commit. Note that the kernel itself doesn't build either as after 5.15 it now does try to run the dt_binding_check for which the kernel recipe does not provide whatever stuff is needed. I just did a revert patch for commit 53182e81f47d ("kbuild: Enable DT schema checks for %.dtb targets") to get over that. Max Krummenacher (2): perf: sort-pmuevents: don't drop elements perf: sort-pmuevents: allow for additional type qualifiers and storage class .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- 1 file changed, 16 insertions(+), 12 deletions(-) -- 2.20.1 ^ permalink raw reply [flat|nested] 13+ messages in thread
* [oe][OE-core][Patch 1/2] perf: sort-pmuevents: don't drop elements 2021-11-22 16:34 [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Max Krummenacher @ 2021-11-22 16:34 ` Max Krummenacher 2021-11-22 16:34 ` [oe][OE-core][Patch 2/2] perf: sort-pmuevents: allow for additional type qualifiers and storage class Max Krummenacher ` (2 subsequent siblings) 3 siblings, 0 replies; 13+ messages in thread From: Max Krummenacher @ 2021-11-22 16:34 UTC (permalink / raw) To: openembedded-core; +Cc: bruce.ashfield, Max Krummenacher If a struct element neither has an element cpuid or name it gets silenty dropped. Kernel 5.15 for some ARCHs have at least one array of structs matching this. e.g. for arm pmu-events.c: |#include "pmu-events/pmu-events.h" struct pmu_events_map pmu_events_map[] = { { .cpuid = 0, .version = 0, .type = 0, .table = 0, }, }; struct pmu_sys_events pmu_sys_event_tables[] = { { .table = 0 }, }; Before this patch the second struct is translated to an empty array:: struct pmu_sys_events pmu_sys_event_tables[] = { }; Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com> --- meta/recipes-kernel/perf/perf/sort-pmuevents.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py index 5ddf0f144f..4f841eb822 100755 --- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py +++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py @@ -61,6 +61,8 @@ for struct in re.findall( struct_block_regex, data ): #print( " name found: %s" % name.group(1) ) entry_dict[struct[1]]['fields'][name.group(1)] = entry + if not entry_dict[struct[1]]['fields']: + entry_dict[struct[1]]['fields']['0'] = entry # created ordered dictionaries from the captured values. These are ordered by # a sorted() iteration of the keys. We don't care about the order we read -- 2.20.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [oe][OE-core][Patch 2/2] perf: sort-pmuevents: allow for additional type qualifiers and storage class 2021-11-22 16:34 [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Max Krummenacher 2021-11-22 16:34 ` [oe][OE-core][Patch 1/2] perf: sort-pmuevents: don't drop elements Max Krummenacher @ 2021-11-22 16:34 ` Max Krummenacher 2021-11-22 16:51 ` [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Bruce Ashfield 2021-11-23 1:41 ` Bruce Ashfield 3 siblings, 0 replies; 13+ messages in thread From: Max Krummenacher @ 2021-11-22 16:34 UTC (permalink / raw) To: openembedded-core; +Cc: bruce.ashfield, Max Krummenacher With kernel 5.16 some structs in pmu-events do get a const qualifier, some a static const storage class and qualifier. The current sort-pmuevents cannot cope with that and drops all struct arrays with such additional elements. This then leads to compiler errors. Allow '^struct', '^const struct', '^static struct', '^static const struct'. Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com> --- .../perf/perf/sort-pmuevents.py | 30 ++++++++++--------- 1 file changed, 16 insertions(+), 14 deletions(-) diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py b/meta/recipes-kernel/perf/perf/sort-pmuevents.py index 4f841eb822..09ba3328a7 100755 --- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py +++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py @@ -33,10 +33,10 @@ if os.path.exists(outfile): with open(infile, 'r') as file: data = file.read() -preamble_regex = re.compile( '^(.*?)^struct', re.MULTILINE | re.DOTALL ) +preamble_regex = re.compile( '^(.*?)^(struct|const struct|static struct|static const struct)', re.MULTILINE | re.DOTALL ) preamble = re.search( preamble_regex, data ) -struct_block_regex = re.compile( '^struct.*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL ) +struct_block_regex = re.compile( '^(struct|const struct|static struct|static const struct).*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL ) field_regex = re.compile( '{.*?},', re.MULTILINE | re.DOTALL ) cpuid_regex = re.compile( '\.cpuid = (.*?),', re.MULTILINE | re.DOTALL ) name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL ) @@ -45,24 +45,25 @@ name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL ) # types and then their fields. entry_dict = {} for struct in re.findall( struct_block_regex, data ): - # print( "struct: %s %s" % (struct[0],struct[1]) ) - entry_dict[struct[1]] = {} - entry_dict[struct[1]]['type'] = struct[0] - entry_dict[struct[1]]['fields'] = {} - for entry in re.findall( field_regex, struct[2] ): + # print( "struct: %s %s %s" % (struct[0],struct[1],struct[2]) ) + entry_dict[struct[2]] = {} + entry_dict[struct[2]]['type_prefix'] = struct[0] + entry_dict[struct[2]]['type'] = struct[1] + entry_dict[struct[2]]['fields'] = {} + for entry in re.findall( field_regex, struct[3] ): #print( " entry: %s" % entry ) cpuid = re.search( cpuid_regex, entry ) if cpuid: #print( " cpuid found: %s" % cpuid.group(1) ) - entry_dict[struct[1]]['fields'][cpuid.group(1)] = entry - + entry_dict[struct[2]]['fields'][cpuid.group(1)] = entry + name = re.search( name_regex, entry ) if name: #print( " name found: %s" % name.group(1) ) - entry_dict[struct[1]]['fields'][name.group(1)] = entry - - if not entry_dict[struct[1]]['fields']: - entry_dict[struct[1]]['fields']['0'] = entry + entry_dict[struct[2]]['fields'][name.group(1)] = entry + + if not entry_dict[struct[2]]['fields']: + entry_dict[struct[2]]['fields']['0'] = entry # created ordered dictionaries from the captured values. These are ordered by # a sorted() iteration of the keys. We don't care about the order we read @@ -74,6 +75,7 @@ for struct in re.findall( struct_block_regex, data ): entry_dict_sorted = OrderedDict() for i in sorted(entry_dict.keys()): entry_dict_sorted[i] = {} + entry_dict_sorted[i]['type_prefix'] = entry_dict[i]['type_prefix'] entry_dict_sorted[i]['type'] = entry_dict[i]['type'] entry_dict_sorted[i]['fields'] = {} for f in sorted(entry_dict[i]['fields'].keys()): @@ -85,7 +87,7 @@ outf = open( outfile, 'w' ) print( preamble.group(1) ) outf.write( preamble.group(1) ) for d in entry_dict_sorted: - outf.write( "struct %s %s[] = {\n" % (entry_dict_sorted[d]['type'],d) ) + outf.write( "%s %s %s[] = {\n" % (entry_dict_sorted[d]['type_prefix'], entry_dict_sorted[d]['type'],d) ) for f in entry_dict_sorted[d]['fields']: outf.write( entry_dict_sorted[d]['fields'][f] + '\n' ) -- 2.20.1 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-22 16:34 [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Max Krummenacher 2021-11-22 16:34 ` [oe][OE-core][Patch 1/2] perf: sort-pmuevents: don't drop elements Max Krummenacher 2021-11-22 16:34 ` [oe][OE-core][Patch 2/2] perf: sort-pmuevents: allow for additional type qualifiers and storage class Max Krummenacher @ 2021-11-22 16:51 ` Bruce Ashfield 2021-11-22 17:23 ` Daniel Díaz 2021-11-23 1:41 ` Bruce Ashfield 3 siblings, 1 reply; 13+ messages in thread From: Bruce Ashfield @ 2021-11-22 16:51 UTC (permalink / raw) To: Max Krummenacher Cc: Patches and discussions about the oe-core layer, Max Krummenacher On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found > that perf fails to build in do_compile with such a setup. > Root cause is that the kernel did add 'static' and 'const' to the > arrays of structs which sort-pmuevents does sort. > This is addressed in the second commit. > > While at it I noticed that there are some struct arrays which neither > have a name or cpuid element with 1 array element. That 1 element was > dropped. I did not verify if that causes any issues at runtime, however > that behaviour was improved with the first commit. > > Note that the kernel itself doesn't build either as after 5.15 it now > does try to run the dt_binding_check for which the kernel recipe does > not provide whatever stuff is needed. > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > DT schema checks for %.dtb targets") to get over that. I have a fixefor that particular problem (the bindings), so you can safely continue to ignore it .. but I hadn't gotten around to fixing perf yet. The changes look fine to me for perf. The sorting of events is hopefully something we can drop soon, as there have been other reproducibility changes to perf upstream, that will make it unnecessary. Bruce > > Max Krummenacher (2): > perf: sort-pmuevents: don't drop elements > perf: sort-pmuevents: allow for additional type qualifiers and storage > class > > .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- > 1 file changed, 16 insertions(+), 12 deletions(-) > > -- > 2.20.1 > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-22 16:51 ` [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Bruce Ashfield @ 2021-11-22 17:23 ` Daniel Díaz 2021-11-23 1:39 ` Bruce Ashfield 0 siblings, 1 reply; 13+ messages in thread From: Daniel Díaz @ 2021-11-22 17:23 UTC (permalink / raw) To: Bruce Ashfield Cc: Max Krummenacher, Patches and discussions about the oe-core layer, Max Krummenacher Hello! On Mon, 22 Nov 2021 at 10:51, Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: [...] > > Note that the kernel itself doesn't build either as after 5.15 it now > > does try to run the dt_binding_check for which the kernel recipe does > > not provide whatever stuff is needed. > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > DT schema checks for %.dtb targets") to get over that. > > I have a fixefor that particular problem (the bindings), so you can > safely continue to ignore it .. but I hadn't gotten around to fixing > perf yet. [...] I don't mean to hijack the subject, but do you have some quick pointers on that? We're seeing the same on linux-next and linux-mainline since that patch landed in those trees, and have soft-reverted the commit for now. This (incomplete) recipe has not helped us get through that: https://github.com/mrchapp/meta-lkft/blob/d/add-dtschema/recipes-devtools/python/dtschema_2021.10.bb Thanks and greetings! Daniel Díaz daniel.diaz@linaro.org ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-22 17:23 ` Daniel Díaz @ 2021-11-23 1:39 ` Bruce Ashfield 2021-11-23 5:05 ` Bruce Ashfield 0 siblings, 1 reply; 13+ messages in thread From: Bruce Ashfield @ 2021-11-23 1:39 UTC (permalink / raw) To: Daniel Díaz Cc: Max Krummenacher, Patches and discussions about the oe-core layer, Max Krummenacher On Mon, Nov 22, 2021 at 12:23 PM Daniel Díaz <daniel.diaz@linaro.org> wrote: > > Hello! > > On Mon, 22 Nov 2021 at 10:51, Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > [...] > > > Note that the kernel itself doesn't build either as after 5.15 it now > > > does try to run the dt_binding_check for which the kernel recipe does > > > not provide whatever stuff is needed. > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > > DT schema checks for %.dtb targets") to get over that. > > > > I have a fixefor that particular problem (the bindings), so you can > > safely continue to ignore it .. but I hadn't gotten around to fixing > > perf yet. > [...] > > I don't mean to hijack the subject, but do you have some quick > pointers on that? We're seeing the same on linux-next and > linux-mainline since that patch landed in those trees, and have I'm trying to come up with a way to fix this without needing a kernel patch. Sounds like you are already aware of the pkgconfig issue, which is what I'm attempting to fix via environment manipulation versus a patch. We do need python dtschema as well, as a new native dependency of the bindings check, which unfortunately pulls in ruamel and jsonschema. Those are both in meta-python, so they need to migrate to core to support the new checking. There's also a problem with nproc being used, and it was giving me an error on one of my servers, I haven't looked into it more yet. I have a quick recipe here as well. And with those two pieces in place, I'm up and running again. The right thing for the short term, is that revert. There's quite a bit of sorting out to do, to get this in decent shape for core ... personally, I'd like to make the checking optional for those that don't want it .. but that goes against what Rob is trying to do in the kernel :D If those issues sound familiar, and there's interest, I can drop my WIP commits into a branch and share. Bruce > soft-reverted the commit for now. This (incomplete) recipe has not > helped us get through that: > > https://github.com/mrchapp/meta-lkft/blob/d/add-dtschema/recipes-devtools/python/dtschema_2021.10.bb > > Thanks and greetings! > > Daniel Díaz > daniel.diaz@linaro.org -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-23 1:39 ` Bruce Ashfield @ 2021-11-23 5:05 ` Bruce Ashfield 0 siblings, 0 replies; 13+ messages in thread From: Bruce Ashfield @ 2021-11-23 5:05 UTC (permalink / raw) To: Daniel Díaz Cc: Max Krummenacher, Patches and discussions about the oe-core layer, Max Krummenacher On Mon, Nov 22, 2021 at 8:39 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > On Mon, Nov 22, 2021 at 12:23 PM Daniel Díaz <daniel.diaz@linaro.org> wrote: > > > > Hello! > > > > On Mon, 22 Nov 2021 at 10:51, Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > [...] > > > > Note that the kernel itself doesn't build either as after 5.15 it now > > > > does try to run the dt_binding_check for which the kernel recipe does > > > > not provide whatever stuff is needed. > > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > > > DT schema checks for %.dtb targets") to get over that. > > > > > > I have a fixefor that particular problem (the bindings), so you can > > > safely continue to ignore it .. but I hadn't gotten around to fixing > > > perf yet. > > [...] > > > > I don't mean to hijack the subject, but do you have some quick > > pointers on that? We're seeing the same on linux-next and > > linux-mainline since that patch landed in those trees, and have > > I'm trying to come up with a way to fix this without needing a kernel patch. > FYI: I have this sorted out now. The commits to get 5.16 building with device tree validation are here: https://github.com/zeddii/openembedded-core/tree/zedd/kernel (I would have pushed it to my poky-contrib zedd/kernel branch, but there's something broken on the machine tonight). You'll need to add the DEPENDS to your kernel recipe, like I did for linux-yocto-dev, but outside of that the pkg-config tweaking that I did avoided the need for kernel patches. The dtschema recipe + the imported meta-python recipes should "just work". I haven't done a second pass to make sure I found all the dependencies, but it is close .. and if you have meta-python in your layers, you won't have any issues. For now, the DEPENDS are recipe specific, since they are fairly heavy, and the mapping of PV -> ARCH -> conditional dependencies is still a WIP (it gets brittle/complex very quickly) .. but moving the DEPENDS into kernel.bbclass is something that I'm working on as well. I still need more cleanup, but I expect that I can submit the queue by the end of the week. .. and the proof: qemuarm64 login: root uroot@qemuarm64:~# uname -a Linux qemuarm64 5.16.0-rc2-yoctodev-standard #1 SMP PREEMPT Sun Nov 21 21:47:39 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux root@qemuarm64:~# Cheers, Bruce > Sounds like you are already aware of the pkgconfig issue, which is > what I'm attempting to fix via environment manipulation versus a > patch. > > We do need python dtschema as well, as a new native dependency of the > bindings check, which unfortunately pulls in ruamel and jsonschema. > Those are both in meta-python, so they need to migrate to core to > support the new checking. > > There's also a problem with nproc being used, and it was giving me an > error on one of my servers, I haven't looked into it more yet. > > I have a quick recipe here as well. And with those two pieces in > place, I'm up and running again. > > The right thing for the short term, is that revert. There's quite a > bit of sorting out to do, to get this in decent shape for core ... > personally, I'd like to make the checking optional for those that > don't want it .. but that goes against what Rob is trying to do in the > kernel :D > > If those issues sound familiar, and there's interest, I can drop my > WIP commits into a branch and share. > > Bruce > > > soft-reverted the commit for now. This (incomplete) recipe has not > > helped us get through that: > > > > https://github.com/mrchapp/meta-lkft/blob/d/add-dtschema/recipes-devtools/python/dtschema_2021.10.bb > > > > Thanks and greetings! > > > > Daniel Díaz > > daniel.diaz@linaro.org > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-22 16:34 [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Max Krummenacher ` (2 preceding siblings ...) 2021-11-22 16:51 ` [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Bruce Ashfield @ 2021-11-23 1:41 ` Bruce Ashfield 2021-11-23 13:48 ` Max Krummenacher 3 siblings, 1 reply; 13+ messages in thread From: Bruce Ashfield @ 2021-11-23 1:41 UTC (permalink / raw) To: Max Krummenacher Cc: Patches and discussions about the oe-core layer, Max Krummenacher On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found > that perf fails to build in do_compile with such a setup. > Root cause is that the kernel did add 'static' and 'const' to the > arrays of structs which sort-pmuevents does sort. > This is addressed in the second commit. > > While at it I noticed that there are some struct arrays which neither > have a name or cpuid element with 1 array element. That 1 element was > dropped. I did not verify if that causes any issues at runtime, however > that behaviour was improved with the first commit. What MACHINE were you testing against ? When I build perf against my -rc2 dev kernel, I still run into the build issues with these commits in place. I just want to make sure that I'm not seeing something different .. so I can do some runs against a similar config as you are using. Bruce > > Note that the kernel itself doesn't build either as after 5.15 it now > does try to run the dt_binding_check for which the kernel recipe does > not provide whatever stuff is needed. > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > DT schema checks for %.dtb targets") to get over that. > > Max Krummenacher (2): > perf: sort-pmuevents: don't drop elements > perf: sort-pmuevents: allow for additional type qualifiers and storage > class > > .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- > 1 file changed, 16 insertions(+), 12 deletions(-) > > -- > 2.20.1 > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-23 1:41 ` Bruce Ashfield @ 2021-11-23 13:48 ` Max Krummenacher 2021-11-23 13:55 ` Bruce Ashfield 0 siblings, 1 reply; 13+ messages in thread From: Max Krummenacher @ 2021-11-23 13:48 UTC (permalink / raw) To: Bruce Ashfield Cc: Patches and discussions about the oe-core layer, Max Krummenacher Hi Bruce, On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found > > that perf fails to build in do_compile with such a setup. > > Root cause is that the kernel did add 'static' and 'const' to the > > arrays of structs which sort-pmuevents does sort. > > This is addressed in the second commit. > > > > While at it I noticed that there are some struct arrays which neither > > have a name or cpuid element with 1 array element. That 1 element was > > dropped. I did not verify if that causes any issues at runtime, however > > that behaviour was improved with the first commit. > > What MACHINE were you testing against ? When I build perf against my > -rc2 dev kernel, I still run into the build issues with these commits > in place. > > I just want to make sure that I'm not seeing something different .. so > I can do some runs against a similar config as you are using. > I did the inital work against a Toradex machine, apalis-imx6. However I built before sending the patches with qemuarm64 qemuarm qemux86-64 with the following modification to use my 'master' kernel recipe: +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline" +KBUILD_DEFCONFIG:arm = "tegra_defconfig" +KBUILD_DEFCONFIG:aarch64 = "defconfig" +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig" I tried again. With the patches perf builds for all three machines, reverting the patches and it fails again. The x86 compiles forever, for the two ARM machines I get: ... Traceback (most recent call last): File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py", line 83, in <module> print( preamble.group(1) ) AttributeError: 'NoneType' object has no attribute 'group' ... Max > > > > > Note that the kernel itself doesn't build either as after 5.15 it now > > does try to run the dt_binding_check for which the kernel recipe does > > not provide whatever stuff is needed. > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > DT schema checks for %.dtb targets") to get over that. > > > > Max Krummenacher (2): > > perf: sort-pmuevents: don't drop elements > > perf: sort-pmuevents: allow for additional type qualifiers and storage > > class > > > > .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- > > 1 file changed, 16 insertions(+), 12 deletions(-) > > > > -- > > 2.20.1 > > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-23 13:48 ` Max Krummenacher @ 2021-11-23 13:55 ` Bruce Ashfield 2021-11-23 13:58 ` Max Krummenacher 0 siblings, 1 reply; 13+ messages in thread From: Bruce Ashfield @ 2021-11-23 13:55 UTC (permalink / raw) To: Max Krummenacher Cc: Patches and discussions about the oe-core layer, Max Krummenacher On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > Hi Bruce, > > On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found > > > that perf fails to build in do_compile with such a setup. > > > Root cause is that the kernel did add 'static' and 'const' to the > > > arrays of structs which sort-pmuevents does sort. > > > This is addressed in the second commit. > > > > > > While at it I noticed that there are some struct arrays which neither > > > have a name or cpuid element with 1 array element. That 1 element was > > > dropped. I did not verify if that causes any issues at runtime, however > > > that behaviour was improved with the first commit. > > > > What MACHINE were you testing against ? When I build perf against my > > -rc2 dev kernel, I still run into the build issues with these commits > > in place. > > > > I just want to make sure that I'm not seeing something different .. so > > I can do some runs against a similar config as you are using. > > > > I did the inital work against a Toradex machine, apalis-imx6. > However I built before sending the patches with qemuarm64 qemuarm qemux86-64 > with the following modification to use my 'master' kernel recipe: > +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline" > +KBUILD_DEFCONFIG:arm = "tegra_defconfig" > +KBUILD_DEFCONFIG:aarch64 = "defconfig" > +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig" > > I tried again. With the patches perf builds for all three machines, > reverting the patches and it fails again. > The x86 compiles forever, for the two ARM machines I get: > ... > Traceback (most recent call last): > File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py", > line 83, in <module> > print( preamble.group(1) ) > AttributeError: 'NoneType' object has no attribute 'group' Definitely still fails here for some of my configurations. I'll look into it more today, and see if there's something in my reference kernel configurations that are causing it. I need to sort this out, and can put it in a queue with my -dev upgrades to 5.16, since without those version bumps, we have nothing in core that exercises the change. Cheers, Bruce > ... > > Max > > > > > > > > > Note that the kernel itself doesn't build either as after 5.15 it now > > > does try to run the dt_binding_check for which the kernel recipe does > > > not provide whatever stuff is needed. > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > > DT schema checks for %.dtb targets") to get over that. > > > > > > Max Krummenacher (2): > > > perf: sort-pmuevents: don't drop elements > > > perf: sort-pmuevents: allow for additional type qualifiers and storage > > > class > > > > > > .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- > > > 1 file changed, 16 insertions(+), 12 deletions(-) > > > > > > -- > > > 2.20.1 > > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-23 13:55 ` Bruce Ashfield @ 2021-11-23 13:58 ` Max Krummenacher 2021-11-23 14:04 ` Bruce Ashfield [not found] ` <16BA31E66A619EB2.13215@lists.openembedded.org> 0 siblings, 2 replies; 13+ messages in thread From: Max Krummenacher @ 2021-11-23 13:58 UTC (permalink / raw) To: Bruce Ashfield Cc: Patches and discussions about the oe-core layer, Max Krummenacher Let me know if there is anything I can do/provide from my side. I know you already did, but does 'bitbake perf -c cleansstate' help? Max On Tue, Nov 23, 2021 at 2:55 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > Hi Bruce, > > > > On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > > > > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > > > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found > > > > that perf fails to build in do_compile with such a setup. > > > > Root cause is that the kernel did add 'static' and 'const' to the > > > > arrays of structs which sort-pmuevents does sort. > > > > This is addressed in the second commit. > > > > > > > > While at it I noticed that there are some struct arrays which neither > > > > have a name or cpuid element with 1 array element. That 1 element was > > > > dropped. I did not verify if that causes any issues at runtime, however > > > > that behaviour was improved with the first commit. > > > > > > What MACHINE were you testing against ? When I build perf against my > > > -rc2 dev kernel, I still run into the build issues with these commits > > > in place. > > > > > > I just want to make sure that I'm not seeing something different .. so > > > I can do some runs against a similar config as you are using. > > > > > > > I did the inital work against a Toradex machine, apalis-imx6. > > However I built before sending the patches with qemuarm64 qemuarm qemux86-64 > > with the following modification to use my 'master' kernel recipe: > > +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline" > > +KBUILD_DEFCONFIG:arm = "tegra_defconfig" > > +KBUILD_DEFCONFIG:aarch64 = "defconfig" > > +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig" > > > > I tried again. With the patches perf builds for all three machines, > > reverting the patches and it fails again. > > The x86 compiles forever, for the two ARM machines I get: > > ... > > Traceback (most recent call last): > > File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py", > > line 83, in <module> > > print( preamble.group(1) ) > > AttributeError: 'NoneType' object has no attribute 'group' > > Definitely still fails here for some of my configurations. I'll look > into it more today, and see if there's something in my reference > kernel configurations that are causing it. > > I need to sort this out, and can put it in a queue with my -dev > upgrades to 5.16, since without those version bumps, we have nothing > in core that exercises the change. > > Cheers, > > Bruce > > > ... > > > > Max > > > > > > > > > > > > > Note that the kernel itself doesn't build either as after 5.15 it now > > > > does try to run the dt_binding_check for which the kernel recipe does > > > > not provide whatever stuff is needed. > > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > > > DT schema checks for %.dtb targets") to get over that. > > > > > > > > Max Krummenacher (2): > > > > perf: sort-pmuevents: don't drop elements > > > > perf: sort-pmuevents: allow for additional type qualifiers and storage > > > > class > > > > > > > > .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- > > > > 1 file changed, 16 insertions(+), 12 deletions(-) > > > > > > > > -- > > > > 2.20.1 > > > > > > > > > > > > > -- > > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > > thee at its end > > > - "Use the force Harry" - Gandalf, Star Trek II > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes 2021-11-23 13:58 ` Max Krummenacher @ 2021-11-23 14:04 ` Bruce Ashfield [not found] ` <16BA31E66A619EB2.13215@lists.openembedded.org> 1 sibling, 0 replies; 13+ messages in thread From: Bruce Ashfield @ 2021-11-23 14:04 UTC (permalink / raw) To: Max Krummenacher Cc: Patches and discussions about the oe-core layer, Max Krummenacher On Tue, Nov 23, 2021 at 8:58 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > Let me know if there is anything I can do/provide from my side. > I know you already did, but does 'bitbake perf -c cleansstate' help? The builds were clean, but to rule out old artifacts, I just started a new build on a separate server, without my normal sstate sharing configuration. Fingers crossed :D Bruce > > Max > > On Tue, Nov 23, 2021 at 2:55 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > > > On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > > > Hi Bruce, > > > > > > On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > > > > > > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > > > > > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found > > > > > that perf fails to build in do_compile with such a setup. > > > > > Root cause is that the kernel did add 'static' and 'const' to the > > > > > arrays of structs which sort-pmuevents does sort. > > > > > This is addressed in the second commit. > > > > > > > > > > While at it I noticed that there are some struct arrays which neither > > > > > have a name or cpuid element with 1 array element. That 1 element was > > > > > dropped. I did not verify if that causes any issues at runtime, however > > > > > that behaviour was improved with the first commit. > > > > > > > > What MACHINE were you testing against ? When I build perf against my > > > > -rc2 dev kernel, I still run into the build issues with these commits > > > > in place. > > > > > > > > I just want to make sure that I'm not seeing something different .. so > > > > I can do some runs against a similar config as you are using. > > > > > > > > > > I did the inital work against a Toradex machine, apalis-imx6. > > > However I built before sending the patches with qemuarm64 qemuarm qemux86-64 > > > with the following modification to use my 'master' kernel recipe: > > > +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline" > > > +KBUILD_DEFCONFIG:arm = "tegra_defconfig" > > > +KBUILD_DEFCONFIG:aarch64 = "defconfig" > > > +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig" > > > > > > I tried again. With the patches perf builds for all three machines, > > > reverting the patches and it fails again. > > > The x86 compiles forever, for the two ARM machines I get: > > > ... > > > Traceback (most recent call last): > > > File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py", > > > line 83, in <module> > > > print( preamble.group(1) ) > > > AttributeError: 'NoneType' object has no attribute 'group' > > > > Definitely still fails here for some of my configurations. I'll look > > into it more today, and see if there's something in my reference > > kernel configurations that are causing it. > > > > I need to sort this out, and can put it in a queue with my -dev > > upgrades to 5.16, since without those version bumps, we have nothing > > in core that exercises the change. > > > > Cheers, > > > > Bruce > > > > > ... > > > > > > Max > > > > > > > > > > > > > > > > > Note that the kernel itself doesn't build either as after 5.15 it now > > > > > does try to run the dt_binding_check for which the kernel recipe does > > > > > not provide whatever stuff is needed. > > > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > > > > DT schema checks for %.dtb targets") to get over that. > > > > > > > > > > Max Krummenacher (2): > > > > > perf: sort-pmuevents: don't drop elements > > > > > perf: sort-pmuevents: allow for additional type qualifiers and storage > > > > > class > > > > > > > > > > .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- > > > > > 1 file changed, 16 insertions(+), 12 deletions(-) > > > > > > > > > > -- > > > > > 2.20.1 > > > > > > > > > > > > > > > > > -- > > > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > > > thee at its end > > > > - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
[parent not found: <16BA31E66A619EB2.13215@lists.openembedded.org>]
* Re: [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes [not found] ` <16BA31E66A619EB2.13215@lists.openembedded.org> @ 2021-11-23 21:40 ` Bruce Ashfield 0 siblings, 0 replies; 13+ messages in thread From: Bruce Ashfield @ 2021-11-23 21:40 UTC (permalink / raw) To: Bruce Ashfield Cc: Max Krummenacher, Patches and discussions about the oe-core layer, Max Krummenacher On Tue, Nov 23, 2021 at 9:05 AM Bruce Ashfield via lists.openembedded.org <bruce.ashfield=gmail.com@lists.openembedded.org> wrote: > > On Tue, Nov 23, 2021 at 8:58 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > Let me know if there is anything I can do/provide from my side. > > I know you already did, but does 'bitbake perf -c cleansstate' help? > > The builds were clean, but to rule out old artifacts, I just started a > new build on a separate server, without my normal sstate sharing > configuration. Fingers crossed :D It is building on my completely fresh box. Which does imply I was getting the old script before .. but I can't say for sure why. Regardless, I'm sending a consolidated pull request shortly with some v5.16 fixes, and have these stacked on top, just to confirm they are working in my setup and that I've tested them. Bruce > > Bruce > > > > > Max > > > > On Tue, Nov 23, 2021 at 2:55 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > > > > > On Tue, Nov 23, 2021 at 8:48 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > > > > > Hi Bruce, > > > > > > > > On Tue, Nov 23, 2021 at 2:41 AM Bruce Ashfield <bruce.ashfield@gmail.com> wrote: > > > > > > > > > > On Mon, Nov 22, 2021 at 11:35 AM Max Krummenacher <max.oss.09@gmail.com> wrote: > > > > > > > > > > > > I am trying out a kernel recipe using linux master, 5.16-rc2. I found > > > > > > that perf fails to build in do_compile with such a setup. > > > > > > Root cause is that the kernel did add 'static' and 'const' to the > > > > > > arrays of structs which sort-pmuevents does sort. > > > > > > This is addressed in the second commit. > > > > > > > > > > > > While at it I noticed that there are some struct arrays which neither > > > > > > have a name or cpuid element with 1 array element. That 1 element was > > > > > > dropped. I did not verify if that causes any issues at runtime, however > > > > > > that behaviour was improved with the first commit. > > > > > > > > > > What MACHINE were you testing against ? When I build perf against my > > > > > -rc2 dev kernel, I still run into the build issues with these commits > > > > > in place. > > > > > > > > > > I just want to make sure that I'm not seeing something different .. so > > > > > I can do some runs against a similar config as you are using. > > > > > > > > > > > > > I did the inital work against a Toradex machine, apalis-imx6. > > > > However I built before sending the patches with qemuarm64 qemuarm qemux86-64 > > > > with the following modification to use my 'master' kernel recipe: > > > > +PREFERRED_PROVIDER_virtual/kernel = "linux-toradex-mainline" > > > > +KBUILD_DEFCONFIG:arm = "tegra_defconfig" > > > > +KBUILD_DEFCONFIG:aarch64 = "defconfig" > > > > +KBUILD_DEFCONFIG:qemux86-64 = "x86_64_defconfig" > > > > > > > > I tried again. With the patches perf builds for all three machines, > > > > reverting the patches and it fails again. > > > > The x86 compiles forever, for the two ARM machines I get: > > > > ... > > > > Traceback (most recent call last): > > > > File "/build/krm/oe-core_master/build/tmp/work/qemuarm64-tdx-linux/perf/1.0-r9/perf-1.0/sort-pmuevents.py", > > > > line 83, in <module> > > > > print( preamble.group(1) ) > > > > AttributeError: 'NoneType' object has no attribute 'group' > > > > > > Definitely still fails here for some of my configurations. I'll look > > > into it more today, and see if there's something in my reference > > > kernel configurations that are causing it. > > > > > > I need to sort this out, and can put it in a queue with my -dev > > > upgrades to 5.16, since without those version bumps, we have nothing > > > in core that exercises the change. > > > > > > Cheers, > > > > > > Bruce > > > > > > > ... > > > > > > > > Max > > > > > > > > > > > > > > > > > > > > > Note that the kernel itself doesn't build either as after 5.15 it now > > > > > > does try to run the dt_binding_check for which the kernel recipe does > > > > > > not provide whatever stuff is needed. > > > > > > I just did a revert patch for commit 53182e81f47d ("kbuild: Enable > > > > > > DT schema checks for %.dtb targets") to get over that. > > > > > > > > > > > > Max Krummenacher (2): > > > > > > perf: sort-pmuevents: don't drop elements > > > > > > perf: sort-pmuevents: allow for additional type qualifiers and storage > > > > > > class > > > > > > > > > > > > .../perf/perf/sort-pmuevents.py | 28 +++++++++++-------- > > > > > > 1 file changed, 16 insertions(+), 12 deletions(-) > > > > > > > > > > > > -- > > > > > > 2.20.1 > > > > > > > > > > > > > > > > > > > > > -- > > > > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > > > > thee at its end > > > > > - "Use the force Harry" - Gandalf, Star Trek II > > > > > > > > > > > > -- > > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > > thee at its end > > > - "Use the force Harry" - Gandalf, Star Trek II > > > > -- > - Thou shalt not follow the NULL pointer, for chaos and madness await > thee at its end > - "Use the force Harry" - Gandalf, Star Trek II > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#158604): https://lists.openembedded.org/g/openembedded-core/message/158604 > Mute This Topic: https://lists.openembedded.org/mt/87238903/1050810 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2021-11-23 21:40 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-11-22 16:34 [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Max Krummenacher 2021-11-22 16:34 ` [oe][OE-core][Patch 1/2] perf: sort-pmuevents: don't drop elements Max Krummenacher 2021-11-22 16:34 ` [oe][OE-core][Patch 2/2] perf: sort-pmuevents: allow for additional type qualifiers and storage class Max Krummenacher 2021-11-22 16:51 ` [oe][OE-core][Patch 0/2] perf: sort-pmuevents: some fixes Bruce Ashfield 2021-11-22 17:23 ` Daniel Díaz 2021-11-23 1:39 ` Bruce Ashfield 2021-11-23 5:05 ` Bruce Ashfield 2021-11-23 1:41 ` Bruce Ashfield 2021-11-23 13:48 ` Max Krummenacher 2021-11-23 13:55 ` Bruce Ashfield 2021-11-23 13:58 ` Max Krummenacher 2021-11-23 14:04 ` Bruce Ashfield [not found] ` <16BA31E66A619EB2.13215@lists.openembedded.org> 2021-11-23 21:40 ` Bruce Ashfield
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.