[OE-core][Patch v1 1/2] perf: fix build with latest kernel

[OE-core][Patch v1 1/2] perf: fix build with latest kernel

[max.oss.09]

From: Max Krummenacher <max.krummenacher@toradex.com>

Kernel commit 9eea8fafe33e ("libbpf: fix __arg_ctx type enforcement for
perf_event programs") added with 6.9-rc1 tools/include/uapi/asm/bpf_perf_event.h
which for arc, arm64, loongarch, riscv and s390 includes headers from
the arch/${ARCH}/... directories.
Thus the build in OE fails as those headers are not present.

Fix this by additioanly copying all files in arch/${ARCH}/include/.

Fixes:
ERROR: perf-1.0-r0 do_compile: oe_runmake failed
| In file included from work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/linux/bpf_perf_event.h:11,
|                  from libbpf.c:36:
| work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/asm/bpf_perf_event.h:2:10: fatal error: ../../arch/arm64/include/uapi/asm/bpf_perf_event.h: No such file or directory
|     2 | #include "../../arch/arm64/include/uapi/asm/bpf_perf_event.h"
|       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
---
 meta/recipes-kernel/perf/perf.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 4f26813de0..6408c65462 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -139,6 +139,7 @@ PERF_SRC ?= "Makefile \
              tools/scripts \
              scripts/ \
              arch/arm64/tools \
+             arch/${ARCH}/include \
              arch/${ARCH}/Makefile \
 "
 
-- 
2.42.0

[OE-core][Patch v1 2/2] perf: tests: fix qa error, missing perl

[max.oss.09]

From: Max Krummenacher <max.krummenacher@toradex.com>

Kernel commit 61d348f1e96f ("perf testsuite: Add common output checking
helpers") added with 6.9-rc1 added addional testscripts written in perl.

Fixes:
| ERROR: perf-1.0-r0 do_package_qa: QA Issue: /usr/libexec/perf-core/tests/shell/common/check_no_patterns_found.pl contained in package perf-tests requires /usr/bin/perl, but no providers found in RDEPENDS:perf-tests? [file-rdeps]

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
---
 meta/recipes-kernel/perf/perf.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
index 6408c65462..4b6834e614 100644
--- a/meta/recipes-kernel/perf/perf.bb
+++ b/meta/recipes-kernel/perf/perf.bb
@@ -383,7 +383,7 @@ RDEPENDS:${PN} += "elfutils bash"
 RDEPENDS:${PN}-archive =+ "bash"
 RDEPENDS:${PN}-python =+ "bash python3 python3-modules ${@bb.utils.contains('PACKAGECONFIG', 'audit', 'audit-python', '', d)}"
 RDEPENDS:${PN}-perl =+ "bash perl perl-modules"
-RDEPENDS:${PN}-tests =+ "python3 bash"
+RDEPENDS:${PN}-tests =+ "perl python3 bash"
 
 RSUGGESTS:${PN} += "${PN}-archive ${PN}-tests \
                     ${@bb.utils.contains('PACKAGECONFIG', 'perl', '${PN}-perl', '', d)} \
-- 
2.42.0

Re: [OE-core][Patch v1 2/2] perf: tests: fix qa error, missing perl

[Richard Purdie]

On Mon, 2024-03-25 at 11:20 +0100, Max Krummenacher wrote:
> From: Max Krummenacher <max.krummenacher@toradex.com>
> 
> Kernel commit 61d348f1e96f ("perf testsuite: Add common output checking
> helpers") added with 6.9-rc1 added addional testscripts written in perl.
> 
> Fixes:
> > ERROR: perf-1.0-r0 do_package_qa: QA Issue: /usr/libexec/perf-core/tests/shell/common/check_no_patterns_found.pl contained in package perf-tests requires /usr/bin/perl, but no providers found in RDEPENDS:perf-tests? [file-rdeps]
> 
> Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
> ---
>  meta/recipes-kernel/perf/perf.bb | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> index 6408c65462..4b6834e614 100644
> --- a/meta/recipes-kernel/perf/perf.bb
> +++ b/meta/recipes-kernel/perf/perf.bb
> @@ -383,7 +383,7 @@ RDEPENDS:${PN} += "elfutils bash"
>  RDEPENDS:${PN}-archive =+ "bash"
>  RDEPENDS:${PN}-python =+ "bash python3 python3-modules ${@bb.utils.contains('PACKAGECONFIG', 'audit', 'audit-python', '', d)}"
>  RDEPENDS:${PN}-perl =+ "bash perl perl-modules"
> -RDEPENDS:${PN}-tests =+ "python3 bash"
> +RDEPENDS:${PN}-tests =+ "perl python3 bash"
>  
>  RSUGGESTS:${PN} += "${PN}-archive ${PN}-tests \
>                      ${@bb.utils.contains('PACKAGECONFIG', 'perl', '${PN}-perl', '', d)} \

Does perf avoid perl dependencies currently? Would it make more sense
to put that script in ${PN}-perl?

Cheers,

Richard

Re: [OE-core][Patch v1 1/2] perf: fix build with latest kernel

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 2489 bytes --]

On Mon, Mar 25, 2024 at 6:21 AM <max.oss.09@gmail.com> wrote:

> From: Max Krummenacher <max.krummenacher@toradex.com>
>
> Kernel commit 9eea8fafe33e ("libbpf: fix __arg_ctx type enforcement for
> perf_event programs") added with 6.9-rc1
> tools/include/uapi/asm/bpf_perf_event.h
> which for arc, arm64, loongarch, riscv and s390 includes headers from
> the arch/${ARCH}/... directories.
> Thus the build in OE fails as those headers are not present.
>
> Fix this by additioanly copying all files in arch/${ARCH}/include/.
>
> Fixes:
> ERROR: perf-1.0-r0 do_compile: oe_runmake failed
> | In file included from
> work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/linux/bpf_perf_event.h:11,
> |                  from libbpf.c:36:
> |
> work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/asm/bpf_perf_event.h:2:10:
> fatal error: ../../arch/arm64/include/uapi/asm/bpf_perf_event.h: No such
> file or directory
> |     2 | #include "../../arch/arm64/include/uapi/asm/bpf_perf_event.h"
> |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
> ---
>  meta/recipes-kernel/perf/perf.bb | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/
> perf.bb
> index 4f26813de0..6408c65462 100644
> --- a/meta/recipes-kernel/perf/perf.bb
> +++ b/meta/recipes-kernel/perf/perf.bb
> @@ -139,6 +139,7 @@ PERF_SRC ?= "Makefile \
>               tools/scripts \
>               scripts/ \
>               arch/arm64/tools \
> +             arch/${ARCH}/include \
>

We've always tried to stay within the tools/ source umbrella since we really
don't want to copy any more of the kernel than we have to, since otherwise,
we might as well go back to simplifying things and just building against the
entire kernel source tree.

I'd like to see if there's a way that we could limit this to the uapi (and
hence
use our uapi headers), and avoid needing to make the copy. If we need a
copy, then being as specific as possible would be the goal.

I'm not moving linux-yocto-dev to the latest -dev until next week, but I
can
look into this more then.

Bruce



>               arch/${ARCH}/Makefile \
>  "
>
> --
> 2.42.0
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 4510 bytes --]

Re: [OE-core][Patch v1 1/2] perf: fix build with latest kernel

[Bruce Ashfield]

[-- Attachment #1: Type: text/plain, Size: 3135 bytes --]

On Tue, Mar 26, 2024 at 1:46 PM Bruce Ashfield <bruce.ashfield@gmail.com>
wrote:

>
>
> On Mon, Mar 25, 2024 at 6:21 AM <max.oss.09@gmail.com> wrote:
>
>> From: Max Krummenacher <max.krummenacher@toradex.com>
>>
>> Kernel commit 9eea8fafe33e ("libbpf: fix __arg_ctx type enforcement for
>> perf_event programs") added with 6.9-rc1
>> tools/include/uapi/asm/bpf_perf_event.h
>> which for arc, arm64, loongarch, riscv and s390 includes headers from
>> the arch/${ARCH}/... directories.
>> Thus the build in OE fails as those headers are not present.
>>
>> Fix this by additioanly copying all files in arch/${ARCH}/include/.
>>
>> Fixes:
>> ERROR: perf-1.0-r0 do_compile: oe_runmake failed
>> | In file included from
>> work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/linux/bpf_perf_event.h:11,
>> |                  from libbpf.c:36:
>> |
>> work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/asm/bpf_perf_event.h:2:10:
>> fatal error: ../../arch/arm64/include/uapi/asm/bpf_perf_event.h: No such
>> file or directory
>> |     2 | #include "../../arch/arm64/include/uapi/asm/bpf_perf_event.h"
>> |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>
>> Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
>> ---
>>  meta/recipes-kernel/perf/perf.bb | 1 +
>>  1 file changed, 1 insertion(+)
>>
>> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/
>> perf.bb
>> index 4f26813de0..6408c65462 100644
>> --- a/meta/recipes-kernel/perf/perf.bb
>> +++ b/meta/recipes-kernel/perf/perf.bb
>> @@ -139,6 +139,7 @@ PERF_SRC ?= "Makefile \
>>               tools/scripts \
>>               scripts/ \
>>               arch/arm64/tools \
>> +             arch/${ARCH}/include \
>>
>
> We've always tried to stay within the tools/ source umbrella since we
> really
> don't want to copy any more of the kernel than we have to, since
> otherwise,
> we might as well go back to simplifying things and just building against
> the
> entire kernel source tree.
>
> I'd like to see if there's a way that we could limit this to the uapi (and
> hence
> use our uapi headers), and avoid needing to make the copy. If we need a
> copy, then being as specific as possible would be the goal.
>


I meant to also add that I'm not talking about the libc-headers for the
uapi bits, as they are not tracking the latest kernel by design. Although
it would be interesting to see how perf does treat the older uapi against
the new kernel, so we can document the tested combinations.

Bruce



>
> I'm not moving linux-yocto-dev to the latest -dev until next week, but I
> can
> look into this more then.
>
> Bruce
>
>
>
>>               arch/${ARCH}/Makefile \
>>  "
>>
>> --
>> 2.42.0
>>
>>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await thee
> at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
>

-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await thee
at its end
- "Use the force Harry" - Gandalf, Star Trek II

[-- Attachment #2: Type: text/html, Size: 5838 bytes --]

Re: [OE-core][Patch v1 2/2] perf: tests: fix qa error, missing perl

[Max Krummenacher]

On Tue, Mar 26, 2024 at 6:10 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2024-03-25 at 11:20 +0100, Max Krummenacher wrote:
> > From: Max Krummenacher <max.krummenacher@toradex.com>
> >
> > Kernel commit 61d348f1e96f ("perf testsuite: Add common output checking
> > helpers") added with 6.9-rc1 added addional testscripts written in perl.
> >
> > Fixes:
> > > ERROR: perf-1.0-r0 do_package_qa: QA Issue: /usr/libexec/perf-core/tests/shell/common/check_no_patterns_found.pl contained in package perf-tests requires /usr/bin/perl, but no providers found in RDEPENDS:perf-tests? [file-rdeps]
> >
> > Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
> > ---
> >  meta/recipes-kernel/perf/perf.bb | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> > index 6408c65462..4b6834e614 100644
> > --- a/meta/recipes-kernel/perf/perf.bb
> > +++ b/meta/recipes-kernel/perf/perf.bb
> > @@ -383,7 +383,7 @@ RDEPENDS:${PN} += "elfutils bash"
> >  RDEPENDS:${PN}-archive =+ "bash"
> >  RDEPENDS:${PN}-python =+ "bash python3 python3-modules ${@bb.utils.contains('PACKAGECONFIG', 'audit', 'audit-python', '', d)}"
> >  RDEPENDS:${PN}-perl =+ "bash perl perl-modules"
> > -RDEPENDS:${PN}-tests =+ "python3 bash"
> > +RDEPENDS:${PN}-tests =+ "perl python3 bash"
> >
> >  RSUGGESTS:${PN} += "${PN}-archive ${PN}-tests \
> >                      ${@bb.utils.contains('PACKAGECONFIG', 'perl', '${PN}-perl', '', d)} \
>
> Does perf avoid perl dependencies currently? Would it make more sense
> to put that script in ${PN}-perl?

There's a packageconfig which enables the ${PN}-perl content. Whatever
that stuff is used for.

However kernel 6.9-rc1 added new *.pl files which are called from a
shell script deployed into ${PN}-tests. So I think those new perl
scripts need to go into ${PN}-tests.

Regards
Max

>
> Cheers,
>
> Richard
>
>

Re: [OE-core][Patch v1 1/2] perf: fix build with latest kernel

[Max Krummenacher]

On Tue, Mar 26, 2024 at 6:48 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>
>
>
> On Tue, Mar 26, 2024 at 1:46 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
>>
>>
>>
>> On Mon, Mar 25, 2024 at 6:21 AM <max.oss.09@gmail.com> wrote:
>>>
>>> From: Max Krummenacher <max.krummenacher@toradex.com>
>>>
>>> Kernel commit 9eea8fafe33e ("libbpf: fix __arg_ctx type enforcement for
>>> perf_event programs") added with 6.9-rc1 tools/include/uapi/asm/bpf_perf_event.h
>>> which for arc, arm64, loongarch, riscv and s390 includes headers from
>>> the arch/${ARCH}/... directories.
>>> Thus the build in OE fails as those headers are not present.
>>>
>>> Fix this by additioanly copying all files in arch/${ARCH}/include/.
>>>
>>> Fixes:
>>> ERROR: perf-1.0-r0 do_compile: oe_runmake failed
>>> | In file included from work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/linux/bpf_perf_event.h:11,
>>> |                  from libbpf.c:36:
>>> | work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/asm/bpf_perf_event.h:2:10: fatal error: ../../arch/arm64/include/uapi/asm/bpf_perf_event.h: No such file or directory
>>> |     2 | #include "../../arch/arm64/include/uapi/asm/bpf_perf_event.h"
>>> |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>>>
>>> Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
>>> ---
>>>  meta/recipes-kernel/perf/perf.bb | 1 +
>>>  1 file changed, 1 insertion(+)
>>>
>>> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
>>> index 4f26813de0..6408c65462 100644
>>> --- a/meta/recipes-kernel/perf/perf.bb
>>> +++ b/meta/recipes-kernel/perf/perf.bb
>>> @@ -139,6 +139,7 @@ PERF_SRC ?= "Makefile \
>>>               tools/scripts \
>>>               scripts/ \
>>>               arch/arm64/tools \
>>> +             arch/${ARCH}/include \
>>
>>
>> We've always tried to stay within the tools/ source umbrella since we really
>> don't want to copy any more of the kernel than we have to, since otherwise,
>> we might as well go back to simplifying things and just building against the
>> entire kernel source tree.
>>
>> I'd like to see if there's a way that we could limit this to the uapi (and hence
>> use our uapi headers), and avoid needing to make the copy. If we need a
>> copy, then being as specific as possible would be the goal.

I was too lazy to limit this when I came up with the patch.
Currently the only header missing is
../../arch/arm64/include/uapi/asm/bpf_perf_event.h,
so limiting to arch/${ARCH}/include/uapi should work fine, I'm unsure if
the header file itself exists for every ${ARCH} so only copying the one
file is probably a dangerous way forward.

>
>
>
> I meant to also add that I'm not talking about the libc-headers for the
> uapi bits, as they are not tracking the latest kernel by design. Although
> it would be interesting to see how perf does treat the older uapi against
> the new kernel, so we can document the tested combinations.
>
> Bruce

Let's wait till you give linux-yocto-dev a try run.Happy Easter.
Max
>
>
>>
>>
>> I'm not moving linux-yocto-dev to the latest -dev until next week, but I  can
>> look into this more then.
>>
>> Bruce
>>
>>
>>>
>>>               arch/${ARCH}/Makefile \
>>>  "
>>>
>>> --
>>> 2.42.0
>>>
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>>
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>

Re: [OE-core][Patch v1 1/2] perf: fix build with latest kernel

[Bruce Ashfield]

On Thu, Mar 28, 2024 at 1:55 PM Max Krummenacher <max.oss.09@gmail.com> wrote:
>
> On Tue, Mar 26, 2024 at 6:48 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >
> >
> >
> > On Tue, Mar 26, 2024 at 1:46 PM Bruce Ashfield <bruce.ashfield@gmail.com> wrote:
> >>
> >>
> >>
> >> On Mon, Mar 25, 2024 at 6:21 AM <max.oss.09@gmail.com> wrote:
> >>>
> >>> From: Max Krummenacher <max.krummenacher@toradex.com>
> >>>
> >>> Kernel commit 9eea8fafe33e ("libbpf: fix __arg_ctx type enforcement for
> >>> perf_event programs") added with 6.9-rc1 tools/include/uapi/asm/bpf_perf_event.h
> >>> which for arc, arm64, loongarch, riscv and s390 includes headers from
> >>> the arch/${ARCH}/... directories.
> >>> Thus the build in OE fails as those headers are not present.
> >>>
> >>> Fix this by additioanly copying all files in arch/${ARCH}/include/.
> >>>
> >>> Fixes:
> >>> ERROR: perf-1.0-r0 do_compile: oe_runmake failed
> >>> | In file included from work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/linux/bpf_perf_event.h:11,
> >>> |                  from libbpf.c:36:
> >>> | work/verdin_imx8mm-tdx-linux/perf/1.0/perf-1.0/tools/include/uapi/asm/bpf_perf_event.h:2:10: fatal error: ../../arch/arm64/include/uapi/asm/bpf_perf_event.h: No such file or directory
> >>> |     2 | #include "../../arch/arm64/include/uapi/asm/bpf_perf_event.h"
> >>> |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >>>
> >>> Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
> >>> ---
> >>>  meta/recipes-kernel/perf/perf.bb | 1 +
> >>>  1 file changed, 1 insertion(+)
> >>>
> >>> diff --git a/meta/recipes-kernel/perf/perf.bb b/meta/recipes-kernel/perf/perf.bb
> >>> index 4f26813de0..6408c65462 100644
> >>> --- a/meta/recipes-kernel/perf/perf.bb
> >>> +++ b/meta/recipes-kernel/perf/perf.bb
> >>> @@ -139,6 +139,7 @@ PERF_SRC ?= "Makefile \
> >>>               tools/scripts \
> >>>               scripts/ \
> >>>               arch/arm64/tools \
> >>> +             arch/${ARCH}/include \
> >>
> >>
> >> We've always tried to stay within the tools/ source umbrella since we really
> >> don't want to copy any more of the kernel than we have to, since otherwise,
> >> we might as well go back to simplifying things and just building against the
> >> entire kernel source tree.
> >>
> >> I'd like to see if there's a way that we could limit this to the uapi (and hence
> >> use our uapi headers), and avoid needing to make the copy. If we need a
> >> copy, then being as specific as possible would be the goal.
>
> I was too lazy to limit this when I came up with the patch.
> Currently the only header missing is
> ../../arch/arm64/include/uapi/asm/bpf_perf_event.h,
> so limiting to arch/${ARCH}/include/uapi should work fine, I'm unsure if
> the header file itself exists for every ${ARCH} so only copying the one
> file is probably a dangerous way forward.
>
> >
> >
> >
> > I meant to also add that I'm not talking about the libc-headers for the
> > uapi bits, as they are not tracking the latest kernel by design. Although
> > it would be interesting to see how perf does treat the older uapi against
> > the new kernel, so we can document the tested combinations.
> >
> > Bruce
>
> Let's wait till you give linux-yocto-dev a try run.Happy Easter.

I have linux-yocto-dev on 6.9-rc2 now.

I have a few non-perf packages to fix before I can send it to the list,
but to finish up this thread.

I tested with your patch, as well as the finer grained included, and
that does indeed fix the problem.

At a minimum, we should go with the uapi as the include addition.

I'm tempted to experiment with just the single file, and ensure that
our source copy routines in perf can handle it. The reason why I'm
focused on it, is that this is just temporary until I upgrade our
libc-headers recipe in the next release cycle, after that, it should
provide what is needed for perf and we'll have what is essentially
an extra copy (although harmless).

I agree that the best place for the perl dependency is on the -tests
package, as I don't see big gains on creating a perl-tests package
for the dependency .. since if it is used to process the other results,
most everything would just have the dependency anyway.

Bruce

> Max
> >
> >
> >>
> >>
> >> I'm not moving linux-yocto-dev to the latest -dev until next week, but I  can
> >> look into this more then.
> >>
> >> Bruce
> >>
> >>
> >>>
> >>>               arch/${ARCH}/Makefile \
> >>>  "
> >>>
> >>> --
> >>> 2.42.0
> >>>
> >>
> >>
> >> --
> >> - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
> >> - "Use the force Harry" - Gandalf, Star Trek II
> >>
> >
> >
> > --
> > - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end
> > - "Use the force Harry" - Gandalf, Star Trek II
> >



-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II