Re: [PATCH 21/34] selftests: powerpc: Fix incorrect kernel headers search path

Re: [PATCH 21/34] selftests: powerpc: Fix incorrect kernel headers search path

[Shuah Khan]

On 1/27/23 06:57, Mathieu Desnoyers wrote:
> Use $(KHDR_INCLUDES) as lookup path for kernel headers. This prevents
> building against kernel headers from the build environment in scenarios
> where kernel headers are installed into a specific output directory
> (O=...).
> 
> Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
> Cc: Shuah Khan <shuah@kernel.org>
> Cc: linux-kselftest@vger.kernel.org
> Cc: Ingo Molnar <mingo@redhat.com>
> Cc: <stable@vger.kernel.org>    [5.18+]
> ---
>   tools/testing/selftests/powerpc/ptrace/Makefile   | 2 +-
>   tools/testing/selftests/powerpc/security/Makefile | 2 +-
>   tools/testing/selftests/powerpc/syscalls/Makefile | 2 +-
>   tools/testing/selftests/powerpc/tm/Makefile       | 2 +-
>   4 files changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/tools/testing/selftests/powerpc/ptrace/Makefile b/tools/testing/selftests/powerpc/ptrace/Makefile
> index 2f02cb54224d..cbeeaeae8837 100644
> --- a/tools/testing/selftests/powerpc/ptrace/Makefile
> +++ b/tools/testing/selftests/powerpc/ptrace/Makefile
> @@ -33,7 +33,7 @@ TESTS_64 := $(patsubst %,$(OUTPUT)/%,$(TESTS_64))
>   $(TESTS_64): CFLAGS += -m64
>   $(TM_TESTS): CFLAGS += -I../tm -mhtm
>   
> -CFLAGS += -I../../../../../usr/include -fno-pie
> +CFLAGS += $(KHDR_INCLUDES) -fno-pie
>   
>   $(OUTPUT)/ptrace-gpr: ptrace-gpr.S
>   $(OUTPUT)/ptrace-pkey $(OUTPUT)/core-pkey: LDLIBS += -pthread
> diff --git a/tools/testing/selftests/powerpc/security/Makefile b/tools/testing/selftests/powerpc/security/Makefile
> index 7488315fd847..e0d979ab0204 100644
> --- a/tools/testing/selftests/powerpc/security/Makefile
> +++ b/tools/testing/selftests/powerpc/security/Makefile
> @@ -5,7 +5,7 @@ TEST_PROGS := mitigation-patching.sh
>   
>   top_srcdir = ../../../../..
>   
> -CFLAGS += -I../../../../../usr/include
> +CFLAGS += $(KHDR_INCLUDES)
>   
>   include ../../lib.mk
>   
> diff --git a/tools/testing/selftests/powerpc/syscalls/Makefile b/tools/testing/selftests/powerpc/syscalls/Makefile
> index b63f8459c704..d1f2648b112b 100644
> --- a/tools/testing/selftests/powerpc/syscalls/Makefile
> +++ b/tools/testing/selftests/powerpc/syscalls/Makefile
> @@ -1,7 +1,7 @@
>   # SPDX-License-Identifier: GPL-2.0-only
>   TEST_GEN_PROGS := ipc_unmuxed rtas_filter
>   
> -CFLAGS += -I../../../../../usr/include
> +CFLAGS += $(KHDR_INCLUDES)
>   
>   top_srcdir = ../../../../..
>   include ../../lib.mk
> diff --git a/tools/testing/selftests/powerpc/tm/Makefile b/tools/testing/selftests/powerpc/tm/Makefile
> index 5881e97c73c1..3876805c2f31 100644
> --- a/tools/testing/selftests/powerpc/tm/Makefile
> +++ b/tools/testing/selftests/powerpc/tm/Makefile
> @@ -17,7 +17,7 @@ $(TEST_GEN_PROGS): ../harness.c ../utils.c
>   CFLAGS += -mhtm
>   
>   $(OUTPUT)/tm-syscall: tm-syscall-asm.S
> -$(OUTPUT)/tm-syscall: CFLAGS += -I../../../../../usr/include
> +$(OUTPUT)/tm-syscall: CFLAGS += $(KHDR_INCLUDES)
>   $(OUTPUT)/tm-tmspr: CFLAGS += -pthread
>   $(OUTPUT)/tm-vmx-unavail: CFLAGS += -pthread -m64
>   $(OUTPUT)/tm-resched-dscr: ../pmu/lib.c

Adding powerpc maitainers.

Would you me to take this patch through kselftest tree? If you
decide to take this through yours:

Acked-by: Shuah Khan <skhan@linuxfoundation.org>

thanks,
-- Shuah

Re: [PATCH 00/34] selftests: Fix incorrect kernel headers search path

[Shuah Khan]

On 1/27/23 06:57, Mathieu Desnoyers wrote:
> Hi,
> 
> This series fixes incorrect kernel header search path in kernel
> selftests.
> 
> Near the end of the series, a few changes are not tagged as "Fixes"
> because the current behavior is to rely on the kernel sources uapi files
> rather than on the installed kernel header files. Nevertheless, those
> are updated for consistency.
> 
> There are situations where "../../../../include/" was added to -I search
> path, which is bogus for userspace tests and caused issues with types.h.
> Those are removed.
> 
> Thanks,
> 
> Mathieu
> 
> Mathieu Desnoyers (34):

The below patches are now applied to linux-kselftest next for Linux 6.3-rc1

>    selftests: arm64: Fix incorrect kernel headers search path
>    selftests: clone3: Fix incorrect kernel headers search path
>    selftests: core: Fix incorrect kernel headers search path
>    selftests: dma: Fix incorrect kernel headers search path
>    selftests: dmabuf-heaps: Fix incorrect kernel headers search path
>    selftests: drivers: Fix incorrect kernel headers search path
>    selftests: filesystems: Fix incorrect kernel headers search path
>    selftests: futex: Fix incorrect kernel headers search path
>    selftests: gpio: Fix incorrect kernel headers search path
>    selftests: ipc: Fix incorrect kernel headers search path
>    selftests: kcmp: Fix incorrect kernel headers search path
>    selftests: media_tests: Fix incorrect kernel headers search path
>    selftests: membarrier: Fix incorrect kernel headers search path
>    selftests: mount_setattr: Fix incorrect kernel headers search path
>    selftests: move_mount_set_group: Fix incorrect kernel headers search
>      path
>    selftests: perf_events: Fix incorrect kernel headers search path
>    selftests: pid_namespace: Fix incorrect kernel headers search path
>    selftests: pidfd: Fix incorrect kernel headers search path
>    selftests: ptp: Fix incorrect kernel headers search path
>    selftests: rseq: Fix incorrect kernel headers search path
>    selftests: sched: Fix incorrect kernel headers search path
>    selftests: seccomp: Fix incorrect kernel headers search path
>    selftests: sync: Fix incorrect kernel headers search path
>    selftests: user_events: Fix incorrect kernel headers search path
>    selftests: vm: Fix incorrect kernel headers search path
>    selftests: x86: Fix incorrect kernel headers search path
>    selftests: iommu: Use installed kernel headers search path
>    selftests: memfd: Use installed kernel headers search path
>    selftests: ptrace: Use installed kernel headers search path
>    selftests: tdx: Use installed kernel headers search path
> 

These will be applied by maintainers to their trees.

>    selftests: bpf: Fix incorrect kernel headers search path # 02/34
>    selftests: net: Fix incorrect kernel headers search path # 17/34
>    selftests: powerpc: Fix incorrect kernel headers search path # 21/34
>    selftests: bpf docs: Use installed kernel headers search path # 30/34

thanks,
-- Shuah

Re: [PATCH 00/34] selftests: Fix incorrect kernel headers search path

[Alexei Starovoitov]

On Mon, Jan 30, 2023 at 2:46 PM Shuah Khan <skhan@linuxfoundation.org> wrote:
>
> On 1/27/23 06:57, Mathieu Desnoyers wrote:
> > Hi,
> >
> > This series fixes incorrect kernel header search path in kernel
> > selftests.
> >
> > Near the end of the series, a few changes are not tagged as "Fixes"
> > because the current behavior is to rely on the kernel sources uapi files
> > rather than on the installed kernel header files. Nevertheless, those
> > are updated for consistency.
> >
> > There are situations where "../../../../include/" was added to -I search
> > path, which is bogus for userspace tests and caused issues with types.h.
> > Those are removed.
> >
> > Thanks,
> >
> > Mathieu
> >
> > Mathieu Desnoyers (34):
>
> The below patches are now applied to linux-kselftest next for Linux 6.3-rc1
>
> >    selftests: arm64: Fix incorrect kernel headers search path
> >    selftests: clone3: Fix incorrect kernel headers search path
> >    selftests: core: Fix incorrect kernel headers search path
> >    selftests: dma: Fix incorrect kernel headers search path
> >    selftests: dmabuf-heaps: Fix incorrect kernel headers search path
> >    selftests: drivers: Fix incorrect kernel headers search path
> >    selftests: filesystems: Fix incorrect kernel headers search path
> >    selftests: futex: Fix incorrect kernel headers search path
> >    selftests: gpio: Fix incorrect kernel headers search path
> >    selftests: ipc: Fix incorrect kernel headers search path
> >    selftests: kcmp: Fix incorrect kernel headers search path
> >    selftests: media_tests: Fix incorrect kernel headers search path
> >    selftests: membarrier: Fix incorrect kernel headers search path
> >    selftests: mount_setattr: Fix incorrect kernel headers search path
> >    selftests: move_mount_set_group: Fix incorrect kernel headers search
> >      path
> >    selftests: perf_events: Fix incorrect kernel headers search path
> >    selftests: pid_namespace: Fix incorrect kernel headers search path
> >    selftests: pidfd: Fix incorrect kernel headers search path
> >    selftests: ptp: Fix incorrect kernel headers search path
> >    selftests: rseq: Fix incorrect kernel headers search path
> >    selftests: sched: Fix incorrect kernel headers search path
> >    selftests: seccomp: Fix incorrect kernel headers search path
> >    selftests: sync: Fix incorrect kernel headers search path
> >    selftests: user_events: Fix incorrect kernel headers search path
> >    selftests: vm: Fix incorrect kernel headers search path
> >    selftests: x86: Fix incorrect kernel headers search path
> >    selftests: iommu: Use installed kernel headers search path
> >    selftests: memfd: Use installed kernel headers search path
> >    selftests: ptrace: Use installed kernel headers search path
> >    selftests: tdx: Use installed kernel headers search path
> >
>
> These will be applied by maintainers to their trees.

Not in this form. They break the build.

> >    selftests: bpf: Fix incorrect kernel headers search path # 02/34
> >    selftests: net: Fix incorrect kernel headers search path # 17/34
> >    selftests: powerpc: Fix incorrect kernel headers search path # 21/34
> >    selftests: bpf docs: Use installed kernel headers search path # 30/34
>
> thanks,
> -- Shuah

Re: [PATCH 00/34] selftests: Fix incorrect kernel headers search path

[Shuah Khan]

On 1/30/23 16:45, Alexei Starovoitov wrote:
> On Mon, Jan 30, 2023 at 2:46 PM Shuah Khan <skhan@linuxfoundation.org> wrote:
>>
>> On 1/27/23 06:57, Mathieu Desnoyers wrote:
>>> Hi,
>>>
>>> This series fixes incorrect kernel header search path in kernel
>>> selftests.
>>>
>>> Near the end of the series, a few changes are not tagged as "Fixes"
>>> because the current behavior is to rely on the kernel sources uapi files
>>> rather than on the installed kernel header files. Nevertheless, those
>>> are updated for consistency.
>>>
>>> There are situations where "../../../../include/" was added to -I search
>>> path, which is bogus for userspace tests and caused issues with types.h.
>>> Those are removed.
>>>
>>> Thanks,
>>>
>>> Mathieu
>>>
>>> Mathieu Desnoyers (34):
>>
>> The below patches are now applied to linux-kselftest next for Linux 6.3-rc1
>>
>>>     selftests: arm64: Fix incorrect kernel headers search path
>>>     selftests: clone3: Fix incorrect kernel headers search path
>>>     selftests: core: Fix incorrect kernel headers search path
>>>     selftests: dma: Fix incorrect kernel headers search path
>>>     selftests: dmabuf-heaps: Fix incorrect kernel headers search path
>>>     selftests: drivers: Fix incorrect kernel headers search path
>>>     selftests: filesystems: Fix incorrect kernel headers search path
>>>     selftests: futex: Fix incorrect kernel headers search path
>>>     selftests: gpio: Fix incorrect kernel headers search path
>>>     selftests: ipc: Fix incorrect kernel headers search path
>>>     selftests: kcmp: Fix incorrect kernel headers search path
>>>     selftests: media_tests: Fix incorrect kernel headers search path
>>>     selftests: membarrier: Fix incorrect kernel headers search path
>>>     selftests: mount_setattr: Fix incorrect kernel headers search path
>>>     selftests: move_mount_set_group: Fix incorrect kernel headers search
>>>       path
>>>     selftests: perf_events: Fix incorrect kernel headers search path
>>>     selftests: pid_namespace: Fix incorrect kernel headers search path
>>>     selftests: pidfd: Fix incorrect kernel headers search path
>>>     selftests: ptp: Fix incorrect kernel headers search path
>>>     selftests: rseq: Fix incorrect kernel headers search path
>>>     selftests: sched: Fix incorrect kernel headers search path
>>>     selftests: seccomp: Fix incorrect kernel headers search path
>>>     selftests: sync: Fix incorrect kernel headers search path
>>>     selftests: user_events: Fix incorrect kernel headers search path
>>>     selftests: vm: Fix incorrect kernel headers search path
>>>     selftests: x86: Fix incorrect kernel headers search path
>>>     selftests: iommu: Use installed kernel headers search path
>>>     selftests: memfd: Use installed kernel headers search path
>>>     selftests: ptrace: Use installed kernel headers search path
>>>     selftests: tdx: Use installed kernel headers search path
>>>
>>
>> These will be applied by maintainers to their trees.
> 
> Not in this form. They break the build.

Mathieu is sending you the patches in the format you requested in
the thread on this patch.

thanks,
-- Shuah

Re: [PATCH 00/34] selftests: Fix incorrect kernel headers search path

[Alexei Starovoitov]

On Mon, Jan 30, 2023 at 3:48 PM Shuah Khan <skhan@linuxfoundation.org> wrote:
>
> >>
> >> These will be applied by maintainers to their trees.
> >
> > Not in this form. They break the build.
>
> Mathieu is sending you the patches in the format you requested in
> the thread on this patch.

It's not the format, but the patch itself is incorrect.

Re: [PATCH 00/34] selftests: Fix incorrect kernel headers search path

[Shuah Khan]

Hi Mathieu,

On 1/30/23 15:29, Shuah Khan wrote:
> On 1/27/23 06:57, Mathieu Desnoyers wrote:
>> Hi,
>>
>> This series fixes incorrect kernel header search path in kernel
>> selftests.
>>
>> Near the end of the series, a few changes are not tagged as "Fixes"
>> because the current behavior is to rely on the kernel sources uapi files
>> rather than on the installed kernel header files. Nevertheless, those
>> are updated for consistency.
>>
>> There are situations where "../../../../include/" was added to -I search
>> path, which is bogus for userspace tests and caused issues with types.h.
>> Those are removed.
>>

Thanks again for taking care of this. I did out of tree build testing on
x86 on linux-kselftest next with these patches below. I haven't seen
any problems introduced by the patch set.

>>    selftests: dma: Fix incorrect kernel headers search path
This one needs a change and I will send a patch on top of yours.
Even with that this test depends on unexported header from the
repo and won't build out of tree. This is not related to your
change.

>>    selftests: mount_setattr: Fix incorrect kernel headers search path
This one fails to build with our without patch - an existing error.

I have to do cross-build tests on arm64 and other arch patches still.
This will happen later this week.

>>    selftests: arm64: Fix incorrect kernel headers search path

drivers patch below had arch specific tests - testing todo

The rest looks good. I will try to run bpf patches on my system.
I do have clang, llvm installed on mine. TODO

thanks,
-- Shuah

Re: [PATCH 00/34] selftests: Fix incorrect kernel headers search path

[Shuah Khan]

On 2/1/23 19:07, Shuah Khan wrote:
> Hi Mathieu,
> 
> On 1/30/23 15:29, Shuah Khan wrote:
>> On 1/27/23 06:57, Mathieu Desnoyers wrote:
>>> Hi,
>>>
>>> This series fixes incorrect kernel header search path in kernel
>>> selftests.
>>>
>>> Near the end of the series, a few changes are not tagged as "Fixes"
>>> because the current behavior is to rely on the kernel sources uapi files
>>> rather than on the installed kernel header files. Nevertheless, those
>>> are updated for consistency.
>>>
>>> There are situations where "../../../../include/" was added to -I search
>>> path, which is bogus for userspace tests and caused issues with types.h.
>>> Those are removed.
>>>
> 
> Thanks again for taking care of this. I did out of tree build testing on
> x86 on linux-kselftest next with these patches below. I haven't seen
> any problems introduced by the patch set.
> 
>>>    selftests: dma: Fix incorrect kernel headers search path
> This one needs a change and I will send a patch on top of yours.
> Even with that this test depends on unexported header from the
> repo and won't build out of tree. This is not related to your
> change.
> 
>>>    selftests: mount_setattr: Fix incorrect kernel headers search path
> This one fails to build with our without patch - an existing error.
> 
> I have to do cross-build tests on arm64 and other arch patches still.
> This will happen later this week.

arm64, s390 patches look good.

thanks,
-- Shuah

Re: [PATCH 00/34] selftests: Fix incorrect kernel headers search path

[Shuah Khan]

On 2/3/23 18:06, Shuah Khan wrote:
> On 2/1/23 19:07, Shuah Khan wrote:
>> Hi Mathieu,
>>
>> On 1/30/23 15:29, Shuah Khan wrote:
>>> On 1/27/23 06:57, Mathieu Desnoyers wrote:
>>>> Hi,
>>>>
>>>> This series fixes incorrect kernel header search path in kernel
>>>> selftests.
>>>>
>>>> Near the end of the series, a few changes are not tagged as "Fixes"
>>>> because the current behavior is to rely on the kernel sources uapi files
>>>> rather than on the installed kernel header files. Nevertheless, those
>>>> are updated for consistency.
>>>>
>>>> There are situations where "../../../../include/" was added to -I search
>>>> path, which is bogus for userspace tests and caused issues with types.h.
>>>> Those are removed.
>>>>
>>
>> Thanks again for taking care of this. I did out of tree build testing on
>> x86 on linux-kselftest next with these patches below. I haven't seen
>> any problems introduced by the patch set.
>>
>>>>    selftests: dma: Fix incorrect kernel headers search path
>> This one needs a change and I will send a patch on top of yours.
>> Even with that this test depends on unexported header from the
>> repo and won't build out of tree. This is not related to your
>> change.
>>
>>>>    selftests: mount_setattr: Fix incorrect kernel headers search path
>> This one fails to build with our without patch - an existing error.
>>
>> I have to do cross-build tests on arm64 and other arch patches still.
>> This will happen later this week.
> 
> arm64, s390 patches look good.
> 

I am seeing problem with selftests/dma and selfttests/user_events.

1. selftests: dma: Fix incorrect kernel headers search path

dma test no longer builds. This test depends on linux/map_benchmark.h
which is not included in uapi

The order of include directorries -isystem followed by installed kernel
headers, breaks the test build with the change to use KHDR_INCLUDES


I am going to revert this patch for now and figure a longer term fix.
The problem is the dependency on a non-uapi file: linux/map_benchmark.h

Fixes: 8ddde07a3d28 ("dma-mapping: benchmark: extract a common
header file for map_benchmark definition") change added this
dependency on including linux/map_benchmark.h

Christoph, Do you see this map_benchmark.h as part of uapi?


2. selftests: user_events: Fix incorrect kernel headers search path
This one depends on linux/user_events.h which has bee removed from
uapi in this commit:

commit 5cfff569cab8bf544bab62c911c5d6efd5af5e05
Author: Steven Rostedt (Google) <rostedt@goodmis.org>
Date:   Fri Apr 1 14:39:03 2022 -0400

     tracing: Move user_events.h temporarily out of include/uapi

This isn't a regression from 6.2 - this test stopped building once
user_events.h has been removed from uapi. I will add a note that
this test depends on a non-uapi header and can't be built at the
moment.

thanks,
-- Shuah