* kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set @ 2019-11-12 16:33 Cristian Marussi 2019-11-27 3:54 ` Michael Ellerman 0 siblings, 1 reply; 14+ messages in thread From: Cristian Marussi @ 2019-11-12 16:33 UTC (permalink / raw) To: Tim.Bird, shuah, linux-kselftest Hi while testing on linux-next I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install fails miserably with: ... ... REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h HDRINST usr/include/asm/kvm.h INSTALL /kselftest/usr/include mkdir: cannot create directory ‘/kselftest’: Permission denied /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed make[2]: *** [headers_install] Error 1 This is fixed by unsetting KBUILD_OUTPUT OR reverting: 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths since bypassing top makefile with -C, the definition of abs-objtree used by the above patch is no more available. As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ ARCH=arm64 -C ../../.. headers_install make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' INSTALL /kselftest/usr/include mkdir: cannot create directory ‘/kselftest’: Permission denied /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed make[2]: *** [headers_install] Error 1 make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' Makefile:179: recipe for target 'sub-make' failed make[1]: *** [sub-make] Error 2 make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' Makefile:142: recipe for target 'khdr' failed make: *** [khdr] Error 2 A possible fix would be (but duplicates in fact the main Makefile logic) diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 319e094c3212..491d8b3ef1c7 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -88,6 +88,15 @@ ifdef building_out_of_srctree override LDFLAGS = endif +ifeq ($(abs_objtree),) +ifneq ($(KBUILD_OUTPUT),) +abs_objtree := $(shell cd $(KBUILD_OUTPUT) && pwd) +abs_objtree := $(realpath $(abs_objtree)) +else +abs_objtree := $(shell pwd) +endif +endif #ifeq ($(abs_objtree),) + ifneq ($(O),) BUILD := $(abs_objtree) else Any thoughts ? ... or am I missing something ? (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) Thanks Cristian ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-12 16:33 kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set Cristian Marussi @ 2019-11-27 3:54 ` Michael Ellerman 2019-11-27 10:54 ` Cristian Marussi 0 siblings, 1 reply; 14+ messages in thread From: Michael Ellerman @ 2019-11-27 3:54 UTC (permalink / raw) To: Cristian Marussi, Tim.Bird, shuah, linux-kselftest Cristian Marussi <cristian.marussi@arm.com> writes: > Hi > > while testing on linux-next > > I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) > > $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install > > works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) > > make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install > > fails miserably with: > ... > ... > REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h > HDRINST usr/include/asm/kvm.h > INSTALL /kselftest/usr/include > mkdir: cannot create directory ‘/kselftest’: Permission denied > /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed > make[2]: *** [headers_install] Error 1 > > > This is fixed by unsetting KBUILD_OUTPUT OR reverting: > > 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths > > since bypassing top makefile with -C, the definition of abs-objtree used by the above patch > is no more available. > > As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. > > $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next > ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... > make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ > ARCH=arm64 -C ../../.. headers_install > make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' > make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' > INSTALL /kselftest/usr/include > mkdir: cannot create directory ‘/kselftest’: Permission denied > /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed > make[2]: *** [headers_install] Error 1 > make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' > Makefile:179: recipe for target 'sub-make' failed > make[1]: *** [sub-make] Error 2 > make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' > Makefile:142: recipe for target 'khdr' failed > make: *** [khdr] Error 2 > > ... > Any thoughts ? ... or am I missing something ? You're not missing anything, this is broken. > (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) It just hit my travis jobs when I merged up to master: https://travis-ci.org/linuxppc/linux/jobs/617482001 Shuah can we please get this reverted? cheers ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 3:54 ` Michael Ellerman @ 2019-11-27 10:54 ` Cristian Marussi 2019-11-27 16:04 ` shuah 0 siblings, 1 reply; 14+ messages in thread From: Cristian Marussi @ 2019-11-27 10:54 UTC (permalink / raw) To: Michael Ellerman, Tim.Bird, shuah, linux-kselftest Hi On 27/11/2019 03:54, Michael Ellerman wrote: > Cristian Marussi <cristian.marussi@arm.com> writes: >> Hi >> >> while testing on linux-next >> >> I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) >> >> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install >> >> works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) >> >> make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install >> >> fails miserably with: >> ... >> ... >> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h >> HDRINST usr/include/asm/kvm.h >> INSTALL /kselftest/usr/include >> mkdir: cannot create directory ‘/kselftest’: Permission denied >> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >> make[2]: *** [headers_install] Error 1 >> >> >> This is fixed by unsetting KBUILD_OUTPUT OR reverting: >> >> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths >> >> since bypassing top makefile with -C, the definition of abs-objtree used by the above patch >> is no more available. >> >> As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. >> >> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next >> ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... >> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >> ARCH=arm64 -C ../../.. headers_install >> make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' >> make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >> INSTALL /kselftest/usr/include >> mkdir: cannot create directory ‘/kselftest’: Permission denied >> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >> make[2]: *** [headers_install] Error 1 >> make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >> Makefile:179: recipe for target 'sub-make' failed >> make[1]: *** [sub-make] Error 2 >> make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' >> Makefile:142: recipe for target 'khdr' failed >> make: *** [khdr] Error 2 >> >> > ... >> Any thoughts ? ... or am I missing something ? > > You're not missing anything, this is broken. Thanks for the feedback ! Cristian > >> (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) > > It just hit my travis jobs when I merged up to master: > > https://travis-ci.org/linuxppc/linux/jobs/617482001 > > Shuah can we please get this reverted? > > cheers > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 10:54 ` Cristian Marussi @ 2019-11-27 16:04 ` shuah 2019-11-27 16:23 ` Cristian Marussi 0 siblings, 1 reply; 14+ messages in thread From: shuah @ 2019-11-27 16:04 UTC (permalink / raw) To: Cristian Marussi, Michael Ellerman, Tim.Bird, linux-kselftest, shuah On 11/27/19 3:54 AM, Cristian Marussi wrote: > Hi > > On 27/11/2019 03:54, Michael Ellerman wrote: >> Cristian Marussi <cristian.marussi@arm.com> writes: >>> Hi >>> >>> while testing on linux-next >>> >>> I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) >>> >>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install >>> >>> works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) >>> >>> make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install >>> >>> fails miserably with: >>> ... >>> ... >>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h >>> HDRINST usr/include/asm/kvm.h >>> INSTALL /kselftest/usr/include >>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>> make[2]: *** [headers_install] Error 1 >>> >>> >>> This is fixed by unsetting KBUILD_OUTPUT OR reverting: >>> >>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths >>> >>> since bypassing top makefile with -C, the definition of abs-objtree used by the above patch >>> is no more available. >>> >>> As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. >>> >>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next >>> ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... >>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >>> ARCH=arm64 -C ../../.. headers_install >>> make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>> make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>> INSTALL /kselftest/usr/include >>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>> make[2]: *** [headers_install] Error 1 >>> make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>> Makefile:179: recipe for target 'sub-make' failed >>> make[1]: *** [sub-make] Error 2 >>> make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>> Makefile:142: recipe for target 'khdr' failed >>> make: *** [khdr] Error 2 >>> >>> >> ... >>> Any thoughts ? ... or am I missing something ? >> >> You're not missing anything, this is broken. > Thanks for the feedback ! > > Cristian > >> >>> (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) >> >> It just hit my travis jobs when I merged up to master: >> >> https://travis-ci.org/linuxppc/linux/jobs/617482001 >> >> Shuah can we please get this reverted? >> I will take care of this. Supporting all these use-cases has been maintenance problem. Fixing one use-case breaks another. :( I will look into fixing this once for all. thanks, -- Shuah ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 16:04 ` shuah @ 2019-11-27 16:23 ` Cristian Marussi 2019-11-27 16:42 ` shuah 0 siblings, 1 reply; 14+ messages in thread From: Cristian Marussi @ 2019-11-27 16:23 UTC (permalink / raw) To: shuah, Michael Ellerman, Tim.Bird, linux-kselftest On 27/11/2019 16:04, shuah wrote: > On 11/27/19 3:54 AM, Cristian Marussi wrote: >> Hi >> >> On 27/11/2019 03:54, Michael Ellerman wrote: >>> Cristian Marussi <cristian.marussi@arm.com> writes: >>>> Hi >>>> >>>> while testing on linux-next >>>> >>>> I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) >>>> >>>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install >>>> >>>> works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) >>>> >>>> make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install >>>> >>>> fails miserably with: >>>> ... >>>> ... >>>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h >>>> HDRINST usr/include/asm/kvm.h >>>> INSTALL /kselftest/usr/include >>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>> make[2]: *** [headers_install] Error 1 >>>> >>>> >>>> This is fixed by unsetting KBUILD_OUTPUT OR reverting: >>>> >>>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths >>>> >>>> since bypassing top makefile with -C, the definition of abs-objtree used by the above patch >>>> is no more available. >>>> >>>> As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. >>>> >>>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next >>>> ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... >>>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >>>> ARCH=arm64 -C ../../.. headers_install >>>> make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>> make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>> INSTALL /kselftest/usr/include >>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>> make[2]: *** [headers_install] Error 1 >>>> make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>> Makefile:179: recipe for target 'sub-make' failed >>>> make[1]: *** [sub-make] Error 2 >>>> make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>> Makefile:142: recipe for target 'khdr' failed >>>> make: *** [khdr] Error 2 >>>> >>>> >>> ... >>>> Any thoughts ? ... or am I missing something ? >>> >>> You're not missing anything, this is broken. >> Thanks for the feedback ! >> >> Cristian >> >>> >>>> (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) >>> >>> It just hit my travis jobs when I merged up to master: >>> >>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>> >>> Shuah can we please get this reverted? >>> > > I will take care of this. Supporting all these use-cases has been > maintenance problem. Fixing one use-case breaks another. :( > > I will look into fixing this once for all. Thanks. In fact all of the above works fine with the new kselftest-install target you added recently but I think a lot of CI is still using the old -C method. Regards Cristian > > thanks, > -- Shuah > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 16:23 ` Cristian Marussi @ 2019-11-27 16:42 ` shuah 2019-11-27 16:53 ` Cristian Marussi 2019-11-28 4:20 ` Michael Ellerman 0 siblings, 2 replies; 14+ messages in thread From: shuah @ 2019-11-27 16:42 UTC (permalink / raw) To: Cristian Marussi, Michael Ellerman, Tim.Bird, linux-kselftest, shuah On 11/27/19 9:23 AM, Cristian Marussi wrote: > On 27/11/2019 16:04, shuah wrote: >> On 11/27/19 3:54 AM, Cristian Marussi wrote: >>> Hi >>> >>> On 27/11/2019 03:54, Michael Ellerman wrote: >>>> Cristian Marussi <cristian.marussi@arm.com> writes: >>>>> Hi >>>>> >>>>> while testing on linux-next >>>>> >>>>> I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) >>>>> >>>>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install >>>>> >>>>> works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) >>>>> >>>>> make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install >>>>> >>>>> fails miserably with: >>>>> ... >>>>> ... >>>>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h >>>>> HDRINST usr/include/asm/kvm.h >>>>> INSTALL /kselftest/usr/include >>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>>> make[2]: *** [headers_install] Error 1 >>>>> >>>>> >>>>> This is fixed by unsetting KBUILD_OUTPUT OR reverting: >>>>> >>>>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths >>>>> >>>>> since bypassing top makefile with -C, the definition of abs-objtree used by the above patch >>>>> is no more available. >>>>> >>>>> As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. >>>>> >>>>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next >>>>> ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... >>>>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >>>>> ARCH=arm64 -C ../../.. headers_install >>>>> make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>> make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>> INSTALL /kselftest/usr/include >>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>>> make[2]: *** [headers_install] Error 1 >>>>> make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>> Makefile:179: recipe for target 'sub-make' failed >>>>> make[1]: *** [sub-make] Error 2 >>>>> make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>> Makefile:142: recipe for target 'khdr' failed >>>>> make: *** [khdr] Error 2 >>>>> >>>>> >>>> ... >>>>> Any thoughts ? ... or am I missing something ? >>>> >>>> You're not missing anything, this is broken. >>> Thanks for the feedback ! >>> >>> Cristian >>> >>>> >>>>> (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) >>>> >>>> It just hit my travis jobs when I merged up to master: >>>> >>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>>> >>>> Shuah can we please get this reverted? >>>> >> >> I will take care of this. Supporting all these use-cases has been >> maintenance problem. Fixing one use-case breaks another. :( >> >> I will look into fixing this once for all. > > Thanks. > > In fact all of the above works fine with the new kselftest-install target you added recently > but I think a lot of CI is still using the old -C method. > > Regards If all of the above works with kselftest-install target, I would rather ask for CI's to change, so we can get rid of make -C usage for install. In which case, I would rather not revert this and request CIs to start using kselftest-install target. I would like to see CI's use kselftest-* targets from the main makefile for build/run/install. It has been very difficult to support "make -C" use-cases in general and especially for install and relocatable builds. thanks, -- Shuah ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 16:42 ` shuah @ 2019-11-27 16:53 ` Cristian Marussi 2019-11-27 17:32 ` shuah 2019-11-28 4:20 ` Michael Ellerman 1 sibling, 1 reply; 14+ messages in thread From: Cristian Marussi @ 2019-11-27 16:53 UTC (permalink / raw) To: shuah, Michael Ellerman, Tim.Bird, linux-kselftest On 27/11/2019 16:42, shuah wrote: > On 11/27/19 9:23 AM, Cristian Marussi wrote: >> On 27/11/2019 16:04, shuah wrote: >>> On 11/27/19 3:54 AM, Cristian Marussi wrote: >>>> Hi >>>> >>>> On 27/11/2019 03:54, Michael Ellerman wrote: >>>>> Cristian Marussi <cristian.marussi@arm.com> writes: >>>>>> Hi >>>>>> >>>>>> while testing on linux-next >>>>>> >>>>>> I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) >>>>>> >>>>>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install >>>>>> >>>>>> works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) >>>>>> >>>>>> make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install >>>>>> >>>>>> fails miserably with: >>>>>> ... >>>>>> ... >>>>>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h >>>>>> HDRINST usr/include/asm/kvm.h >>>>>> INSTALL /kselftest/usr/include >>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>>>> make[2]: *** [headers_install] Error 1 >>>>>> >>>>>> >>>>>> This is fixed by unsetting KBUILD_OUTPUT OR reverting: >>>>>> >>>>>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths >>>>>> >>>>>> since bypassing top makefile with -C, the definition of abs-objtree used by the above patch >>>>>> is no more available. >>>>>> >>>>>> As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. >>>>>> >>>>>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next >>>>>> ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... >>>>>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >>>>>> ARCH=arm64 -C ../../.. headers_install >>>>>> make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>> make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>> INSTALL /kselftest/usr/include >>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>>>> make[2]: *** [headers_install] Error 1 >>>>>> make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>> Makefile:179: recipe for target 'sub-make' failed >>>>>> make[1]: *** [sub-make] Error 2 >>>>>> make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>> Makefile:142: recipe for target 'khdr' failed >>>>>> make: *** [khdr] Error 2 >>>>>> >>>>>> >>>>> ... >>>>>> Any thoughts ? ... or am I missing something ? >>>>> >>>>> You're not missing anything, this is broken. >>>> Thanks for the feedback ! >>>> >>>> Cristian >>>> >>>>> >>>>>> (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) >>>>> >>>>> It just hit my travis jobs when I merged up to master: >>>>> >>>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>>>> >>>>> Shuah can we please get this reverted? >>>>> >>> >>> I will take care of this. Supporting all these use-cases has been >>> maintenance problem. Fixing one use-case breaks another. :( >>> >>> I will look into fixing this once for all. >> >> Thanks. >> >> In fact all of the above works fine with the new kselftest-install target you added recently >> but I think a lot of CI is still using the old -C method. >> >> Regards > > If all of the above works with kselftest-install target, I would rather > ask for CI's to change, so we can get rid of make -C usage for install. > > In which case, I would rather not revert this and request CIs to start > using kselftest-install target. > > > I would like to see CI's use kselftest-* targets from the main makefile > for build/run/install. > > It has been very difficult to support "make -C" use-cases in general > and especially for install and relocatable builds. > Well I'm not sure if it covers all the build cases/scenario for CIs and I have only verified that this specific failure seems to work using the toplevel Makefile targets (since the offending commit uses abs-objtree). Anyway $ ./kselftest_install.sh is broken too as of now. Thanks Cristian > thanks, > -- Shuah > ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 16:53 ` Cristian Marussi @ 2019-11-27 17:32 ` shuah 2019-11-27 17:57 ` Tim.Bird 0 siblings, 1 reply; 14+ messages in thread From: shuah @ 2019-11-27 17:32 UTC (permalink / raw) To: Cristian Marussi, Michael Ellerman, Tim.Bird, linux-kselftest, shuah On 11/27/19 9:53 AM, Cristian Marussi wrote: > On 27/11/2019 16:42, shuah wrote: >> On 11/27/19 9:23 AM, Cristian Marussi wrote: >>> On 27/11/2019 16:04, shuah wrote: >>>> On 11/27/19 3:54 AM, Cristian Marussi wrote: >>>>> Hi >>>>> >>>>> On 27/11/2019 03:54, Michael Ellerman wrote: >>>>>> Cristian Marussi <cristian.marussi@arm.com> writes: >>>>>>> Hi >>>>>>> >>>>>>> while testing on linux-next >>>>>>> >>>>>>> I see that, when KBUILD_OUTPUT is set in the env, running something like (using TARGETS=exec as a random subsystem here...) >>>>>>> >>>>>>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next kselftest-install >>>>>>> >>>>>>> works fine as usual, WHILE the alternative invocation (still documented in Documentation/dev-tools/kselftest.rst) >>>>>>> >>>>>>> make -C tools/testing/selftests/ TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next install >>>>>>> >>>>>>> fails miserably with: >>>>>>> ... >>>>>>> ... >>>>>>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes-abi.h >>>>>>> HDRINST usr/include/asm/kvm.h >>>>>>> INSTALL /kselftest/usr/include >>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>>>>> make[2]: *** [headers_install] Error 1 >>>>>>> >>>>>>> >>>>>>> This is fixed by unsetting KBUILD_OUTPUT OR reverting: >>>>>>> >>>>>>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT handling for relative paths >>>>>>> >>>>>>> since bypassing top makefile with -C, the definition of abs-objtree used by the above patch >>>>>>> is no more available. >>>>>>> >>>>>>> As a side effect when KBUILD_OUTPUT is set, this breaks also the usage kselftest_install.sh. >>>>>>> >>>>>>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next >>>>>>> ./kselftest_install.sh: Installing in specified location - /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_full_next ... >>>>>>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >>>>>>> ARCH=arm64 -C ../../.. headers_install >>>>>>> make[1]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>>> make[2]: Entering directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>>> INSTALL /kselftest/usr/include >>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe for target 'headers_install' failed >>>>>>> make[2]: *** [headers_install] Error 1 >>>>>>> make[2]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>>> Makefile:179: recipe for target 'sub-make' failed >>>>>>> make[1]: *** [sub-make] Error 2 >>>>>>> make[1]: Leaving directory '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>>> Makefile:142: recipe for target 'khdr' failed >>>>>>> make: *** [khdr] Error 2 >>>>>>> >>>>>>> >>>>>> ... >>>>>>> Any thoughts ? ... or am I missing something ? >>>>>> >>>>>> You're not missing anything, this is broken. >>>>> Thanks for the feedback ! >>>>> >>>>> Cristian >>>>> >>>>>> >>>>>>> (I think I'm starting to see this in latest CI linaro kselftest while they cross-compile for arm64) >>>>>> >>>>>> It just hit my travis jobs when I merged up to master: >>>>>> >>>>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>>>>> >>>>>> Shuah can we please get this reverted? >>>>>> >>>> >>>> I will take care of this. Supporting all these use-cases has been >>>> maintenance problem. Fixing one use-case breaks another. :( >>>> >>>> I will look into fixing this once for all. >>> >>> Thanks. >>> >>> In fact all of the above works fine with the new kselftest-install target you added recently >>> but I think a lot of CI is still using the old -C method. >>> >>> Regards >> >> If all of the above works with kselftest-install target, I would rather >> ask for CI's to change, so we can get rid of make -C usage for install. >> >> In which case, I would rather not revert this and request CIs to start >> using kselftest-install target. >> >> >> I would like to see CI's use kselftest-* targets from the main makefile >> for build/run/install. >> >> It has been very difficult to support "make -C" use-cases in general >> and especially for install and relocatable builds. >> > > Well I'm not sure if it covers all the build cases/scenario for CIs and I have > only verified that this specific failure seems to work using the toplevel Makefile > targets (since the offending commit uses abs-objtree). > Anyway $ ./kselftest_install.sh is broken too as of now. > With the revert, it will go back to not having support for relative paths :( I will look into reverting the offending commit or fixing it. thanks, -- Shuah ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 17:32 ` shuah @ 2019-11-27 17:57 ` Tim.Bird 2019-11-27 18:16 ` shuah 2019-11-28 4:17 ` Michael Ellerman 0 siblings, 2 replies; 14+ messages in thread From: Tim.Bird @ 2019-11-27 17:57 UTC (permalink / raw) To: shuah, cristian.marussi, mpe, linux-kselftest > -----Original Message----- > From: linux-kselftest-owner@vger.kernel.org [mailto:linux-kselftest- > owner@vger.kernel.org] On Behalf Of shuah > Sent: Wednesday, November 27, 2019 5:33 PM > To: Cristian Marussi <cristian.marussi@arm.com>; Michael Ellerman > <mpe@ellerman.id.au>; Bird, Tim <Tim.Bird@sony.com>; linux- > kselftest@vger.kernel.org; shuah <shuah@kernel.org> > Subject: Re: kselftest: failed to build with -C tool/testing/selftests when > KBUILD_OUTPUT is set > > On 11/27/19 9:53 AM, Cristian Marussi wrote: > > On 27/11/2019 16:42, shuah wrote: > >> On 11/27/19 9:23 AM, Cristian Marussi wrote: > >>> On 27/11/2019 16:04, shuah wrote: > >>>> On 11/27/19 3:54 AM, Cristian Marussi wrote: > >>>>> Hi > >>>>> > >>>>> On 27/11/2019 03:54, Michael Ellerman wrote: > >>>>>> Cristian Marussi <cristian.marussi@arm.com> writes: > >>>>>>> Hi > >>>>>>> > >>>>>>> while testing on linux-next > >>>>>>> > >>>>>>> I see that, when KBUILD_OUTPUT is set in the env, running > something like (using TARGETS=exec as a random subsystem here...) > >>>>>>> > >>>>>>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian- > aarch64-rootfs/opt/KSFT_next kselftest-install > >>>>>>> > >>>>>>> works fine as usual, WHILE the alternative invocation (still > documented in Documentation/dev-tools/kselftest.rst) > >>>>>>> > >>>>>>> make -C tools/testing/selftests/ TARGETS=exec > INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next > install > >>>>>>> > >>>>>>> fails miserably with: > >>>>>>> ... > >>>>>>> ... > >>>>>>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes- > abi.h > >>>>>>> HDRINST usr/include/asm/kvm.h > >>>>>>> INSTALL /kselftest/usr/include > >>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied > >>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe > for target 'headers_install' failed > >>>>>>> make[2]: *** [headers_install] Error 1 > >>>>>>> > >>>>>>> > >>>>>>> This is fixed by unsetting KBUILD_OUTPUT OR reverting: > >>>>>>> > >>>>>>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT > handling for relative paths > >>>>>>> > >>>>>>> since bypassing top makefile with -C, the definition of abs-objtree > used by the above patch > >>>>>>> is no more available. > >>>>>>> > >>>>>>> As a side effect when KBUILD_OUTPUT is set, this breaks also the > usage kselftest_install.sh. > >>>>>>> > >>>>>>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP- > official-debian-aarch64-rootfs/opt/KSFT_full_next > >>>>>>> ./kselftest_install.sh: Installing in specified location - > /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64- > rootfs/opt/KSFT_full_next ... > >>>>>>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ > >>>>>>> ARCH=arm64 -C ../../.. headers_install > >>>>>>> make[1]: Entering directory > '/home/crimar01/ARM/dev/src/pdsw/linux' > >>>>>>> make[2]: Entering directory > '/home/crimar01/ARM/dev/src/pdsw/out_linux' > >>>>>>> INSTALL /kselftest/usr/include > >>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied > >>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe > for target 'headers_install' failed > >>>>>>> make[2]: *** [headers_install] Error 1 > >>>>>>> make[2]: Leaving directory > '/home/crimar01/ARM/dev/src/pdsw/out_linux' > >>>>>>> Makefile:179: recipe for target 'sub-make' failed > >>>>>>> make[1]: *** [sub-make] Error 2 > >>>>>>> make[1]: Leaving directory > '/home/crimar01/ARM/dev/src/pdsw/linux' > >>>>>>> Makefile:142: recipe for target 'khdr' failed > >>>>>>> make: *** [khdr] Error 2 > >>>>>>> > >>>>>>> > >>>>>> ... > >>>>>>> Any thoughts ? ... or am I missing something ? > >>>>>> > >>>>>> You're not missing anything, this is broken. > >>>>> Thanks for the feedback ! > >>>>> > >>>>> Cristian > >>>>> > >>>>>> > >>>>>>> (I think I'm starting to see this in latest CI linaro kselftest while they > cross-compile for arm64) > >>>>>> > >>>>>> It just hit my travis jobs when I merged up to master: > >>>>>> > >>>>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 > >>>>>> > >>>>>> Shuah can we please get this reverted? > >>>>>> > >>>> > >>>> I will take care of this. Supporting all these use-cases has been > >>>> maintenance problem. Fixing one use-case breaks another. :( > >>>> > >>>> I will look into fixing this once for all. > >>> > >>> Thanks. > >>> > >>> In fact all of the above works fine with the new kselftest-install target > you added recently > >>> but I think a lot of CI is still using the old -C method. > >>> > >>> Regards > >> > >> If all of the above works with kselftest-install target, I would rather > >> ask for CI's to change, so we can get rid of make -C usage for install. > >> > >> In which case, I would rather not revert this and request CIs to start > >> using kselftest-install target. > >> > >> > >> I would like to see CI's use kselftest-* targets from the main makefile > >> for build/run/install. > >> > >> It has been very difficult to support "make -C" use-cases in general > >> and especially for install and relocatable builds. > >> > > > > Well I'm not sure if it covers all the build cases/scenario for CIs and I have > > only verified that this specific failure seems to work using the toplevel > Makefile > > targets (since the offending commit uses abs-objtree). > > Anyway $ ./kselftest_install.sh is broken too as of now. > > > > With the revert, it will go back to not having support for relative > paths :( > > I will look into reverting the offending commit or fixing it. There may not be many people using relative paths for KBUILD_OUTPUT. I was affected by it, and it's quite handy given some of the mechanisms in my CI system, but it's not that hard to work around. (That is, I can just convert relative paths to absolute paths in a few key places in my CI, before calling the kernel make.) Given that I'm the only person (seemingly) who reported this, and I'm ok working around it, I think a revert is OK for now. Thanks. -- Tim ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 17:57 ` Tim.Bird @ 2019-11-27 18:16 ` shuah 2019-11-27 18:48 ` Cristian Marussi 2019-11-28 4:17 ` Michael Ellerman 1 sibling, 1 reply; 14+ messages in thread From: shuah @ 2019-11-27 18:16 UTC (permalink / raw) To: Tim.Bird, cristian.marussi, mpe, linux-kselftest, shuah On 11/27/19 10:57 AM, Tim.Bird@sony.com wrote: > > >> -----Original Message----- >> From: linux-kselftest-owner@vger.kernel.org [mailto:linux-kselftest- >> owner@vger.kernel.org] On Behalf Of shuah >> Sent: Wednesday, November 27, 2019 5:33 PM >> To: Cristian Marussi <cristian.marussi@arm.com>; Michael Ellerman >> <mpe@ellerman.id.au>; Bird, Tim <Tim.Bird@sony.com>; linux- >> kselftest@vger.kernel.org; shuah <shuah@kernel.org> >> Subject: Re: kselftest: failed to build with -C tool/testing/selftests when >> KBUILD_OUTPUT is set >> >> On 11/27/19 9:53 AM, Cristian Marussi wrote: >>> On 27/11/2019 16:42, shuah wrote: >>>> On 11/27/19 9:23 AM, Cristian Marussi wrote: >>>>> On 27/11/2019 16:04, shuah wrote: >>>>>> On 11/27/19 3:54 AM, Cristian Marussi wrote: >>>>>>> Hi >>>>>>> >>>>>>> On 27/11/2019 03:54, Michael Ellerman wrote: >>>>>>>> Cristian Marussi <cristian.marussi@arm.com> writes: >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> while testing on linux-next >>>>>>>>> >>>>>>>>> I see that, when KBUILD_OUTPUT is set in the env, running >> something like (using TARGETS=exec as a random subsystem here...) >>>>>>>>> >>>>>>>>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian- >> aarch64-rootfs/opt/KSFT_next kselftest-install >>>>>>>>> >>>>>>>>> works fine as usual, WHILE the alternative invocation (still >> documented in Documentation/dev-tools/kselftest.rst) >>>>>>>>> >>>>>>>>> make -C tools/testing/selftests/ TARGETS=exec >> INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next >> install >>>>>>>>> >>>>>>>>> fails miserably with: >>>>>>>>> ... >>>>>>>>> ... >>>>>>>>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes- >> abi.h >>>>>>>>> HDRINST usr/include/asm/kvm.h >>>>>>>>> INSTALL /kselftest/usr/include >>>>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe >> for target 'headers_install' failed >>>>>>>>> make[2]: *** [headers_install] Error 1 >>>>>>>>> >>>>>>>>> >>>>>>>>> This is fixed by unsetting KBUILD_OUTPUT OR reverting: >>>>>>>>> >>>>>>>>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT >> handling for relative paths >>>>>>>>> >>>>>>>>> since bypassing top makefile with -C, the definition of abs-objtree >> used by the above patch >>>>>>>>> is no more available. >>>>>>>>> >>>>>>>>> As a side effect when KBUILD_OUTPUT is set, this breaks also the >> usage kselftest_install.sh. >>>>>>>>> >>>>>>>>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP- >> official-debian-aarch64-rootfs/opt/KSFT_full_next >>>>>>>>> ./kselftest_install.sh: Installing in specified location - >> /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64- >> rootfs/opt/KSFT_full_next ... >>>>>>>>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >>>>>>>>> ARCH=arm64 -C ../../.. headers_install >>>>>>>>> make[1]: Entering directory >> '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>>>>> make[2]: Entering directory >> '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>>>>> INSTALL /kselftest/usr/include >>>>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe >> for target 'headers_install' failed >>>>>>>>> make[2]: *** [headers_install] Error 1 >>>>>>>>> make[2]: Leaving directory >> '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>>>>> Makefile:179: recipe for target 'sub-make' failed >>>>>>>>> make[1]: *** [sub-make] Error 2 >>>>>>>>> make[1]: Leaving directory >> '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>>>>> Makefile:142: recipe for target 'khdr' failed >>>>>>>>> make: *** [khdr] Error 2 >>>>>>>>> >>>>>>>>> >>>>>>>> ... >>>>>>>>> Any thoughts ? ... or am I missing something ? >>>>>>>> >>>>>>>> You're not missing anything, this is broken. >>>>>>> Thanks for the feedback ! >>>>>>> >>>>>>> Cristian >>>>>>> >>>>>>>> >>>>>>>>> (I think I'm starting to see this in latest CI linaro kselftest while they >> cross-compile for arm64) >>>>>>>> >>>>>>>> It just hit my travis jobs when I merged up to master: >>>>>>>> >>>>>>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>>>>>>> >>>>>>>> Shuah can we please get this reverted? >>>>>>>> >>>>>> >>>>>> I will take care of this. Supporting all these use-cases has been >>>>>> maintenance problem. Fixing one use-case breaks another. :( >>>>>> >>>>>> I will look into fixing this once for all. >>>>> >>>>> Thanks. >>>>> >>>>> In fact all of the above works fine with the new kselftest-install target >> you added recently >>>>> but I think a lot of CI is still using the old -C method. >>>>> >>>>> Regards >>>> >>>> If all of the above works with kselftest-install target, I would rather >>>> ask for CI's to change, so we can get rid of make -C usage for install. >>>> >>>> In which case, I would rather not revert this and request CIs to start >>>> using kselftest-install target. >>>> >>>> >>>> I would like to see CI's use kselftest-* targets from the main makefile >>>> for build/run/install. >>>> >>>> It has been very difficult to support "make -C" use-cases in general >>>> and especially for install and relocatable builds. >>>> >>> >>> Well I'm not sure if it covers all the build cases/scenario for CIs and I have >>> only verified that this specific failure seems to work using the toplevel >> Makefile >>> targets (since the offending commit uses abs-objtree). >>> Anyway $ ./kselftest_install.sh is broken too as of now. >>> >> >> With the revert, it will go back to not having support for relative >> paths :( >> >> I will look into reverting the offending commit or fixing it. > > There may not be many people using relative paths for KBUILD_OUTPUT. > I was affected by it, and it's quite handy given some of the mechanisms in > my CI system, but it's not that hard to work around. (That is, I can just > convert relative paths to absolute paths in a few key places in my > CI, before calling the kernel make.) > > Given that I'm the only person (seemingly) who reported this, and I'm ok > working around it, I think a revert is OK for now. > Sorry Cristian! You reported this a couple of weeks ago and I missed your email. I was away on family emergency and this email got buried in my Inbox. No excuses though. My bad. Cristian suggested this fix. There is another issue here. INSTALL_PATH conflicts with main Makefile variable. The recent kbuild changes impacted kselftest use-cases and I had to drop support for INSTALL_PATH in kselftest_install.sh and move to KSFT_INSTALL_PATH - sorry the document is outdated. I should update it. I would like to see CIs migrate to using KSFT_INSTALL_PATH If we go with the below fix, does it fix for both O and KBUILD_OUTPUT cases, does it cover all cases? A possible fix would be (but duplicates in fact the main Makefile logic) diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile index 319e094c3212..491d8b3ef1c7 100644 --- a/tools/testing/selftests/Makefile +++ b/tools/testing/selftests/Makefile @@ -88,6 +88,15 @@ ifdef building_out_of_srctree override LDFLAGS = endif +ifeq ($(abs_objtree),) +ifneq ($(KBUILD_OUTPUT),) +abs_objtree := $(shell cd $(KBUILD_OUTPUT) && pwd) +abs_objtree := $(realpath $(abs_objtree)) +else +abs_objtree := $(shell pwd) +endif +endif #ifeq ($(abs_objtree),) + ifneq ($(O),) BUILD := $(abs_objtree) else thanks, -- Shuah ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 18:16 ` shuah @ 2019-11-27 18:48 ` Cristian Marussi 0 siblings, 0 replies; 14+ messages in thread From: Cristian Marussi @ 2019-11-27 18:48 UTC (permalink / raw) To: shuah, Tim.Bird, mpe, linux-kselftest On 27/11/2019 18:16, shuah wrote: > On 11/27/19 10:57 AM, Tim.Bird@sony.com wrote: >> >> >>> -----Original Message----- >>> From: linux-kselftest-owner@vger.kernel.org [mailto:linux-kselftest- >>> owner@vger.kernel.org] On Behalf Of shuah >>> Sent: Wednesday, November 27, 2019 5:33 PM >>> To: Cristian Marussi <cristian.marussi@arm.com>; Michael Ellerman >>> <mpe@ellerman.id.au>; Bird, Tim <Tim.Bird@sony.com>; linux- >>> kselftest@vger.kernel.org; shuah <shuah@kernel.org> >>> Subject: Re: kselftest: failed to build with -C tool/testing/selftests when >>> KBUILD_OUTPUT is set >>> >>> On 11/27/19 9:53 AM, Cristian Marussi wrote: >>>> On 27/11/2019 16:42, shuah wrote: >>>>> On 11/27/19 9:23 AM, Cristian Marussi wrote: >>>>>> On 27/11/2019 16:04, shuah wrote: >>>>>>> On 11/27/19 3:54 AM, Cristian Marussi wrote: >>>>>>>> Hi >>>>>>>> >>>>>>>> On 27/11/2019 03:54, Michael Ellerman wrote: >>>>>>>>> Cristian Marussi <cristian.marussi@arm.com> writes: >>>>>>>>>> Hi >>>>>>>>>> >>>>>>>>>> while testing on linux-next >>>>>>>>>> >>>>>>>>>> I see that, when KBUILD_OUTPUT is set in the env, running >>> something like (using TARGETS=exec as a random subsystem here...) >>>>>>>>>> >>>>>>>>>> $ make TARGETS=exec INSTALL_PATH=/nfs/LTP-official-debian- >>> aarch64-rootfs/opt/KSFT_next kselftest-install >>>>>>>>>> >>>>>>>>>> works fine as usual, WHILE the alternative invocation (still >>> documented in Documentation/dev-tools/kselftest.rst) >>>>>>>>>> >>>>>>>>>> make -C tools/testing/selftests/ TARGETS=exec >>> INSTALL_PATH=/nfs/LTP-official-debian-aarch64-rootfs/opt/KSFT_next >>> install >>>>>>>>>> >>>>>>>>>> fails miserably with: >>>>>>>>>> ... >>>>>>>>>> ... >>>>>>>>>> REMOVE usr/include/rdma/cxgb3-abi.h usr/include/rdma/nes- >>> abi.h >>>>>>>>>> HDRINST usr/include/asm/kvm.h >>>>>>>>>> INSTALL /kselftest/usr/include >>>>>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe >>> for target 'headers_install' failed >>>>>>>>>> make[2]: *** [headers_install] Error 1 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> This is fixed by unsetting KBUILD_OUTPUT OR reverting:17a7b03f2049 >>>>>>>>>> >>>>>>>>>> 303e6218ecec (ksft/fixes) selftests: Fix O= and KBUILD_OUTPUT >>> handling for relative paths >>>>>>>>>> >>>>>>>>>> since bypassing top makefile with -C, the definition of abs-objtree >>> used by the above patch >>>>>>>>>> is no more available. >>>>>>>>>> >>>>>>>>>> As a side effect when KBUILD_OUTPUT is set, this breaks also the >>> usage kselftest_install.sh. >>>>>>>>>> >>>>>>>>>> $ ./kselftest_install.sh /home/crimar01/ARM/dev/nfs/LTP- >>> official-debian-aarch64-rootfs/opt/KSFT_full_next >>>>>>>>>> ./kselftest_install.sh: Installing in specified location - >>> /home/crimar01/ARM/dev/nfs/LTP-official-debian-aarch64- >>> rootfs/opt/KSFT_full_next ... >>>>>>>>>> make --no-builtin-rules INSTALL_HDR_PATH=$BUILD/usr \ >>>>>>>>>> ARCH=arm64 -C ../../.. headers_install >>>>>>>>>> make[1]: Entering directory >>> '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>>>>>> make[2]: Entering directory >>> '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>>>>>> INSTALL /kselftest/usr/include >>>>>>>>>> mkdir: cannot create directory ‘/kselftest’: Permission denied >>>>>>>>>> /home/crimar01/ARM/dev/src/pdsw/linux/Makefile:1187: recipe >>> for target 'headers_install' failed >>>>>>>>>> make[2]: *** [headers_install] Error 1 >>>>>>>>>> make[2]: Leaving directory >>> '/home/crimar01/ARM/dev/src/pdsw/out_linux' >>>>>>>>>> Makefile:179: recipe for target 'sub-make' failed >>>>>>>>>> make[1]: *** [sub-make] Error 2 >>>>>>>>>> make[1]: Leaving directory >>> '/home/crimar01/ARM/dev/src/pdsw/linux' >>>>>>>>>> Makefile:142: recipe for target 'khdr' failed >>>>>>>>>> make: *** [khdr] Error 2 >>>>>>>>>> >>>>>>>>>> >>>>>>>>> ... >>>>>>>>>> Any thoughts ? ... or am I missing something ? >>>>>>>>> >>>>>>>>> You're not missing anything, this is broken. >>>>>>>> Thanks for the feedback ! >>>>>>>> >>>>>>>> Cristian >>>>>>>> >>>>>>>>> >>>>>>>>>> (I think I'm starting to see this in latest CI linaro kselftest while they >>> cross-compile for arm64) >>>>>>>>> >>>>>>>>> It just hit my travis jobs when I merged up to master: >>>>>>>>> >>>>>>>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>>>>>>>> >>>>>>>>> Shuah can we please get this reverted? >>>>>>>>> >>>>>>> >>>>>>> I will take care of this. Supporting all these use-cases has been >>>>>>> maintenance problem. Fixing one use-case breaks another. :( >>>>>>> >>>>>>> I will look into fixing this once for all. >>>>>> >>>>>> Thanks. >>>>>> >>>>>> In fact all of the above works fine with the new kselftest-install target >>> you added recently >>>>>> but I think a lot of CI is still using the old -C method. >>>>>> >>>>>> Regards >>>>> >>>>> If all of the above works with kselftest-install target, I would rather >>>>> ask for CI's to change, so we can get rid of make -C usage for install. >>>>> >>>>> In which case, I would rather not revert this and request CIs to start >>>>> using kselftest-install target. >>>>> >>>>> >>>>> I would like to see CI's use kselftest-* targets from the main makefile >>>>> for build/run/install. >>>>> >>>>> It has been very difficult to support "make -C" use-cases in general >>>>> and especially for install and relocatable builds. >>>>> >>>> >>>> Well I'm not sure if it covers all the build cases/scenario for CIs and I have >>>> only verified that this specific failure seems to work using the toplevel >>> Makefile >>>> targets (since the offending commit uses abs-objtree). >>>> Anyway $ ./kselftest_install.sh is broken too as of now. >>>> >>> >>> With the revert, it will go back to not having support for relative >>> paths :( >>> >>> I will look into reverting the offending commit or fixing it. >> >> There may not be many people using relative paths for KBUILD_OUTPUT. >> I was affected by it, and it's quite handy given some of the mechanisms in >> my CI system, but it's not that hard to work around. (That is, I can just >> convert relative paths to absolute paths in a few key places in my >> CI, before calling the kernel make.) >> >> Given that I'm the only person (seemingly) who reported this, and I'm ok >> working around it, I think a revert is OK for now. >> > > Sorry Cristian! You reported this a couple of weeks ago and I missed > your email. I was away on family emergency and this email got buried > in my Inbox. No excuses though. My bad. No worries :D > > Cristian suggested this fix. There is another issue here. > > INSTALL_PATH conflicts with main Makefile variable. The recent > kbuild changes impacted kselftest use-cases and I had to drop > support for INSTALL_PATH in kselftest_install.sh and move > to KSFT_INSTALL_PATH - sorry the document is outdated. > I saw that because I saw your change merged, but it did not seem to make any difference using the proper KSFT_INSTALL_PATH ... (but...I cannot remember why now :D ... tomorrow I'll double check about this) > I should update it. I would like to see CIs migrate to using > KSFT_INSTALL_PATH > > If we go with the below fix, does it fix for both O and KBUILD_OUTPUT > cases, does it cover all cases? > I'll have a look, I tested that tentative fix only with the KBUILD_OUTPUT interaction which is in my setup and local CI, so it immediately broke all. Not sure what Linaro CI is running though, I saw those logs and I can see same errors but then they carry on partially, so I don't have a clear view of all the possible scenarios here...I'll try to test more extensively next days. Regards Cristian > A possible fix would be (but duplicates in fact the main Makefile logic) > > > diff --git a/tools/testing/selftests/Makefile > b/tools/testing/selftests/Makefile > index 319e094c3212..491d8b3ef1c7 100644 > --- a/tools/testing/selftests/Makefile > +++ b/tools/testing/selftests/Makefile > @@ -88,6 +88,15 @@ ifdef building_out_of_srctree > override LDFLAGS = > endif > > +ifeq ($(abs_objtree),) > +ifneq ($(KBUILD_OUTPUT),) > +abs_objtree := $(shell cd $(KBUILD_OUTPUT) && pwd) > +abs_objtree := $(realpath $(abs_objtree)) > +else > +abs_objtree := $(shell pwd) > +endif > +endif #ifeq ($(abs_objtree),) > + > ifneq ($(O),) > BUILD := $(abs_objtree) > else > > thanks, > -- Shuah > ^ permalink raw reply [flat|nested] 14+ messages in thread
* RE: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 17:57 ` Tim.Bird 2019-11-27 18:16 ` shuah @ 2019-11-28 4:17 ` Michael Ellerman 1 sibling, 0 replies; 14+ messages in thread From: Michael Ellerman @ 2019-11-28 4:17 UTC (permalink / raw) To: Tim.Bird, shuah, cristian.marussi, linux-kselftest <Tim.Bird@sony.com> writes: >> -----Original Message----- >> From: linux-kselftest-owner@vger.kernel.org [mailto:linux-kselftest- >> owner@vger.kernel.org] On Behalf Of shuah >> Sent: Wednesday, November 27, 2019 5:33 PM >> To: Cristian Marussi <cristian.marussi@arm.com>; Michael Ellerman >> <mpe@ellerman.id.au>; Bird, Tim <Tim.Bird@sony.com>; linux- >> kselftest@vger.kernel.org; shuah <shuah@kernel.org> >> Subject: Re: kselftest: failed to build with -C tool/testing/selftests when >> KBUILD_OUTPUT is set >> >> On 11/27/19 9:53 AM, Cristian Marussi wrote: ... >> > Well I'm not sure if it covers all the build cases/scenario for CIs and I have >> > only verified that this specific failure seems to work using the toplevel Makefile >> > targets (since the offending commit uses abs-objtree). >> > Anyway $ ./kselftest_install.sh is broken too as of now. >> >> With the revert, it will go back to not having support for relative >> paths :( >> >> I will look into reverting the offending commit or fixing it. > > There may not be many people using relative paths for KBUILD_OUTPUT. > I was affected by it, and it's quite handy given some of the mechanisms in > my CI system, but it's not that hard to work around. (That is, I can just > convert relative paths to absolute paths in a few key places in my > CI, before calling the kernel make.) > > Given that I'm the only person (seemingly) who reported this, and I'm ok > working around it, I think a revert is OK for now. I think that would be best at this stage, every day this is unfixed is another day that selftests aren't getting built and tested. It should just be a matter of passing your output directory to `realpath` somewhere in your script. cheers ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-27 16:42 ` shuah 2019-11-27 16:53 ` Cristian Marussi @ 2019-11-28 4:20 ` Michael Ellerman 2019-11-28 23:33 ` shuah 1 sibling, 1 reply; 14+ messages in thread From: Michael Ellerman @ 2019-11-28 4:20 UTC (permalink / raw) To: shuah, Cristian Marussi, Tim.Bird, linux-kselftest, shuah shuah <shuah@kernel.org> writes: > On 11/27/19 9:23 AM, Cristian Marussi wrote: >> On 27/11/2019 16:04, shuah wrote: >>>> On 27/11/2019 03:54, Michael Ellerman wrote: ... >>>>> >>>>> It just hit my travis jobs when I merged up to master: >>>>> >>>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>>>> >>>>> Shuah can we please get this reverted? >>>>> >>> >>> I will take care of this. Supporting all these use-cases has been >>> maintenance problem. Fixing one use-case breaks another. :( ... > > If all of the above works with kselftest-install target, I would rather > ask for CI's to change, so we can get rid of make -C usage for install. > > In which case, I would rather not revert this and request CIs to start > using kselftest-install target. > > I would like to see CI's use kselftest-* targets from the main makefile > for build/run/install. The problem is old kernels (stable branches & distro kernels) don't support that (or it doesn't work correctly). So then the CIs have to invoke the selftest build differently depending on which kernel version they're building, which gets ugly for them. cheers ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set 2019-11-28 4:20 ` Michael Ellerman @ 2019-11-28 23:33 ` shuah 0 siblings, 0 replies; 14+ messages in thread From: shuah @ 2019-11-28 23:33 UTC (permalink / raw) To: Michael Ellerman, Cristian Marussi, Tim.Bird, linux-kselftest, shuah On 11/27/19 9:20 PM, Michael Ellerman wrote: > shuah <shuah@kernel.org> writes: >> On 11/27/19 9:23 AM, Cristian Marussi wrote: >>> On 27/11/2019 16:04, shuah wrote: >>>>> On 27/11/2019 03:54, Michael Ellerman wrote: > ... >>>>>> >>>>>> It just hit my travis jobs when I merged up to master: >>>>>> >>>>>> https://travis-ci.org/linuxppc/linux/jobs/617482001 >>>>>> >>>>>> Shuah can we please get this reverted? >>>>>> >>>> >>>> I will take care of this. Supporting all these use-cases has been >>>> maintenance problem. Fixing one use-case breaks another. :( > ... >> >> If all of the above works with kselftest-install target, I would rather >> ask for CI's to change, so we can get rid of make -C usage for install. >> >> In which case, I would rather not revert this and request CIs to start >> using kselftest-install target. >> Okay reverted now and pushed it to linux-kselftest fixes. Will send it to Linus in a couple of days. >> I would like to see CI's use kselftest-* targets from the main makefile >> for build/run/install. > > The problem is old kernels (stable branches & distro kernels) don't > support that (or it doesn't work correctly). > > So then the CIs have to invoke the selftest build differently depending > on which kernel version they're building, which gets ugly for them. > Let's move there slowly as stables get retired. I would like to see using kselftest main Makefile files as the supported mode going forward. This is necessary for us to make it easier to support and maintain the use-cases. thanks, -- Shuah ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2019-11-28 23:33 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-11-12 16:33 kselftest: failed to build with -C tool/testing/selftests when KBUILD_OUTPUT is set Cristian Marussi 2019-11-27 3:54 ` Michael Ellerman 2019-11-27 10:54 ` Cristian Marussi 2019-11-27 16:04 ` shuah 2019-11-27 16:23 ` Cristian Marussi 2019-11-27 16:42 ` shuah 2019-11-27 16:53 ` Cristian Marussi 2019-11-27 17:32 ` shuah 2019-11-27 17:57 ` Tim.Bird 2019-11-27 18:16 ` shuah 2019-11-27 18:48 ` Cristian Marussi 2019-11-28 4:17 ` Michael Ellerman 2019-11-28 4:20 ` Michael Ellerman 2019-11-28 23:33 ` shuah
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).