BISECTED: Compile error on 5.4-rc1

BISECTED: Compile error on 5.4-rc1

[Harald Arnesen]

I just tried to compile kernel 5.4-rc1 on my ThinkPad, which runs Devuan
Beowulf. Got the following:

$ make bindeb-pkg
  UPD     include/config/kernel.release
sh ./scripts/package/mkdebian
dpkg-buildpackage -r"fakeroot -u" -a$(cat debian/arch)  -b -nc -uc
dpkg-buildpackage: info: source package linux-5.4.0-rc1-00014-gcc3a7bfe62b9
dpkg-buildpackage: info: source version 5.4.0-rc1-00014-gcc3a7bfe62b9-1
dpkg-buildpackage: info: source distribution beowulf
dpkg-buildpackage: info: source changed by Harald Arnesen
<harald@skogtun.org>
dpkg-architecture: warning: specified GNU system type x86_64-linux-gnu
does not match CC system type x86_64-pc-linux-gnu, try setting a correct
CC environment variable
dpkg-buildpackage: info: host architecture amd64
 dpkg-source --before-build .
dpkg-source: warning: can't parse dependency -n libelf-dev
dpkg-source: error: error occurred while parsing Build-Depends
dpkg-buildpackage: error: dpkg-source --before-build . subprocess
returned exit status 255
make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 255
make: *** [Makefile:1448: bindeb-pkg] Error 2


Bisecting gives me

858805b336be1cabb3d9033adaa3676574d12e37 is the first bad commit
commit 858805b336be1cabb3d9033adaa3676574d12e37
Author: Masahiro Yamada <yamada.masahiro@socionext.com>
Date:   Sun Aug 25 22:28:37 2019 +0900
...

By reverting commit 858805b336be1cabb3d9033adaa3676574d12e37 I could
compile the kernel.
-- 
Hilsen Harald

Re: BISECTED: Compile error on 5.4-rc1

[Masahiro Yamada]

On Fri, Oct 4, 2019 at 5:26 PM Harald Arnesen <harald@skogtun.org> wrote:
>
> I just tried to compile kernel 5.4-rc1 on my ThinkPad, which runs Devuan
> Beowulf. Got the following:
>
> $ make bindeb-pkg
>   UPD     include/config/kernel.release
> sh ./scripts/package/mkdebian
> dpkg-buildpackage -r"fakeroot -u" -a$(cat debian/arch)  -b -nc -uc
> dpkg-buildpackage: info: source package linux-5.4.0-rc1-00014-gcc3a7bfe62b9
> dpkg-buildpackage: info: source version 5.4.0-rc1-00014-gcc3a7bfe62b9-1
> dpkg-buildpackage: info: source distribution beowulf
> dpkg-buildpackage: info: source changed by Harald Arnesen
> <harald@skogtun.org>
> dpkg-architecture: warning: specified GNU system type x86_64-linux-gnu
> does not match CC system type x86_64-pc-linux-gnu, try setting a correct
> CC environment variable
> dpkg-buildpackage: info: host architecture amd64
>  dpkg-source --before-build .
> dpkg-source: warning: can't parse dependency -n libelf-dev
> dpkg-source: error: error occurred while parsing Build-Depends
> dpkg-buildpackage: error: dpkg-source --before-build . subprocess
> returned exit status 255
> make[1]: *** [scripts/Makefile.package:83: bindeb-pkg] Error 255
> make: *** [Makefile:1448: bindeb-pkg] Error 2
>
>
> Bisecting gives me
>
> 858805b336be1cabb3d9033adaa3676574d12e37 is the first bad commit
> commit 858805b336be1cabb3d9033adaa3676574d12e37
> Author: Masahiro Yamada <yamada.masahiro@socionext.com>
> Date:   Sun Aug 25 22:28:37 2019 +0900
> ...
>
> By reverting commit 858805b336be1cabb3d9033adaa3676574d12e37 I could
> compile the kernel.
> --
> Hilsen Harald
>


I cannot reproduce it.

I tested bindeb-pkg for the latest Linus tree successfully.



masahiro@grover:~/ref/linux$ git log --oneline -1
4ea655343ce4 (HEAD -> master, origin/master, origin/HEAD) Merge tag
'mips_fixes_5.4_1' of
git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux
masahiro@grover:~/ref/linux$ make bindeb-pkg
sh ./scripts/package/mkdebian
dpkg-buildpackage -r"fakeroot -u" -a$(cat debian/arch)  -b -nc -uc
dpkg-buildpackage: info: source package linux-5.4.0-rc1+
dpkg-buildpackage: info: source version 5.4.0-rc1+-4
dpkg-buildpackage: info: source distribution bionic
dpkg-buildpackage: info: source changed by Masahiro Yamada
<yamada.masahiro@socionext.com>
dpkg-buildpackage: info: host architecture amd64
 dpkg-source --before-build linux
 debian/rules build
make KERNELRELEASE=5.4.0-rc1+ ARCH=x86 KBUILD_BUILD_VERSION=4 -f ./Makefile
  CALL    scripts/checksyscalls.sh
  CALL    scripts/atomic/check-atomics.sh
  DESCEND  objtool
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  AR      init/built-in.a
  GEN     .version
  CHK     include/generated/compile.h
  LD      vmlinux.o
  MODPOST vmlinux.o
  MODINFO modules.builtin.modinfo
  LD      .tmp_vmlinux1
  KSYM    .tmp_kallsyms1.o
  LD      .tmp_vmlinux2
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map
  VOFFSET arch/x86/boot/compressed/../voffset.h
  CC      arch/x86/boot/compressed/misc.o
  OBJCOPY arch/x86/boot/compressed/vmlinux.bin
  RELOCS  arch/x86/boot/compressed/vmlinux.relocs
  GZIP    arch/x86/boot/compressed/vmlinux.bin.gz
  MKPIGGY arch/x86/boot/compressed/piggy.S
  AS      arch/x86/boot/compressed/piggy.o
  CC      arch/x86/boot/compressed/kaslr.o
  LD      arch/x86/boot/compressed/vmlinux
  ZOFFSET arch/x86/boot/zoffset.h
  AS      arch/x86/boot/header.o
  CC      arch/x86/boot/version.o
  LD      arch/x86/boot/setup.elf
  OBJCOPY arch/x86/boot/setup.bin
  OBJCOPY arch/x86/boot/vmlinux.bin
  BUILD   arch/x86/boot/bzImage
Setup is 16124 bytes (padded to 16384 bytes).
System is 8661 kB
CRC fab8ebd9
Kernel: arch/x86/boot/bzImage is ready  (#5)
  Building modules, stage 2.
  MODPOST 12 modules
 fakeroot -u debian/rules binary
make KERNELRELEASE=5.4.0-rc1+ ARCH=x86 KBUILD_BUILD_VERSION=4 -f
./Makefile intdeb-pkg
sh ./scripts/package/builddeb
  INSTALL drivers/thermal/intel/x86_pkg_temp_thermal.ko
  INSTALL fs/efivarfs/efivarfs.ko
  INSTALL net/ipv4/netfilter/iptable_nat.ko
  INSTALL net/ipv4/netfilter/nf_log_arp.ko
  INSTALL net/ipv4/netfilter/nf_log_ipv4.ko
  INSTALL net/ipv6/netfilter/nf_log_ipv6.ko
  INSTALL net/netfilter/nf_log_common.ko
  INSTALL net/netfilter/xt_LOG.ko
  INSTALL net/netfilter/xt_MASQUERADE.ko
  INSTALL net/netfilter/xt_addrtype.ko
  INSTALL net/netfilter/xt_mark.ko
  INSTALL net/netfilter/xt_nat.ko
  DEPMOD  5.4.0-rc1+
  INSTALL ./debian/headertmp/usr/include
dpkg-deb: building package 'linux-headers-5.4.0-rc1+' in
'../linux-headers-5.4.0-rc1+_5.4.0-rc1+-4_amd64.deb'.
dpkg-deb: building package 'linux-libc-dev' in
'../linux-libc-dev_5.4.0-rc1+-4_amd64.deb'.
dpkg-deb: building package 'linux-image-5.4.0-rc1+' in
'../linux-image-5.4.0-rc1+_5.4.0-rc1+-4_amd64.deb'.
 dpkg-genbuildinfo --build=binary
 dpkg-genchanges --build=binary >../linux-5.4.0-rc1+_5.4.0-rc1+-4_amd64.changes
dpkg-genchanges: warning: package linux-image-5.4.0-rc1+-dbg in
control file but not in files list
dpkg-genchanges: info: binary-only upload (no source code included)
 dpkg-source --after-build linux
dpkg-buildpackage: info: binary-only upload (no source included)



-- 
Best Regards
Masahiro Yamada

Re: BISECTED: Compile error on 5.4-rc1

[Harald Arnesen]

Masahiro Yamada [05.10.2019 05:24]:

> I cannot reproduce it.
> 
> I tested bindeb-pkg for the latest Linus tree successfully.

Strange, I have now tried another machine running the same distro
(Devuan Beowulf), and I get the same result there. Will check further.
-- 
Hilsen Harald

Re: BISECTED: Compile error on 5.4-rc1

[Harald Arnesen]

Harald Arnesen [05.10.2019 11:03]:

> Masahiro Yamada [05.10.2019 05:24]:
> 
>> I cannot reproduce it.
>> 
>> I tested bindeb-pkg for the latest Linus tree successfully.
> 
> Strange, I have now tried another machine running the same distro
> (Devuan Beowulf), and I get the same result there. Will check further.

I found out what was wrong.

Both machines have been used for dvd burning, and I have used Jörg
Schilling's cdrecord - and I had installed all of the "Schily Tools",
which unfortunately includes a shell command. Now, I had (by mistake)
/opt/schily/bin early in my path, and his OpenSolaris-derived shell
didn't work as expected.

Moving /opt/schily/bin to the end of the path fixes the problem.

But shouldn't it work even if "sh" is not equal to "/bin/sh"?
-- 
Hilsen Harald

Re: BISECTED: Compile error on 5.4-rc1

[Masahiro Yamada]

On Sat, Oct 5, 2019 at 6:36 PM Harald Arnesen <harald@skogtun.org> wrote:
>
> Harald Arnesen [05.10.2019 11:03]:
>
> > Masahiro Yamada [05.10.2019 05:24]:
> >
> >> I cannot reproduce it.
> >>
> >> I tested bindeb-pkg for the latest Linus tree successfully.
> >
> > Strange, I have now tried another machine running the same distro
> > (Devuan Beowulf), and I get the same result there. Will check further.
>
> I found out what was wrong.
>
> Both machines have been used for dvd burning, and I have used Jörg
> Schilling's cdrecord - and I had installed all of the "Schily Tools",
> which unfortunately includes a shell command. Now, I had (by mistake)
> /opt/schily/bin early in my path, and his OpenSolaris-derived shell
> didn't work as expected.
>
> Moving /opt/schily/bin to the end of the path fixes the problem.
>
> But shouldn't it work even if "sh" is not equal to "/bin/sh"?


CONFIG_SHELL previously fell back to 'sh' when bash is not installed,
so I just kept it as it was.

If we had used the exact absolute path /bin/sh,
it would have worked irrespective of the PATH environment.

But, there is a counter option like this:


commit 16f8259ca77d04f95e5ca90be1b1894ed45816c0
Author: Bjørn Forsman <bjorn.forsman@gmail.com>
Date:   Sun Nov 5 10:44:16 2017 +0100

    kbuild: /bin/pwd -> pwd

    Most places use pwd and rely on $PATH lookup. Moving the remaining
    absolute path /bin/pwd users over for consistency.

    Also, a reason for doing /bin/pwd -> pwd instead of the other way around
    is because I believe build systems should make little assumptions on
    host filesystem layout. Case in point, we do this kind of patching
    already in NixOS.

    Ref. commit 028568d84da3cfca49f5f846eeeef01441d70451
    ("kbuild: revert $(realpath ...) to $(shell cd ... && /bin/pwd)").

    Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
    Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>



I cannot find a way to satisfy everybody.



-- 
Best Regards
Masahiro Yamada

Re: BISECTED: Compile error on 5.4-rc1

[Harald Arnesen]

Masahiro Yamada [05.10.2019 12:19]:

> CONFIG_SHELL previously fell back to 'sh' when bash is not installed,
> so I just kept it as it was.
> 
> If we had used the exact absolute path /bin/sh,
> it would have worked irrespective of the PATH environment.
> 
> But, there is a counter option like this:
> 
> 
> commit 16f8259ca77d04f95e5ca90be1b1894ed45816c0
> Author: Bjørn Forsman <bjorn.forsman@gmail.com>
> Date:   Sun Nov 5 10:44:16 2017 +0100
> 
>     kbuild: /bin/pwd -> pwd
> 
>     Most places use pwd and rely on $PATH lookup. Moving the remaining
>     absolute path /bin/pwd users over for consistency.
> 
>     Also, a reason for doing /bin/pwd -> pwd instead of the other way around
>     is because I believe build systems should make little assumptions on
>     host filesystem layout. Case in point, we do this kind of patching
>     already in NixOS.
> 
>     Ref. commit 028568d84da3cfca49f5f846eeeef01441d70451
>     ("kbuild: revert $(realpath ...) to $(shell cd ... && /bin/pwd)").
> 
>     Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
>     Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> 
> 
> 
> I cannot find a way to satisfy everybody.
> 

I'm totally fine with the way it is now, now that I know how it works.
However, doesn't Posix dictate that there is a /bin/sh?
-- 
Hilsen Harald

Re: BISECTED: Compile error on 5.4-rc1

[Masahiro Yamada]

On Sat, Oct 5, 2019 at 7:50 PM Harald Arnesen <harald@skogtun.org> wrote:
>
> Masahiro Yamada [05.10.2019 12:19]:
>
> > CONFIG_SHELL previously fell back to 'sh' when bash is not installed,
> > so I just kept it as it was.
> >
> > If we had used the exact absolute path /bin/sh,
> > it would have worked irrespective of the PATH environment.
> >
> > But, there is a counter option like this:
> >
> >
> > commit 16f8259ca77d04f95e5ca90be1b1894ed45816c0
> > Author: Bjørn Forsman <bjorn.forsman@gmail.com>
> > Date:   Sun Nov 5 10:44:16 2017 +0100
> >
> >     kbuild: /bin/pwd -> pwd
> >
> >     Most places use pwd and rely on $PATH lookup. Moving the remaining
> >     absolute path /bin/pwd users over for consistency.
> >
> >     Also, a reason for doing /bin/pwd -> pwd instead of the other way around
> >     is because I believe build systems should make little assumptions on
> >     host filesystem layout. Case in point, we do this kind of patching
> >     already in NixOS.
> >
> >     Ref. commit 028568d84da3cfca49f5f846eeeef01441d70451
> >     ("kbuild: revert $(realpath ...) to $(shell cd ... && /bin/pwd)").
> >
> >     Signed-off-by: Bjørn Forsman <bjorn.forsman@gmail.com>
> >     Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
> >
> >
> >
> > I cannot find a way to satisfy everybody.
> >
>
> I'm totally fine with the way it is now, now that I know how it works.
> However, doesn't Posix dictate that there is a /bin/sh?


As for POSIX, I found this:

---------------------->8----------------------------
Applications should note that the standard PATH to the shell cannot
be assumed to be either /bin/sh or /usr/bin/sh, and should be determined
by interrogation of the PATH returned by getconf PATH , ensuring that
the returned pathname is an absolute pathname and not a shell built-in.
---------------------->8----------------------------

https://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.html


-- 
Best Regards
Masahiro Yamada

Re: BISECTED: Compile error on 5.4-rc1

[Harald Arnesen]

Masahiro Yamada [05.10.2019 15:17]:

> As for POSIX, I found this:
> 
> ---------------------->8----------------------------
> Applications should note that the standard PATH to the shell cannot
> be assumed to be either /bin/sh or /usr/bin/sh, and should be determined
> by interrogation of the PATH returned by getconf PATH , ensuring that
> the returned pathname is an absolute pathname and not a shell built-in.
> ---------------------->8----------------------------
> 
> https://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.html

Okay. Then it's the Schilling/OpenSolaris shell that is the problem, not
that I use it anyway.
-- 
Hilsen Harald