Re: make -jN modules_install breakage in next-20230831

[Masahiro Yamada <masahiroy@kernel.org>]

On Fri, Sep 1, 2023 at 2:13 PM Chen-Yu Tsai <wenst@chromium.org> wrote:
>
> On Thu, Aug 31, 2023 at 11:55:35PM +0900, Masahiro Yamada wrote:
> > On Thu, Aug 31, 2023 at 5:59 PM Chen-Yu Tsai <wenst@chromium.org> wrote:
> > >
> > > Hi,
> > >
> > > I observed some breakage with `make -jN moudles_install`. The scripts produce
> > > some bogus paths, which `cp` then complains it can't find.
> > >
> > > I run the following for my workflow.
> > >
> > >   $ nice -n 10 sh -c "make -j72 Image.lzma dtbs && ./pack-mod.sh && cb/sign.sh"
> > >
> > > pack-mod.sh is what fails. It basically does:
> > >
> > >     make -j72 modules
> > >     make -j72 modules_install INSTALL_MOD_PATH=${TMPDIR}
> > >     depmod ...
> > >     tar ...
> > >     cleanup
> > >
> > > I get errors like the following:
> > >
> > >
> > >   CALL    scripts/checksyscalls.sh
> > >   CALL    scripts/checksyscalls.sh
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/arch/arm64/crypto/sha1-ce.ko
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/arch/arm64/crypto/chacha-neon.ko
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/arch/arm64/crypto/poly1305-neon.ko
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/kernel/time/test_udelay.ko
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/kernel/configs.ko
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/fs/nfs_common/grace.ko
> > > cp: cannot create regular file
> > > 'tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/kernel/time/test_udelay.ko':
> > > No such file or directory
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/fs/nls/nls_cp437.ko
> > > make[2]: *** [scripts/Makefile.modinst:127:
> > > tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/kernel/time/test_udelay.ko]
> > > Error 1
> > > make[2]: *** Waiting for unfinished jobs....
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/fs/nls/nls_ascii.ko
> > > cp: cannot create regular file
> > > 'tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/kernel/configs.ko':
> > > No such file or directory
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/fs/nls/nls_iso8859-1.ko
> > >   INSTALL tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/fs/nls/nls_utf8.ko
> > > make[2]: *** [scripts/Makefile.modinst:128:
> > > tmp.7i3QCV3ac8/lib/modules/6.5.0-next-20230831-13697-g09d3bd189478-dirty/kernel/kernel/configs.ko]
> > > Error 1
> > > make[1]: *** [/usr/local/google/home/wenst/linux/mtk/Makefile:1822:
> > > modules_install] Error 2
> > > make: *** [Makefile:234: __sub-make] Error 2
> > >
> > >
> > > Note the duplicate "kernel/" in the path for test_udelay.ko and configs.ko.
> >
> >
> >
> > The paths to test_udelay.ko and configs.ko look quite normal to me.
> >
> > "kernel/kernel/" is correct.
> >
> >
> > The first "kernel/" means in-tree modules.
> >
> > All in-tree modules are installed
> > under /lib/modules/<ver>/kernel/,
> > while external modules are installed
> > under /lib/modules/<ver>/updates/.
> >
> >
> > The second "kernel/" is the directory in the source tree.
> >
> >
> >
> > I cannot reproduce the error.
> >
> >
> >
> >
> > But, indeed. Your installation log looks weird overall.
> > No idea what is happening in your case.
> >
> >
> > If you add V=1, Kbuild will show more logs, and we may get more hints.
> >
> >   make V=1 -j72 modules_install INSTALL_MOD_PATH=${TMPDIR}
>
> Here's a cleaner repro on next-20230831:
>
>     $ export ARCH=arm64
>     $ export CROSS_COMPILE="aarch64-linux-gnu-"
>     ### defconfig attached at the end
>     $ make olddefconfig
>     $ make -j72 Image.gz modules
>     $ make -j72 V=1 modules_install INSTALL_MOD_PATH=tmp
>     make --no-print-directory -C /usr/local/google/home/wenst/linux/mtk \
>     -f /usr/local/google/home/wenst/linux/mtk/Makefile modules_install
>     make -f ./scripts/Makefile.modinst \
>     sign-only=
>     rm -fr tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/build
>     mkdir -p tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/arch/arm64/crypto/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/crypto/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/block/zram/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/bluetooth/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/cdrom/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/hid/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/i2c/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/iio/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/iio/common/cros_ec_sensors/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/iio/light/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/iio/pressure/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/iio/trigger/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/input/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/input/joystick/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/input/joystick/iforce/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/input/misc/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/input/rmi4/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/input/serio/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/input/touchscreen/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/common/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/common/v4l2-tpg/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/common/videobuf2/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/i2c/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/platform/mediatek/jpeg/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/platform/mediatek/mdp/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/platform/mediatek/vcodec/common/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/platform/mediatek/vcodec/decoder/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/platform/mediatek/vcodec/encoder/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/platform/mediatek/vpu/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/test-drivers/visl/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/usb/uvc/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/media/v4l2-core/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/mmc/core/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/phy/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/usb/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireguard/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/ath/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/ath/ath10k/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/legacy/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/marvell/mwifiex/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/mediatek/mt76/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/mediatek/mt76/mt7921/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/mediatek/mt76/mt7996/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/ralink/rt2x00/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/realtek/rtl8xxxu/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/realtek/rtw88/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/realtek/rtw89/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/net/wireless/virtual/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/platform/chrome/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/remoteproc/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/rpmsg/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/scsi/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/usb/class/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/usb/gadget/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/usb/gadget/function/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/usb/misc/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/usb/serial/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/usb/usbip/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/vhost/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/drivers/watchdog/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/fat/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/fuse/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/hfsplus/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/isofs/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/lockd/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nfs/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nfs_common/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/udf/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/time/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/lib/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/lib/crypto/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/802/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/8021q/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/bluetooth/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/bluetooth/hidp/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/bluetooth/rfcomm/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/bridge/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/dns_resolver/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/ipv4/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/ipv6/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/ipv6/netfilter/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/llc/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/mac80211/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/netfilter/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/sched/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/sunrpc/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/sunrpc/auth_gss/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/vmw_vsock/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/net/wireless/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/core/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/core/seq/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/drivers/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/soc/sof/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/soc/sof/mediatek/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/soc/sof/mediatek/mt8186/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/soc/sof/mediatek/mt8195/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/soc/sof/xtensa/ tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/sound/usb/
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/arch/arm64/crypto/sha1-ce.ko
>       cp arch/arm64/crypto/sha1-ce.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/arch/arm64/crypto/sha1-ce.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/arch/arm64/crypto/chacha-neon.ko
>       cp arch/arm64/crypto/chacha-neon.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/arch/arm64/crypto/chacha-neon.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/arch/arm64/crypto/poly1305-neon.ko
>       cp arch/arm64/crypto/poly1305-neon.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/arch/arm64/crypto/poly1305-neon.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/time/test_udelay.ko
>       cp kernel/time/test_udelay.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/time/test_udelay.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/configs.ko
>       cp kernel/configs.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/configs.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nfs_common/grace.ko
>       cp fs/nfs_common/grace.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nfs_common/grace.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_cp437.ko
>       cp fs/nls/nls_cp437.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_cp437.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_ascii.ko
>       cp fs/nls/nls_ascii.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_ascii.ko
>     cp: cannot create regular file 'tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/time/test_udelay.ko': No such file or directory
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_iso8859-1.ko
>       cp fs/nls/nls_iso8859-1.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_iso8859-1.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_utf8.ko
>       cp fs/nls/nls_utf8.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/nls/nls_utf8.ko
>     # INSTALL tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/fat/fat.ko
>       cp fs/fat/fat.ko tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/fs/fat/fat.ko
>     make[2]: *** [scripts/Makefile.modinst:128: tmp/lib/modules/6.5.0-next-20230831-13643-ga47fc304d2b6/kernel/kernel/time/test_udelay.ko] Error 1
>     make[2]: *** Waiting for unfinished jobs....
>
> I suspect what is happening is that the prepare target (which does
> `mkdir`) is getting executed at the same time as all the install
> commands. This seems to only happen if module compression is enabled.



Ah, you are right.
The module compression was missing in my mind.

I locally fixed up the code.

Thank you for catching it.



--
Best Regards
Masahiro Yamada