Small update:

This hang is definitely caused by:

https://git.openembedded.org/openembedded-core/commit/?id=00aeb84a5e7c6d4293fb4cd30dd1e28d60c6aac5

I've reproduced this issue with master 5 times with:

oe-selftest -r buildoptions.ImageOptionsTests.test_ccache_tool

then after reverting 00aeb84a5e7c6d4293fb4cd30dd1e28d60c6aac5 it passed 3 times and failed again when I've restored this commit.

I have huge 852M install.man-make-native-4.3.j2.trace.d.log

where it looks like make-native ends in neverending loop, make from gentoo has the same issue (when just called with full path as /usr/bin/make -d --trace -j 2 install.man 'DESTDIR=/OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/image' 2>&1 | tee install.man-make-gentoo-4.3.j2.trace.d.log).

With -j 1 in the same directory it works fine.

perl-native/5.36.0-r0/perl-5.36.0 $ grep Considering.target.file..nonxs_ext install.man-make-gentoo-4.3.j2.trace.d.log | wc -l
7621

The relevant (I hope) part of the log looks like:

Pruning file 'ext/Sys-Hostname/pm_to_blib'.
Pruning file 'ext/XS-APItest/pm_to_blib'.
Pruning file 'ext/XS-Typemap/pm_to_blib'.
Finished prerequisites of target file 'dynamic_ext'.
The prerequisites of 'dynamic_ext' are being made.
Considering target file 'static_ext'.
File 'static_ext' was considered already.
Finished prerequisites of target file 'extensions'.
The prerequisites of 'extensions' are being made.
Finished prerequisites of target file 'modules'.
The prerequisites of 'modules' are being made.
Finished prerequisites of target file 'pod/perltoc.pod'.
The prerequisites of 'pod/perltoc.pod' are being made.
Pruning file 'miniperl'.
Finished prerequisites of target file 'install.man'.
The prerequisites of 'install.man' are being made.
Live child 0x55a28372eaf0 (ext/XS-Typemap/Makefile) PID 517626
Generating a Unix-style Makefile
Writing Makefile for XS::Typemap
Reaping winning child 0x55a28372eaf0 PID 517626
Removing child 0x55a28372eaf0 PID 517626 from chain.
Considering target file 'install.man'.
File 'install.man' does not exist.
Considering target file 'installman'.
File 'installman' was considered already.
Considering target file 'pod/perltoc.pod'.
Considering target file 'pod/buildtoc'.
File 'pod/buildtoc' was considered already.
Considering target file 'miniperl'.
File 'miniperl' was considered already.
Considering target file 'modules'.
File 'modules' does not exist.
Considering target file 'extensions'.
File 'extensions' does not exist.
Considering target file 'cflags'.
File 'cflags' was considered already.
Considering target file 'nonxs_ext'.
File 'nonxs_ext' does not exist.
Considering target file 'cpan/Archive-Tar/pm_to_blib'.
File 'cpan/Archive-Tar/pm_to_blib' was considered already.

Considering target file 'cpan/autodie/pm_to_blib'.
File 'cpan/autodie/pm_to_blib' was considered already.

......

Considering target file 'dist/ExtUtils-CBuilder/pm_to_blib'.
File 'dist/ExtUtils-CBuilder/pm_to_blib' was considered already.
Considering target file 'dist/ExtUtils-ParseXS/pm_to_blib'.
Considering target file 'dist/ExtUtils-ParseXS/Makefile'.
File 'dist/ExtUtils-ParseXS/Makefile' was considered already.
Considering target file 'dist/ExtUtils-ParseXS/ppport.h'.
Considering target file 'dist/Devel-PPPort/ppport.h'.
Considering target file 'dist/Devel-PPPort/PPPort.pm'.
Considering target file 'dist/Devel-PPPort/pm_to_blib'.
Considering target file 'dist/Devel-PPPort/Makefile'.
File 'dist/Devel-PPPort/Makefile' was considered already.
Considering target file 'DynaLoader.o'.
Considering target file 'ext/DynaLoader/pm_to_blib'.
Finished prerequisites of target file 'ext/DynaLoader/pm_to_blib'.
The prerequisites of 'ext/DynaLoader/pm_to_blib' are being made.
Finished prerequisites of target file 'DynaLoader.o'.
The prerequisites of 'DynaLoader.o' are being made.
Pruning file 'DynaLoader.o'.
Pruning file 'dist/ExtUtils-CBuilder/pm_to_blib'.
Finished prerequisites of target file 'dist/Devel-PPPort/pm_to_blib'.
The prerequisites of 'dist/Devel-PPPort/pm_to_blib' are being made.
Finished prerequisites of target file 'dist/Devel-PPPort/PPPort.pm'.
The prerequisites of 'dist/Devel-PPPort/PPPort.pm' are being made.
Pruning file 'miniperl'.
Finished prerequisites of target file 'dist/Devel-PPPort/ppport.h'.
The prerequisites of 'dist/Devel-PPPort/ppport.h' are being made.
Finished prerequisites of target file 'dist/ExtUtils-ParseXS/ppport.h'.
The prerequisites of 'dist/ExtUtils-ParseXS/ppport.h' are being made.
Finished prerequisites of target file 'dist/ExtUtils-ParseXS/pm_to_blib'.
The prerequisites of 'dist/ExtUtils-ParseXS/pm_to_blib' are being made.
Considering target file 'dist/Filter-Simple/pm_to_blib'.
File 'dist/Filter-Simple/pm_to_blib' was considered already.
Considering target file 'dist/FindBin/pm_to_blib'.

.....

Considering target file 'ext/Tie-Memoize/pm_to_blib'.
File 'ext/Tie-Memoize/pm_to_blib' was considered already.
Finished prerequisites of target file 'nonxs_ext'.
The prerequisites of 'nonxs_ext' are being made.
Considering target file 'dynamic_ext'.
File 'dynamic_ext' does not exist.
Considering target file 'cpan/Compress-Raw-Bzip2/pm_to_blib'.
Considering target file 'cpan/Compress-Raw-Bzip2/Makefile'.
File 'cpan/Compress-Raw-Bzip2/Makefile' was considered already.
Pruning file 'DynaLoader.o'.
Pruning file 'dist/ExtUtils-CBuilder/pm_to_blib'.
Finished prerequisites of target file 'cpan/Compress-Raw-Bzip2/pm_to_blib'.
The prerequisites of 'cpan/Compress-Raw-Bzip2/pm_to_blib' are being made.
Considering target file 'cpan/Compress-Raw-Zlib/pm_to_blib'.

....

The prerequisites of 'ext/XS-Typemap/pm_to_blib' are being made.
Finished prerequisites of target file 'dynamic_ext'.
The prerequisites of 'dynamic_ext' are being made.
Considering target file 'static_ext'.
File 'static_ext' was considered already.
Considering target file 'nonxs_ext'.
File 'nonxs_ext' does not exist.
Pruning file 'cpan/Archive-Tar/pm_to_blib'.
Pruning file 'cpan/autodie/pm_to_blib'.
Pruning file 'cpan/AutoLoader/pm_to_blib'.

....

Pruning file 'ext/Tie-Hash-NamedCapture/pm_to_blib'.
Pruning file 'ext/Tie-Memoize/pm_to_blib'.
Finished prerequisites of target file 'nonxs_ext'.
The prerequisites of 'nonxs_ext' are being made.
Considering target file 'dynamic_ext'.
File 'dynamic_ext' does not exist.
Pruning file 'cpan/Compress-Raw-Bzip2/pm_to_blib'.
Pruning file 'cpan/Compress-Raw-Zlib/pm_to_blib'.

...

....

Pruning file 'ext/XS-APItest/pm_to_blib'.
Pruning file 'ext/XS-Typemap/pm_to_blib'.
Finished prerequisites of target file 'dynamic_ext'.
The prerequisites of 'dynamic_ext' are being made.
Considering target file 'static_ext'.
File 'static_ext' was considered already.
Finished prerequisites of target file 'extensions'.
The prerequisites of 'extensions' are being made.
Finished prerequisites of target file 'modules'.
The prerequisites of 'modules' are being made.
Considering target file 'extra.pods'.
File 'extra.pods' was considered already.
Considering target file 'pod/perlapi.pod'.
File 'pod/perlapi.pod' was considered already.
Considering target file 'pod/perlintern.pod'.
File 'pod/perlintern.pod' was considered already.
Considering target file 'pod/perlmodlib.pod'.
File 'pod/perlmodlib.pod' was considered already.
Considering target file 'pod/perluniprops.pod'.
File 'pod/perluniprops.pod' was considered already.
Considering target file 'pod/perl5360delta.pod'.
File 'pod/perl5360delta.pod' was considered already.
Considering target file 'modules'.
File 'modules' does not exist.
Pruning file 'extensions'.
Considering target file 'extensions'.
File 'extensions' does not exist.
Pruning file 'cflags'.
Pruning file 'nonxs_ext'.
Pruning file 'dynamic_ext'.
Pruning file 'static_ext'.
Considering target file 'nonxs_ext'.
File 'nonxs_ext' does not exist.
Pruning file 'cpan/Archive-Tar/pm_to_blib'.
Pruning file 'cpan/autodie/pm_to_blib'.

....

Pruning file 'ext/XS-APItest/pm_to_blib'.
Pruning file 'ext/XS-Typemap/pm_to_blib'.
Finished prerequisites of target file 'dynamic_ext'.
The prerequisites of 'dynamic_ext' are being made.
Considering target file 'static_ext'.
File 'static_ext' was considered already.
Finished prerequisites of target file 'extensions'.
The prerequisites of 'extensions' are being made.
Finished prerequisites of target file 'modules'.
The prerequisites of 'modules' are being made.
Finished prerequisites of target file 'pod/perltoc.pod'.
The prerequisites of 'pod/perltoc.pod' are being made.
Pruning file 'miniperl'.
Finished prerequisites of target file 'install.man'.
The prerequisites of 'install.man' are being made.
Considering target file 'install.man'.
File 'install.man' does not exist.
Considering target file 'installman'.
File 'installman' was considered already.
Considering target file 'pod/perltoc.pod'.
Considering target file 'pod/buildtoc'.
File 'pod/buildtoc' was considered already.
Considering target file 'miniperl'.
File 'miniperl' was considered already.
Considering target file 'modules'.
File 'modules' does not exist.
Considering target file 'extensions'.
File 'extensions' does not exist.
Considering target file 'cflags'.
File 'cflags' was considered already.
Considering target file 'nonxs_ext'.
File 'nonxs_ext' does not exist.
Considering target file 'cpan/Archive-Tar/pm_to_blib'.
File 'cpan/Archive-Tar/pm_to_blib' was considered already.

and then it repeats in never-ending loop, only make using 100% CPU and even on smaller log:

120M install.man-make-gentoo-4.3.j2.trace.d.log

it considers nonxs_ext too many times (before I've killed the make):

x86_64-linux/perl-native/5.36.0-r0/perl-5.36.0 $ grep Considering.target.file..nonxs_ext install.man-make-gentoo-4.3.j2.trace.d.log | wc -l
7621

Will try to narrow this even more and produce the debug/trace output from working case for comparison, just wanted to share the update in case someone sees what's wrong right away.

I still wasn't able to reproduce this outside poky build "bitbake libgcc-initial -f -c compile" with INHERIT += "ccache" in local.conf in build dir without sstate-cache populated isn't enough to reproduce what buildoptions.ImageOptionsTests.test_ccache_tool somehow triggers, but YMMV, let me know if you have seen this or can reproduce it with selftest as well.

Cheers,

On Thu, Jun 9, 2022 at 6:48 PM Martin Jansa via lists.openembedded.org <Martin.Jansa=gmail.com@lists.openembedded.org> wrote:

I was reading
https://bugzilla.yoctoproject.org/show_bug.cgi?id=14096
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13244
and your e-mail about the make issue in perl, but in my case it looks a bit different I think.

It was with the make-native dependency, but happens only in selftest test_ccache_tool:

2022-06-09 11:30:21,855 - oe-selftest - INFO - test_ccache_tool (buildoptions.ImageOptionsTests.test_ccache_tool)
2022-06-09 12:53:41,855 - oe-selftest - INFO - Keepalive message
2022-06-09 14:17:01,856 - oe-selftest - INFO - Keepalive message
2022-06-09 15:40:21,856 - oe-selftest - INFO - Keepalive message
^^ was what triggered me to look where it got stuck.

I can build perl-native fine in my other oe-core nodistro build directory and now I've built it ok also in regular poky build directory as in:
5f249a407f6 vim: put xxd in its own package

While the selftest was stuck I was checking where exactly it got stuck in the WORKDIR and
oe_runmake -j 1 installman 'DESTDIR=/OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/image' install 2>&1 | tee installman.log
finished fine, while:
oe_runmake -j 2 installman 'DESTDIR=/OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/image' install 2>&1 | tee installman.log
was getting stuck:

....
Generating a Unix-style Makefile
Writing Makefile for Sys::Hostname
cd ext/XS-Typemap/ && ../../miniperl_top -I../../lib Makefile.PL \
INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none \
PERL_CORE=1 LIBPERL_A=libperl.so.5.36.0 PERL="../../miniperl_top"
Generating a Unix-style Makefile
Writing Makefile for XS::APItest
Generating a Unix-style Makefile
Writing Makefile for XS::Typemap

With -n, it looks like it got stuck just before building buildtoc:

....
cd ext/XS-APItest/ && ../../miniperl_top -I../../lib Makefile.PL \
INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none \
PERL_CORE=1 LIBPERL_A=libperl.so.5.36.0 PERL="../../miniperl_top"
cd ext/XS-Typemap/ && ../../miniperl_top -I../../lib Makefile.PL \
INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none \
PERL_CORE=1 LIBPERL_A=libperl.so.5.36.0 PERL="../../miniperl_top"
./miniperl_top -f pod/buildtoc -q
./miniperl_top installman --destdir=/OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/image
make[1]: Leaving directory '/OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/perl-5.36.0'

It didn't start miniperl_top (which itself works fine):
martin@jama /OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/perl-5.36.0 $ ./miniperl_top -f pod/buildtoc -q
martin@jama /OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/perl-5.36.0 $ echo $?
0

martin 3016692 0.0 0.0 99860 2788 ? S 17:47 0:00 make -j 40 DESTDIR=/OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/image install
martin 3016738 99.9 0.0 101440 4184 ? R 17:47 56:21 make install.man

Attaching to process 3016738
Reading symbols from /OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/recipe-sysroot-native/usr/bin/make...
(No debugging symbols found in /OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/recipe-sysroot-native/usr/bin/make)
Reading symbols from /OE/build/poky/build-st/tmp/sysroots-uninative/x86_64-linux/lib/libc.so.6...
Reading symbols from /OE/build/poky/build-st/tmp/sysroots-uninative/x86_64-linux/lib/.debug/libc.so.6...
Reading symbols from /OE/build/poky/build-st/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2...
Reading symbols from /OE/build/poky/build-st/tmp/sysroots-uninative/x86_64-linux/lib/.debug/ld-linux-x86-64.so.2...
(No debugging symbols found in /OE/build/poky/build-st/tmp/sysroots-uninative/x86_64-linux/lib/.debug/ld-linux-x86-64.so.2)
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib64/libthread_db.so.1".
0x000055c946453a1f in ?? ()
(gdb) bt
#0 0x000055c946453a1f in ?? ()
#1 0x000055c946452bf3 in ?? ()
#2 0x000055c946453b98 in ?? ()
#3 0x000055c946452bf3 in ?? ()
#4 0x000055c946453b98 in ?? ()
#5 0x000055c946452bf3 in ?? ()
#6 0x000055c946453b98 in ?? ()
#7 0x000055c946452bf3 in ?? ()
#8 0x000055c946453b98 in ?? ()
#9 0x000055c946452bf3 in ?? ()
#10 0x000055c946453fd6 in update_goal_chain ()
#11 0x000055c946438df3 in main ()
(gdb)

I've reproduced this twice today, so I was assuming it's just new separate issue from new perl version - that's why I was asking if someone else is seeing this behavior now.

I'll try to reproduce this with ccache enabled in regular build (I normally don't use poky nor selftest). If it fails again, I'll enable debug for make-native (or switch back to make-4.3 from gentoo).

Regards,

On Thu, Jun 9, 2022 at 6:21 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote:
On Thu, 2022-06-09 at 14:44 +0200, Martin Jansa wrote:
> On Mon, Jun 6, 2022 at 2:02 PM Alexander Kanavin <alex.kanavin@gmail.com> wrote:
> > Signed-off-by: Alexander Kanavin <alex@linutronix.de>
> > ---
> > .../{perlcross_1.3.7.bb => perlcross_1.4.bb} | 2 +-
> > .../perl/files/perl-configpm-switch.patch | 2 +-
> > .../perl/files/perl-dynloader.patch | 2 +-
> > .../perl/files/perl-rdepends.txt | 49 ++++++++++++-------
> > .../perl/{perl_5.34.1.bb => perl_5.36.0.bb} | 2 +-
> > 5 files changed, 35 insertions(+), 22 deletions(-)
> > rename meta/recipes-devtools/perl-cross/{perlcross_1.3.7.bb => perlcross_1.4.bb} (92%)
> > rename meta/recipes-devtools/perl/{perl_5.34.1.bb => perl_5.36.0.bb} (99%)
> >
>
>
> Have anyone else seen perl.do_install getting stuck with this new shiny version as well?
>
> 690770 martin 20 0 10020 3700 3196 S 0.0 0.0 0:00.00 │ └─ /bin/sh /OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/temp/run.do_install.690768
> 690771 martin 20 0 99860 1948 1364 S 0.0 0.0 0:00.00 │ └─ make -j 40 DESTDIR=/OE/build/poky/build-st/tmp/work/x86_64-linux/perl-native/5.36.0-r0/image install
> 690816 martin 20 0 99M 3340 1428 R 102.1 0.0 3h06:44 │ └─ make install.man

I've seen perl and the kernel hang in make for some versions, I did try
and put a warning about the known Fedora issue. Was this with the
recent make-native dependency added? If so that would indicate it is
potentially a bug in the current make version :(

Cheers,

Richard

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#166786): https://lists.openembedded.org/g/openembedded-core/message/166786
Mute This Topic: https://lists.openembedded.org/mt/91575167/3617156
Group Owner: openembedded-core+owner@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Martin.Jansa@gmail.com]
-=-=-=-=-=-=-=-=-=-=-=-