[U-Boot] Fwd: Parallel build is broken

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

---------- Forwarded message ---------
From: Andy Shevchenko <andy.shevchenko@gmail.com>
Date: Tue, Sep 4, 2018 at 3:40 PM
Subject: Parallel build is broken
To: Tom Rini <trini@konsulko.com>
Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>


U-Boot can't be build with make -j16.

Second pass passes fine.

Didn't investigate myself and probably will not have time (near to
vacation time for me).
So, please, fix it before release.

sh: echo: I/O error
make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
make[1]: *** Waiting for unfinished jobs....
 CC      drivers/serial/ns16550.o
 CC      drivers/spi/spi.o
 CC      cmd/mmc.o
 CC      drivers/watchdog/tangier_wdt.o
 CC      drivers/serial/serial_intel_mid.o
 CC      cmd/net.o
 CC      drivers/usb/common/common.o
 CC      arch/x86/lib/acpi.o
 LD      drivers/timer/built-in.o
 CC      arch/x86/lib/acpi_table.o
 CC      drivers/video/console_normal.o
make: *** [Makefile:1377: common] Error 2
make: *** Waiting for unfinished jobs....



--
With Best Regards,
Andy Shevchenko


-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Tom Rini]

On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:

> ---------- Forwarded message ---------
> From: Andy Shevchenko <andy.shevchenko@gmail.com>
> Date: Tue, Sep 4, 2018 at 3:40 PM
> Subject: Parallel build is broken
> To: Tom Rini <trini@konsulko.com>
> Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> 
> 
> U-Boot can't be build with make -j16.
> 
> Second pass passes fine.
> 
> Didn't investigate myself and probably will not have time (near to
> vacation time for me).
> So, please, fix it before release.
> 
> sh: echo: I/O error
> make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> make[1]: *** Waiting for unfinished jobs....
>  CC      drivers/serial/ns16550.o
>  CC      drivers/spi/spi.o
>  CC      cmd/mmc.o
>  CC      drivers/watchdog/tangier_wdt.o
>  CC      drivers/serial/serial_intel_mid.o
>  CC      cmd/net.o
>  CC      drivers/usb/common/common.o
>  CC      arch/x86/lib/acpi.o
>  LD      drivers/timer/built-in.o
>  CC      arch/x86/lib/acpi_table.o
>  CC      drivers/video/console_normal.o
> make: *** [Makefile:1377: common] Error 2
> make: *** Waiting for unfinished jobs....

What build target (and O= or not?) is this with?  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180904/171c6bfe/attachment.sig>

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
>
> > ---------- Forwarded message ---------
> > From: Andy Shevchenko <andy.shevchenko@gmail.com>
> > Date: Tue, Sep 4, 2018 at 3:40 PM
> > Subject: Parallel build is broken
> > To: Tom Rini <trini@konsulko.com>
> > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> >
> >
> > U-Boot can't be build with make -j16.
> >
> > Second pass passes fine.
> >
> > Didn't investigate myself and probably will not have time (near to
> > vacation time for me).
> > So, please, fix it before release.
> >
> > sh: echo: I/O error
> > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> > make[1]: *** Waiting for unfinished jobs....
> >  CC      drivers/serial/ns16550.o
> >  CC      drivers/spi/spi.o
> >  CC      cmd/mmc.o
> >  CC      drivers/watchdog/tangier_wdt.o
> >  CC      drivers/serial/serial_intel_mid.o
> >  CC      cmd/net.o
> >  CC      drivers/usb/common/common.o
> >  CC      arch/x86/lib/acpi.o
> >  LD      drivers/timer/built-in.o
> >  CC      arch/x86/lib/acpi_table.o
> >  CC      drivers/video/console_normal.o
> > make: *** [Makefile:1377: common] Error 2
> > make: *** Waiting for unfinished jobs....
>
> What build target (and O= or not?) is this with?  Thanks!

make clean && make edison_defconfig && make -j16

gcc (Debian 8.2.0-4) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


>
> --
> Tom



-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:
>
> On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> >
> > > ---------- Forwarded message ---------
> > > From: Andy Shevchenko <andy.shevchenko@gmail.com>
> > > Date: Tue, Sep 4, 2018 at 3:40 PM
> > > Subject: Parallel build is broken
> > > To: Tom Rini <trini@konsulko.com>
> > > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> > >
> > >
> > > U-Boot can't be build with make -j16.
> > >
> > > Second pass passes fine.
> > >
> > > Didn't investigate myself and probably will not have time (near to
> > > vacation time for me).
> > > So, please, fix it before release.
> > >
> > > sh: echo: I/O error
> > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> > > make[1]: *** Waiting for unfinished jobs....
> > >  CC      drivers/serial/ns16550.o
> > >  CC      drivers/spi/spi.o
> > >  CC      cmd/mmc.o
> > >  CC      drivers/watchdog/tangier_wdt.o
> > >  CC      drivers/serial/serial_intel_mid.o
> > >  CC      cmd/net.o
> > >  CC      drivers/usb/common/common.o
> > >  CC      arch/x86/lib/acpi.o
> > >  LD      drivers/timer/built-in.o
> > >  CC      arch/x86/lib/acpi_table.o
> > >  CC      drivers/video/console_normal.o
> > > make: *** [Makefile:1377: common] Error 2
> > > make: *** Waiting for unfinished jobs....
> >
> > What build target (and O= or not?) is this with?  Thanks!
>
> make clean && make edison_defconfig && make -j16
>
> gcc (Debian 8.2.0-4) 8.2.0
> Copyright (C) 2018 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

So, first time it's being built, consequent runs usually failed.

-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Tue, Sep 4, 2018 at 5:54 PM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:
>
> On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
> >
> > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:

> > > > U-Boot can't be build with make -j16.
> > > >
> > > > Second pass passes fine.
> > > >
> > > > Didn't investigate myself and probably will not have time (near to
> > > > vacation time for me).
> > > > So, please, fix it before release.
> > > >
> > > > sh: echo: I/O error
> > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> > > > make[1]: *** Waiting for unfinished jobs....

> > > > make: *** [Makefile:1377: common] Error 2
> > > > make: *** Waiting for unfinished jobs....
> > >
> > > What build target (and O= or not?) is this with?  Thanks!
> >
> > make clean && make edison_defconfig && make -j16
> >
> > gcc (Debian 8.2.0-4) 8.2.0
> > Copyright (C) 2018 Free Software Foundation, Inc.
> > This is free software; see the source for copying conditions.  There is NO
> > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
> So, first time it's being built, consequent runs usually failed.

More information:
- when consequential run is being executed no code had been changed
- the object files where it fails is totally arbitrary
- I used ccache, though I disabled for now completely (through
modifying PATH) and problem still exists
- Linux kernel rebuilds on that very same machine fine (like touch
include/linux/perf_event.h; make -j64 in a loop)

It's total mystery to me. It was rebuilding just fine yesterday, I
updated today from GH and even older (v2018.09-rc2) fails.

-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Tue, Sep 4, 2018 at 6:03 PM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:
>
> On Tue, Sep 4, 2018 at 5:54 PM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
> >
> > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
> > <andy.shevchenko@gmail.com> wrote:
> > >
> > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
>
> > > > > U-Boot can't be build with make -j16.
> > > > >
> > > > > Second pass passes fine.
> > > > >
> > > > > Didn't investigate myself and probably will not have time (near to
> > > > > vacation time for me).
> > > > > So, please, fix it before release.
> > > > >
> > > > > sh: echo: I/O error
> > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> > > > > make[1]: *** Waiting for unfinished jobs....
>
> > > > > make: *** [Makefile:1377: common] Error 2
> > > > > make: *** Waiting for unfinished jobs....
> > > >
> > > > What build target (and O= or not?) is this with?  Thanks!
> > >
> > > make clean && make edison_defconfig && make -j16
> > >
> > > gcc (Debian 8.2.0-4) 8.2.0
> > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > This is free software; see the source for copying conditions.  There is NO
> > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> >

Switching to gcc-7 does *not* cure.

> > So, first time it's being built, consequent runs usually failed.
>
> More information:
> - when consequential run is being executed no code had been changed
> - the object files where it fails is totally arbitrary
> - I used ccache, though I disabled for now completely (through
> modifying PATH) and problem still exists
> - Linux kernel rebuilds on that very same machine fine (like touch
> include/linux/perf_event.h; make -j64 in a loop)
>
> It's total mystery to me. It was rebuilding just fine yesterday, I
> updated today from GH and even older (v2018.09-rc2) fails.
>
> --
> With Best Regards,
> Andy Shevchenko



-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Alex Kiernan]

On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:
>
> On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
> >
> > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > >
> > > > ---------- Forwarded message ---------
> > > > From: Andy Shevchenko <andy.shevchenko@gmail.com>
> > > > Date: Tue, Sep 4, 2018 at 3:40 PM
> > > > Subject: Parallel build is broken
> > > > To: Tom Rini <trini@konsulko.com>
> > > > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> > > >
> > > >
> > > > U-Boot can't be build with make -j16.
> > > >
> > > > Second pass passes fine.
> > > >
> > > > Didn't investigate myself and probably will not have time (near to
> > > > vacation time for me).
> > > > So, please, fix it before release.
> > > >
> > > > sh: echo: I/O error
> > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> > > > make[1]: *** Waiting for unfinished jobs....
> > > >  CC      drivers/serial/ns16550.o
> > > >  CC      drivers/spi/spi.o
> > > >  CC      cmd/mmc.o
> > > >  CC      drivers/watchdog/tangier_wdt.o
> > > >  CC      drivers/serial/serial_intel_mid.o
> > > >  CC      cmd/net.o
> > > >  CC      drivers/usb/common/common.o
> > > >  CC      arch/x86/lib/acpi.o
> > > >  LD      drivers/timer/built-in.o
> > > >  CC      arch/x86/lib/acpi_table.o
> > > >  CC      drivers/video/console_normal.o
> > > > make: *** [Makefile:1377: common] Error 2
> > > > make: *** Waiting for unfinished jobs....
> > >
> > > What build target (and O= or not?) is this with?  Thanks!
> >
> > make clean && make edison_defconfig && make -j16
> >
> > gcc (Debian 8.2.0-4) 8.2.0
> > Copyright (C) 2018 Free Software Foundation, Inc.
> > This is free software; see the source for copying conditions.  There is NO
> > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
> So, first time it's being built, consequent runs usually failed.
>

I've seen this problem - building inside a VMware VM. I've always
found that logging out and back in appears to fix it, so possibly
something related to the session... I've similarly failed to figure
out what the problem is as it comes and goes.

-- 
Alex Kiernan

[U-Boot] Fwd: Parallel build is broken

[Adam Ford]

On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan <alex.kiernan@gmail.com> wrote:
>
> On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
> >
> > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
> > <andy.shevchenko@gmail.com> wrote:
> > >
> > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > >
> > > > > ---------- Forwarded message ---------
> > > > > From: Andy Shevchenko <andy.shevchenko@gmail.com>
> > > > > Date: Tue, Sep 4, 2018 at 3:40 PM
> > > > > Subject: Parallel build is broken
> > > > > To: Tom Rini <trini@konsulko.com>
> > > > > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> > > > >
> > > > >
> > > > > U-Boot can't be build with make -j16.
> > > > >
> > > > > Second pass passes fine.
> > > > >
> > > > > Didn't investigate myself and probably will not have time (near to
> > > > > vacation time for me).
> > > > > So, please, fix it before release.
> > > > >
> > > > > sh: echo: I/O error
> > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1

I am no expert, but I/O error to me indicates a hardware problem on
the build machine and not necessarily a software issue.

adam
> > > > > make[1]: *** Waiting for unfinished jobs....
> > > > >  CC      drivers/serial/ns16550.o
> > > > >  CC      drivers/spi/spi.o
> > > > >  CC      cmd/mmc.o
> > > > >  CC      drivers/watchdog/tangier_wdt.o
> > > > >  CC      drivers/serial/serial_intel_mid.o
> > > > >  CC      cmd/net.o
> > > > >  CC      drivers/usb/common/common.o
> > > > >  CC      arch/x86/lib/acpi.o
> > > > >  LD      drivers/timer/built-in.o
> > > > >  CC      arch/x86/lib/acpi_table.o
> > > > >  CC      drivers/video/console_normal.o
> > > > > make: *** [Makefile:1377: common] Error 2
> > > > > make: *** Waiting for unfinished jobs....
> > > >
> > > > What build target (and O= or not?) is this with?  Thanks!
> > >
> > > make clean && make edison_defconfig && make -j16
> > >
> > > gcc (Debian 8.2.0-4) 8.2.0
> > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > This is free software; see the source for copying conditions.  There is NO
> > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> >
> > So, first time it's being built, consequent runs usually failed.
> >
>
> I've seen this problem - building inside a VMware VM. I've always
> found that logging out and back in appears to fix it, so possibly
> something related to the session... I've similarly failed to figure
> out what the problem is as it comes and goes.
>
> --
> Alex Kiernan
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> https://lists.denx.de/listinfo/u-boot

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Wed, Sep 5, 2018 at 2:58 PM Adam Ford <aford173@gmail.com> wrote:
>
> On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan <alex.kiernan@gmail.com> wrote:
> >
> > On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko
> > <andy.shevchenko@gmail.com> wrote:
> > >
> > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
> > > <andy.shevchenko@gmail.com> wrote:
> > > >
> > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > > >
> > > > > > ---------- Forwarded message ---------
> > > > > > From: Andy Shevchenko <andy.shevchenko@gmail.com>
> > > > > > Date: Tue, Sep 4, 2018 at 3:40 PM
> > > > > > Subject: Parallel build is broken
> > > > > > To: Tom Rini <trini@konsulko.com>
> > > > > > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> > > > > >
> > > > > >
> > > > > > U-Boot can't be build with make -j16.
> > > > > >
> > > > > > Second pass passes fine.
> > > > > >
> > > > > > Didn't investigate myself and probably will not have time (near to
> > > > > > vacation time for me).
> > > > > > So, please, fix it before release.
> > > > > >
> > > > > > sh: echo: I/O error
> > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
>
> I am no expert, but I/O error to me indicates a hardware problem on
> the build machine and not necessarily a software issue.

Not exactly. AFAIR it's a message mapped to EIO errno.
The scenario can be like that (hypothetically), make runs compiler,
but instead of compiler ccache is being called and, in case it's
somehow broken, it might try to access to wrong block on the block
device which returns EIO on some (read()?) system call.

-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Trent Piepho]

On Wed, 2018-09-05 at 06:57 -0500, Adam Ford wrote:
> On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan <alex.kiernan@gmail.com> wrote:
> > 
> > On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko
> > <andy.shevchenko@gmail.com> wrote:
> > > 
> > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
> > > <andy.shevchenko@gmail.com> wrote:
> > > > 
> > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > 
> > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > > > 
> > > > > > ---------- Forwarded message ---------
> > > > > > From: Andy Shevchenko <andy.shevchenko@gmail.com>
> > > > > > Date: Tue, Sep 4, 2018 at 3:40 PM
> > > > > > Subject: Parallel build is broken
> > > > > > To: Tom Rini <trini@konsulko.com>
> > > > > > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> > > > > > 
> > > > > > 
> > > > > > U-Boot can't be build with make -j16.
> > > > > > 
> > > > > > Second pass passes fine.
> > > > > > 
> > > > > > Didn't investigate myself and probably will not have time (near to
> > > > > > vacation time for me).
> > > > > > So, please, fix it before release.
> > > > > > 
> > > > > > sh: echo: I/O error
> > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> 
> I am no expert, but I/O error to me indicates a hardware problem on
> the build machine and not necessarily a software issue.

I doubt that is the issue.  While EIO can be returned in response to a
hardware failure, it would be very random and most likely affect
something - the linker, a compiler - that process a lot of files or
data, not an echo command.


> > I've seen this problem - building inside a VMware VM. I've always
> > found that logging out and back in appears to fix it, so possibly
> > something related to the session... I've similarly failed to figure
> > out what the problem is as it comes and goes.

This is likely on track to the real problem.  What happened is
something echo did returned EIO as an errno.  That failing syscall was
probably a write(3), since echo doesn't do a whole lot else!  And why
can write(3) return EIO, besides a hardware error?

       EIO    The process is a member of  a  background  process  group
              attempting  to  write to its controlling terminal, TOSTOP
              is set, the calling thread is not blocking  SIGTTOU,  the
              process is not ignoring SIGTTOU, and the process group of
              the process is orphaned. This error may also be  returned
              under implementation-defined conditions.

I think there's a $(shell echo something &) or something like that
somewhere.

[U-Boot] Fwd: Parallel build is broken

[Max Filippov]

On Mon, Sep 10, 2018 at 11:46 AM, Trent Piepho <tpiepho@impinj.com> wrote:
> On Wed, 2018-09-05 at 06:57 -0500, Adam Ford wrote:
>> On Wed, Sep 5, 2018 at 3:46 AM Alex Kiernan <alex.kiernan@gmail.com> wrote:
>> >
>> > On Tue, Sep 4, 2018 at 3:54 PM Andy Shevchenko
>> > <andy.shevchenko@gmail.com> wrote:
>> > >
>> > > On Tue, Sep 4, 2018 at 5:50 PM Andy Shevchenko
>> > > <andy.shevchenko@gmail.com> wrote:
>> > > >
>> > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
>> > > > >
>> > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
>> > > > >
>> > > > > > ---------- Forwarded message ---------
>> > > > > > From: Andy Shevchenko <andy.shevchenko@gmail.com>
>> > > > > > Date: Tue, Sep 4, 2018 at 3:40 PM
>> > > > > > Subject: Parallel build is broken
>> > > > > > To: Tom Rini <trini@konsulko.com>
>> > > > > > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
>> > > > > >
>> > > > > >
>> > > > > > U-Boot can't be build with make -j16.
>> > > > > >
>> > > > > > Second pass passes fine.
>> > > > > >
>> > > > > > Didn't investigate myself and probably will not have time (near to
>> > > > > > vacation time for me).
>> > > > > > So, please, fix it before release.
>> > > > > >
>> > > > > > sh: echo: I/O error
>> > > > > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
>>
>> I am no expert, but I/O error to me indicates a hardware problem on
>> the build machine and not necessarily a software issue.
>
> I doubt that is the issue.  While EIO can be returned in response to a
> hardware failure, it would be very random and most likely affect
> something - the linker, a compiler - that process a lot of files or
> data, not an echo command.
>
>
>> > I've seen this problem - building inside a VMware VM. I've always
>> > found that logging out and back in appears to fix it, so possibly
>> > something related to the session... I've similarly failed to figure
>> > out what the problem is as it comes and goes.
>
> This is likely on track to the real problem.  What happened is
> something echo did returned EIO as an errno.  That failing syscall was
> probably a write(3), since echo doesn't do a whole lot else!  And why
> can write(3) return EIO, besides a hardware error?
>
>        EIO    The process is a member of  a  background  process  group
>               attempting  to  write to its controlling terminal, TOSTOP
>               is set, the calling thread is not blocking  SIGTTOU,  the
>               process is not ignoring SIGTTOU, and the process group of
>               the process is orphaned. This error may also be  returned
>               under implementation-defined conditions.
>
> I think there's a $(shell echo something &) or something like that
> somewhere.

I observe similar issue when I build QEMU except that its makefiles
use 'printf' instead of 'echo' in the 'quiet-command' macro. But I observe
the same issue even when building with V=1, i.e. when 'printf' is not
used. In that case the error comes from the make itself.
I tested a range of make versions from 3.81 to 4.2.1 and see the issue
with all of them.

When it happens under strace it looks like this (make V=0):

8341  write(1, "  CC      arch/AArch64/AArch64Di"..., 45) = -1 EAGAIN
(Resource temporarily unavailable)

or this (make V=1):

31716 write(1, "c++  -I/usr/include/pixman-1  -W"..., 1024) = 1024
31716 write(1, "tcg-op-gvec.o tcg/tcg-common.o t"..., 10240) = 7936
31716 write(1, "ut-keymap.o ../ui/input-legacy.o"..., 2304) = -1
EAGAIN (Resource temporarily unavailable)
31716 write(1, "\n", 1)                 = 1

I suspect that it's a kernel issue, as I started to notice it after the
system upgrade from Debian 8 to Debian 9.

-- 
Thanks.
-- Max

[U-Boot] Fwd: Parallel build is broken

[Tom Rini]

On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> >
> > > ---------- Forwarded message ---------
> > > From: Andy Shevchenko <andy.shevchenko@gmail.com>
> > > Date: Tue, Sep 4, 2018 at 3:40 PM
> > > Subject: Parallel build is broken
> > > To: Tom Rini <trini@konsulko.com>
> > > Cc: Bin Meng <bmeng.cn@gmail.com>, Simon Glass <sjg@chromium.org>
> > >
> > >
> > > U-Boot can't be build with make -j16.
> > >
> > > Second pass passes fine.
> > >
> > > Didn't investigate myself and probably will not have time (near to
> > > vacation time for me).
> > > So, please, fix it before release.
> > >
> > > sh: echo: I/O error
> > > make[1]: *** [scripts/Makefile.build:278: common/malloc_simple.o] Error 1
> > > make[1]: *** Waiting for unfinished jobs....
> > >  CC      drivers/serial/ns16550.o
> > >  CC      drivers/spi/spi.o
> > >  CC      cmd/mmc.o
> > >  CC      drivers/watchdog/tangier_wdt.o
> > >  CC      drivers/serial/serial_intel_mid.o
> > >  CC      cmd/net.o
> > >  CC      drivers/usb/common/common.o
> > >  CC      arch/x86/lib/acpi.o
> > >  LD      drivers/timer/built-in.o
> > >  CC      arch/x86/lib/acpi_table.o
> > >  CC      drivers/video/console_normal.o
> > > make: *** [Makefile:1377: common] Error 2
> > > make: *** Waiting for unfinished jobs....
> >
> > What build target (and O= or not?) is this with?  Thanks!
> 
> make clean && make edison_defconfig && make -j16
> 
> gcc (Debian 8.2.0-4) 8.2.0
> Copyright (C) 2018 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I assume this is on Debian/unstable?  I can't directly replicate this on
my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
setup a chroot soon, but since you've said -j64 is fine there too I
suspect you have more cores than I.  This may be something you have to
bisect for us if I can't replicate it myself.  Can you confirm how many
cores you have?  I might be able to spin something up in Google compute.
Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180904/51b885d8/attachment.sig>

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Tue, Sep 4, 2018 at 6:14 PM Tom Rini <trini@konsulko.com> wrote:
> On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:

> > make clean && make edison_defconfig && make -j16
> >
> > gcc (Debian 8.2.0-4) 8.2.0
> > Copyright (C) 2018 Free Software Foundation, Inc.
> > This is free software; see the source for copying conditions.  There is NO
> > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
> I assume this is on Debian/unstable?

testing

>  I can't directly replicate this on
> my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
> setup a chroot soon, but since you've said -j64 is fine there too I
> suspect you have more cores than I.  This may be something you have to
> bisect for us if I can't replicate it myself.  Can you confirm how many
> cores you have?  I might be able to spin something up in Google compute.

$ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
cpu cores       : 22

OTOH, I tried on my desktop (where is unstable) and everything works
fine with the same (-j16) command line on any gcc I tried.
$ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
cpu cores       : 4

-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
<andy.shevchenko@gmail.com> wrote:
>
> On Tue, Sep 4, 2018 at 6:14 PM Tom Rini <trini@konsulko.com> wrote:
> > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
>
> > > make clean && make edison_defconfig && make -j16
> > >
> > > gcc (Debian 8.2.0-4) 8.2.0
> > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > This is free software; see the source for copying conditions.  There is NO
> > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> >
> > I assume this is on Debian/unstable?
>
> testing
>
> >  I can't directly replicate this on
> > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
> > setup a chroot soon, but since you've said -j64 is fine there too I
> > suspect you have more cores than I.  This may be something you have to
> > bisect for us if I can't replicate it myself.  Can you confirm how many
> > cores you have?  I might be able to spin something up in Google compute.
>
> $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> cpu cores       : 22

Did few runs (~6) on this machine with -j4, no failures so far.
Reruning same with -j16 brings failure on ~2-3 iteration.

It seems the scope can be narrowed to:
- many cores build system, and
- Debian testing/unstable toolchain, and/or
- U-Boot build system

> OTOH, I tried on my desktop (where is unstable) and everything works
> fine with the same (-j16) command line on any gcc I tried.
> $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> cpu cores       : 4



-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Tom Rini]

On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> <andy.shevchenko@gmail.com> wrote:
> >
> > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini <trini@konsulko.com> wrote:
> > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> >
> > > > make clean && make edison_defconfig && make -j16
> > > >
> > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > This is free software; see the source for copying conditions.  There is NO
> > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > >
> > > I assume this is on Debian/unstable?
> >
> > testing
> >
> > >  I can't directly replicate this on
> > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
> > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > suspect you have more cores than I.  This may be something you have to
> > > bisect for us if I can't replicate it myself.  Can you confirm how many
> > > cores you have?  I might be able to spin something up in Google compute.
> >
> > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > cpu cores       : 22
> 
> Did few runs (~6) on this machine with -j4, no failures so far.
> Reruning same with -j16 brings failure on ~2-3 iteration.
> 
> It seems the scope can be narrowed to:
> - many cores build system, and
> - Debian testing/unstable toolchain, and/or
> - U-Boot build system

I'm pretty sure it's a dependency problem somewhere.  Since this was
working reliably for you recently (right?) and I can't reproduce it and
you can, if you can run a git bisect to figure out what commit is
breaking things, that would be very helpful.  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180904/536b5efa/attachment.sig>

[U-Boot] Fwd: Parallel build is broken

[Andy Shevchenko]

On Tue, Sep 4, 2018 at 9:00 PM Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > <andy.shevchenko@gmail.com> wrote:
> > >
> > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini <trini@konsulko.com> wrote:
> > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > >
> > > > > make clean && make edison_defconfig && make -j16
> > > > >
> > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > This is free software; see the source for copying conditions.  There is NO
> > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > > >
> > > > I assume this is on Debian/unstable?
> > >
> > > testing
> > >
> > > >  I can't directly replicate this on
> > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
> > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > suspect you have more cores than I.  This may be something you have to
> > > > bisect for us if I can't replicate it myself.  Can you confirm how many
> > > > cores you have?  I might be able to spin something up in Google compute.
> > >
> > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > cpu cores       : 22
> >
> > Did few runs (~6) on this machine with -j4, no failures so far.
> > Reruning same with -j16 brings failure on ~2-3 iteration.
> >
> > It seems the scope can be narrowed to:
> > - many cores build system, and
> > - Debian testing/unstable toolchain, and/or
> > - U-Boot build system
>
> I'm pretty sure it's a dependency problem somewhere.  Since this was
> working reliably for you recently (right?) and I can't reproduce it and
> you can, if you can run a git bisect to figure out what commit is
> breaking things, that would be very helpful.  Thanks!

Now after my vacation.
So, I guess it's not so critical for release as I thought at the beginning.

-- 
With Best Regards,
Andy Shevchenko

[U-Boot] Fwd: Parallel build is broken

[Tom Rini]

On Tue, Sep 04, 2018 at 09:05:55PM +0300, Andy Shevchenko wrote:
> On Tue, Sep 4, 2018 at 9:00 PM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > > <andy.shevchenko@gmail.com> wrote:
> > > >
> > > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > >
> > > > > > make clean && make edison_defconfig && make -j16
> > > > > >
> > > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > > This is free software; see the source for copying conditions.  There is NO
> > > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > > > >
> > > > > I assume this is on Debian/unstable?
> > > >
> > > > testing
> > > >
> > > > >  I can't directly replicate this on
> > > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
> > > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > > suspect you have more cores than I.  This may be something you have to
> > > > > bisect for us if I can't replicate it myself.  Can you confirm how many
> > > > > cores you have?  I might be able to spin something up in Google compute.
> > > >
> > > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > > cpu cores       : 22
> > >
> > > Did few runs (~6) on this machine with -j4, no failures so far.
> > > Reruning same with -j16 brings failure on ~2-3 iteration.
> > >
> > > It seems the scope can be narrowed to:
> > > - many cores build system, and
> > > - Debian testing/unstable toolchain, and/or
> > > - U-Boot build system
> >
> > I'm pretty sure it's a dependency problem somewhere.  Since this was
> > working reliably for you recently (right?) and I can't reproduce it and
> > you can, if you can run a git bisect to figure out what commit is
> > breaking things, that would be very helpful.  Thanks!
> 
> Now after my vacation.
> So, I guess it's not so critical for release as I thought at the beginning.

I'd love to fix it, but I can't replicate it.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20180904/3d259907/attachment.sig>

[U-Boot] Fwd: Parallel build is broken

[Bin Meng]

Hi Andy,

On Wed, Sep 5, 2018 at 5:43 AM Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Sep 04, 2018 at 09:05:55PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 9:00 PM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > > > <andy.shevchenko@gmail.com> wrote:
> > > > >
> > > > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > > >
> > > > > > > make clean && make edison_defconfig && make -j16
> > > > > > >
> > > > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > > > This is free software; see the source for copying conditions.  There is NO
> > > > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > > > > >
> > > > > > I assume this is on Debian/unstable?
> > > > >
> > > > > testing
> > > > >
> > > > > >  I can't directly replicate this on
> > > > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
> > > > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > > > suspect you have more cores than I.  This may be something you have to
> > > > > > bisect for us if I can't replicate it myself.  Can you confirm how many
> > > > > > cores you have?  I might be able to spin something up in Google compute.
> > > > >
> > > > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > > > cpu cores       : 22
> > > >
> > > > Did few runs (~6) on this machine with -j4, no failures so far.
> > > > Reruning same with -j16 brings failure on ~2-3 iteration.
> > > >
> > > > It seems the scope can be narrowed to:
> > > > - many cores build system, and
> > > > - Debian testing/unstable toolchain, and/or
> > > > - U-Boot build system
> > >
> > > I'm pretty sure it's a dependency problem somewhere.  Since this was
> > > working reliably for you recently (right?) and I can't reproduce it and
> > > you can, if you can run a git bisect to figure out what commit is
> > > breaking things, that would be very helpful.  Thanks!
> >
> > Now after my vacation.
> > So, I guess it's not so critical for release as I thought at the beginning.
>
> I'd love to fix it, but I can't replicate it.

I can't reproduce this on my machine too, on Ubuntu 16.04 with either
native gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10), or
the kernel.org gcc (gcc version 7.3.0 (GCC)), using below commands:

$ make clean && make edison_defconfig && make -j64

Regards,
Bin

[U-Boot] Fwd: Parallel build is broken

[Joakim Tjernlund]

On Tue, 2018-09-04 at 17:43 -0400, Tom Rini wrote:
> CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> 
> On Tue, Sep 04, 2018 at 09:05:55PM +0300, Andy Shevchenko wrote:
> > On Tue, Sep 4, 2018 at 9:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > 
> > > On Tue, Sep 04, 2018 at 07:33:10PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Sep 4, 2018 at 6:47 PM Andy Shevchenko
> > > > <andy.shevchenko@gmail.com> wrote:
> > > > > 
> > > > > On Tue, Sep 4, 2018 at 6:14 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > > On Tue, Sep 04, 2018 at 05:50:33PM +0300, Andy Shevchenko wrote:
> > > > > > > On Tue, Sep 4, 2018 at 5:00 PM Tom Rini <trini@konsulko.com> wrote:
> > > > > > > > On Tue, Sep 04, 2018 at 03:42:05PM +0300, Andy Shevchenko wrote:
> > > > > > > make clean && make edison_defconfig && make -j16
> > > > > > > 
> > > > > > > gcc (Debian 8.2.0-4) 8.2.0
> > > > > > > Copyright (C) 2018 Free Software Foundation, Inc.
> > > > > > > This is free software; see the source for copying conditions.  There is NO
> > > > > > > warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > > > > > 
> > > > > > I assume this is on Debian/unstable?
> > > > > 
> > > > > testing
> > > > > 
> > > > > >  I can't directly replicate this on
> > > > > > my 24core (Debian/stretch) or 16core(Ubuntu/xenial) machines.  I'll
> > > > > > setup a chroot soon, but since you've said -j64 is fine there too I
> > > > > > suspect you have more cores than I.  This may be something you have to
> > > > > > bisect for us if I can't replicate it myself.  Can you confirm how many
> > > > > > cores you have?  I might be able to spin something up in Google compute.
> > > > > 
> > > > > $ sed -n -e '/cpu cores/ { p; q }' /proc/cpuinfo
> > > > > cpu cores       : 22
> > > > 
> > > > Did few runs (~6) on this machine with -j4, no failures so far.
> > > > Reruning same with -j16 brings failure on ~2-3 iteration.
> > > > 
> > > > It seems the scope can be narrowed to:
> > > > - many cores build system, and
> > > > - Debian testing/unstable toolchain, and/or
> > > > - U-Boot build system
> > > 
> > > I'm pretty sure it's a dependency problem somewhere.  Since this was
> > > working reliably for you recently (right?) and I can't reproduce it and
> > > you can, if you can run a git bisect to figure out what commit is
> > > breaking things, that would be very helpful.  Thanks!
> > 

There was a patch recently to make 4.2.1 to fix some parallel build issue.
No idea if that fix applies to your case though.

       Jocke