[PATCH 1/2] gitlab: Move the n900 test into its own section

["Pali Rohár" <pali@kernel.org>]

On Sunday 31 January 2021 09:51:44 Simon Glass wrote:
> Hi Pali,
> 
> On Sun, 31 Jan 2021 at 08:52, Pali Roh?r <pali@kernel.org> wrote:
> >
> > On Sunday 31 January 2021 08:43:19 Simon Glass wrote:
> > > Hi Pali,
> > >
> > > On Sun, 31 Jan 2021 at 08:04, Pali Roh?r <pali@kernel.org> wrote:
> > > >
> > > > On Sunday 31 January 2021 08:49:20 Tom Rini wrote:
> > > > > On Sun, Jan 31, 2021 at 01:15:20PM +0100, Pali Roh?r wrote:
> > > > > > On Saturday 30 January 2021 22:17:45 Simon Glass wrote:
> > > > > > > This test is not reliable. Quite often (20%?) it makes the build fail and
> > > > > > > a retry succeeds.
> > > > > >
> > > > > > This test should work. Are there any logs with issues?
> > > > >
> > > > > I don't see it failing any more often than other tests do, due to
> > > > > network connectivity issues.  That may be helped by, now that we've
> > > > > dropped Travis, having the container be pre-populated with more of the
> > > > > downloaded files and pre-building the special QEMU.
> > > >
> > > > If there are just network issue problems then pre-downloading required
> > > > files into cache / container should resolve them.
> > >
> > > The flake issues I see are like this:
> > >
> > > https://gitlab.denx.de/u-boot/custodians/u-boot-dm/-/jobs/202441
> > >
> > > I am not sure of the cause, but it would be good to fix it!
> >
> > Hello Simon! This is not a network issue problem but rather some U-Boot
> > regression in mmc code. Second test failed with error:
> >
> >     "Failed to boot kernel from eMMC"
> >
> > Other tests succeed:
> >
> >     "Kernel was successfully booted from RAM"
> >     "Kernel was successfully booted from OneNAND"
> >
> > So problem is really with second boot attempt from eMMC. U-Boot log is
> > also available in output (as second run):
> >
> >     Check if pads/pull-ups of bus are properly configured
> >     Timed out in wait_for_event: status=0000
> >     ...
> >     Timed out in wait_for_event: status=0000
> >     Check if pads/pull-ups of bus are properly configured
> >     Timed out in wait_for_event: status=0000
> >     Check if pads/pull-ups of bus are properly configured
> >     Timed out in wait_for_event: status=0000
> >     Check if pads/pull-ups of bus are properly configured
> >     test/nokia_rx51_test.sh: line 233:  5946 Killed                  ./qemu-system-arm -M n900 -mtdblock mtd_emmc.img -sd emmc_emmc.img -serial /dev/stdout -display none > qemu_emmc.log
> >
> > After 300s was qemu killed and test marked as failure.
> >
> > So this is valid failure and regression in u-boot emmc code. So it would
> > be needed to identify which commit caused it and revert it...
> 
> The problem is that it is intermittent. Can you repeat it?

So when you run this test more times from same sources / git commit,
this error appears only sometimes?

This particular issue I have not seen in qemu yet when I run tests on my
local machine. So I cannot reproduce it.

I saw similar errors, but only on real device (not in qemu) and they
were visible always (not sometimes). And for all my known problems I
have sent patches to mailing list. including i2c, mmc and usb. Some of
them are still waiting for review & merge...

===

I know only one error which is not fixed yet and happens "only
sometimes" which I was not able to debug yet. Probably if u-boot binary
has particular size then it completely crashes (and with same binary it
can be reproduced for every run). But recompiling u-boot binary resolves
this issue and sometimes even without modifying source code. So I
suspect that time&date string (which changes for every recompilation)
must have some effect (maybe some +-1 padding?). Adding new random 100
characters into env variables seems to fix it.

> >
> > > Re the network issues, I have a persistent DNS problem with my
> > > network. I am really not sure of the root cause but sometimes it will
> > > fail to find a host, then succeed 5 seconds later. I spent some time
> > > on it a few weeks ago but will try again.
> 
> Regards,
> Simon