[U-Boot] [U-Boot-Custodians] [U-Boot-Board-Maintainers] [ANN] U-Boot v2019.07-rc4 released

[Marek Vasut <marex@denx.de>]

On 7/2/19 6:04 PM, Troy Benjegerdes wrote:
> 
> 
>> On Jun 22, 2019, at 2:43 PM, Marek Vasut <marex@denx.de> wrote:
>>
>> On 6/22/19 9:12 PM, Heinrich Schuchardt wrote:
>>> On 6/22/19 8:15 PM, Simon Glass wrote:
>>>> Hi,
>>>>
>>>> On Sat, 22 Jun 2019 at 16:10, Andreas Färber <afaerber@suse.de> wrote:
>>>>>
>>>>> Hi Simon,
>>>>>
>>>>> Am 22.06.19 um 16:55 schrieb Simon Glass:
>>>>>> I'd like to better understand the benefits of the 3-month timeline.
>>>>>
>>>>> It takes time to learn about a release, package and build it, test it on
>>>>> various hardware, investigate and report errors, wait for feedback and
>>>>> fixes, rinse and repeat with the next -rc. Many people don't do this as
>>>>> their main job.
>>>>>
>>>>> If we shorten the release cycle, newer boards will get out faster (which
>>>>> is good) but the overall quality of boards not actively worked on
>>>>> (because they were working good enough before) will decay, which is bad.
>>>>> The only way to counteract that would be to automatically test on real
>>>>> hardware rather than just building, and doing that for all these masses
>>>>> of boards seems unrealistic.
>>>>
>>>> Here I think you are talking about distributions. But why not just
>>>> take every second release?
>>>>
>>>> I have certain had the experience of getting a board our of the
>>>> cupboard and finding that the latest U-Boot doesn't work, nor the one
>>>> before, nor the three before that.
>>>>
>>>> Are we actually seeing an improvement in regressions? I feel that
>>>> testing is the only way to get that.
>>>>
>>>> Perhaps we should select a small subset of boards which do get tested,
>>>> and actually have custodians build/test on those for every rc?
>>>
>>> What I have been doing before all my recent pull requests is to boot
>>> both an arm32 (Orange Pi) and and an aarch64 (Pine A64 LTS) board via
>>> bootefi and GRUB. To make this easier I am using a Raspberry with a
>>> relay board and a Tizen SD-Wire card (https://wiki.tizen.org/SDWire)
>>> controlling the system under test,
>>> cf https://pbs.twimg.com/media/D5ugi3iX4AAh1bn.jpg:large
>>> What would be needed is scripts to automate the testing including all
>>> the Python tests.
>>>
>>> It would make sense to have such test automation for all of our
>>> architectures similar to what Kernel CI (https://kernelci.org/) does.
>>
>> So who's gonna set it up and host it ?
>>
> 
> I just got the infrastructure going to do this for the HiFive Unleashed
> (RiscV port), but that’s only one board right now.
> 
> I’d propose that one of the responsibilities of being a custodian/
> maintainer for a board and/or arch is a commitment to run a
>  *simple* automated testing framework on a set of boards.
> 
> I’ve looked into KenrelCI enough to see that it seems rather
> complex to get up and running. We need a dead-simple setup
> (a few debian packages? A container? An SDcard image for a
> BeagleBone?) that can collect serial console output and power
> cycle a board.

My proposal would be that you take a few exceptionally weird, disparate,
boards and try to implement dead-simple CI around those. You'll start
hitting all kinds of weird issues in the process and the dead-simple
will quickly start turning into complicated :)

You can, however, explore the U-Boot test.py, tbot, or whatever other
test frameworks there are and try to come up with some easy setup howto.

> Eventually maybe we should have a Tizen SDWire or something
> like that, however that requires some real money for board
> development since I can’t seem to find a source for where
> I can buy an SDWire.

Not all boards boot from SD cards though, some boot from hard-to-recover
non-replaceable boot media.

> With the HiFive Unleashed in SiFive’s test lab, we use OpenOCD
> for JTAG, all I need is one USB cable and I can load U-boot via JTAG,
> and boot a recovery image, and reload the SDcard, so the SDwire is
> not really necessary for boards that have easy JTAG setup.

btw booting from JTAG and cold-booting from boot media may trigger
different bugs. You most certainly want to install the bootloader and
try both cold and warm boot.

-- 
Best regards,
Marek Vasut