[U-Boot] Sharing a hardware lab

[Harald Seiler <hws@denx.de>]

Hello Simon,

On Sun, 2020-02-23 at 19:34 -0700, Simon Glass wrote:
> Hi Heiko,
> 
> Thanks for the hints! I pushed your things here:
> 
> https://github.com/sglass68/tbot/tree/simon
> 
> Then I try:
> tbot -l kea.py -b pcduino3.py uboot_build
> 
> and get an error:
> 
> tbot starting ...
> type <class 'module'>
> ??Calling uboot_build ...
> ?   ??Fail. (0.000s)
> ??Exception:
> ?   Traceback (most recent call last):
> ?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/main.py",
> line 318, in main
> ?       func(**params)
> ?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/decorators.py",
> line 103, in wrapped
> ?       result = tc(*args, **kwargs)
> ?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/tc/uboot/build.py",
> line 271, in _build
> ?       builder = UBootBuilder._get_selected_builder()
> ?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/tc/uboot/build.py",
> line 160, in _get_selected_builder
> ?       builder = getattr(tbot.selectable.UBootMachine, "build")
> ?   AttributeError: type object 'UBootMachine' has no attribute 'build'
> 
> I'm a bit lost in all the classes and functions. A working example or
> a patch would be great!
> 
> I've pushed all my scripts here:
> 
> https://github.com/sglass68/ubtest
> 
> The top commit is your patches.

I think you mixed a few things up while adding Heiko's stuff.  Instead
of your last commit, attempt the attached patch.  It is untested of
course but should point you in the right direction; here is a short
summary of what it adds:

    1. Your `kea` lab needs to be marked as a build-host.  This means it
       needs the `toolchains` property which returns what toolchains are
       available.  I added a dummy armv7-a toolchain which might need
       a bit more adjustment to work for you.

    2. I created a UBootBuilder for pcduino3.  This builder just
       specifies what defconfig to build and what toolchain to use (need
       to be defined in the selected lab).

       Heiko's builder config is a bit more elaborate and also does some
       patching after applying the defconfig.  This is of course also
       possible if you want to.

    3. I added a U-Boot config for your board which is needed because
       its `build` property specifies which U-Boot builder to use.  This
       will make the `uboot_build` testcase work properly.  Later you
       might want to adjust this U-Boot config to actually work so you
       can make use of it for flashing the new U-Boot binary.

Some more links to documentation:

    - Build-host config: https://tbot.tools/modules/machine_linux.html#builder
    - UBootBuilder class: https://tbot.tools/modules/tc.html#tbot.tc.uboot.UBootBuilder

Hope this helps!
-- 
Harald

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-62  Fax: +49-8142-66989-80   Email: hws at denx.de
 
> Regards,
> Simon
> 
> On Wed, 12 Feb 2020 at 22:49, Heiko Schocher <hs@denx.de> wrote:
> > Hello Simon,
> > 
> > Am 12.02.2020 um 18:14 schrieb Simon Glass:
> > > Hi Heiko,
> > > 
> > > On Wed, 12 Feb 2020 at 01:50, Heiko Schocher <hs@denx.de> wrote:
> > > > Hello Simon,
> > > > 
> > > > Am 05.02.2020 um 15:10 schrieb Simon Glass:
> > > > > Hi Tom,
> > > > > 
> > > > > On Wed, 4 Dec 2019 at 15:30, Tom Rini <trini@konsulko.com> wrote:
> > > > > > On Fri, Nov 29, 2019 at 09:23:43PM -0700, Simon Glass wrote:
> > > > > > 
> > > > > > > Hi Tom,
> > > > > > > 
> > > > > > > I have been meaning to have a crack at setting up a little hardware
> > > > > > > lab for a while.
> > > > > > > 
> > > > > > > I made some progress recently and hooked up a rpi_3 with sdwire for
> > > > > > > USB/SD, ykush for power and a little computer to control it. It builds
> > > > > > > U-Boot, sticks it on the SD card and runs pytest.
> > > > > > > 
> > > > > > > I pushed a tree here and hopefully you can see the 'hwlab' thing at the end:
> > > > > > > 
> > > > > > > https://gitlab.denx.de/u-boot/custodians/u-boot-dm/pipelines/148
> > > > > > > 
> > > > > > > So far it is just running the 'help' test. It seems to hang with
> > > > > > > serial console problems if I try to do more. It is not 100% reliable
> > > > > > > yet. I based it on Stephen's test hooks:
> > > > > > > 
> > > > > > > https://github.com/sglass68/uboot-test-hooks
> > > > > > > 
> > > > > > > Is it possible to share this so that others can use the lab when they
> > > > > > > push trees? Is it as simple as adding to the .gitlab-ci.yml file as I
> > > > > > > have done here?
> > > > > > > 
> > > > > > > https://gitlab.denx.de/u-boot/custodians/u-boot-dm/blob/gitlab-working/.gitlab-ci.yml
> > > > > > > 
> > > > > > > I also got tbot going in a similar way, to test booting into Linux.
> > > > > > > Should we look at integrating that at the same time? It should be
> > > > > > > fairly easy to do.
> > > > > > > 
> > > > > > > I have quite a lot of random boards and in principle it should not be
> > > > > > > too hard to hook up some more of them, with sufficient SDwires, hubs
> > > > > > > and patience.
> > > > > 
> > > > > Bumping this thread as I have now hooked up about about 8 mostly ARM
> > > > > and x86 boards and have tbot and pytest automation mostly working for
> > > > > them.
> > > > 
> > > > Great news!
> > > > 
> > > > added Harald Seiler to cc, as he did the tbot port to python3.6.
> > > > 
> > > > Do you have somewhere your tbot integration online?
> > > 
> > > https://github.com/sglass68/ubtest
> > > 
> > > But it is tbot only.  It would be good if there were a way to upstream
> > > this stuff.
> > > 
> > > For pytest I am sending upstream to:
> > > 
> > > https://github.com/swarren/uboot-test-hooks
> > > 
> > > BTW I have not yet got tbot to build U-Boot and write it onto the
> > > board. Do you have examples for that?
> > 
> > We have them on our gitlab server, but only private as I know.
> > 
> > @Harald: Do you have some good examples somewhere online?
> > 
> > May the doc help here too:
> > 
> > http://tbot.tools/modules/tc.html#u-boot
> > 
> > and see [1]
> > 
> > > > I ask because on our ToDo list is to integrate pytest into tbot and may
> > > > we can share work?
> > > > 
> > > > > > There's two parts of this.  The first part I think is that we need some
> > > > > > good examples of how to have one private CI job poll / monitor other
> > > > > > public jobs and run.  I believe some labs do this today.  This would be
> > > > > > helpful as at least personally I'm kicking my hardware tests manually.
> > > > > > This is because as best I can tell there isn't a way to include an
> > > > > > optional stage/portion of a CI job.
> > > > > 
> > > > > So the model here is that people with a lab 'watch' various repos? I
> > > > > think that would be useful. Stephen Warren does this I think, but I'm
> > > > > not sure how the builds are kicked off.
> > > > > 
> > > > > But what about a full public lab? E.g. is it possible to add some of
> > > > > the boards I have here to every build that people do?
> > > > 
> > > > Here begins the hard game I think, because what happens if 2 builds
> > > > triggered in parallel and want to test a board in the same lab
> > > > at the same time?
> > > 
> > > The gitlab-runner thing seems to handle that.
> > 
> > Ah, so all gitlabs need to use the same gitlab runner, ok.
> > 
> > > > If you trigger the test with tbot, it should be easy to add a locking
> > > > mechanism into tbots lab specific function power_check() [1]
> > > > 
> > > > May in this power_check() function tbot waits until it get the board...
> > > > The locking mechanism itself is lab specific.
> > > > 
> > > > > > The second part is that long term, we need to most likely develop some
> > > > > > LAVA experience as that will get us easier access to various kernelci
> > > > > > labs and in turn be included in kernelci labs, when the overall SoC and
> > > > > > lab support being able to test firmware.
> > > > > 
> > > > > I wonder if these are set up for replacing firmware? It specifically
> > > > > mentions boards getting bricked, so I suspect not.
> > > > 
> > > > Unfortunately I had not yet time for looking into LAVA or kernelci.
> > > 
> > > I haven't much, but I do wonder if we could add firmware testing to it.
> > 
> > bye,
> > Heiko
> > 
> > [1] tbot u-boot build example for the aristainetos board
> > 
> > add in your board config:
> > 
> > class aristainetosUBootBuilder(lab.UBootBuilder):
> >      name = "aristainetos"
> >      defconfig = "aristainetos2_defconfig"
> >      toolchain = "linaro-gnueabi"
> >      remote = "git at gitlab.denx.de:abb/aristainetos-uboot.git"
> > 
> >      def do_checkout(self, target: linux.Path, clean: bool) -> git.GitRepository:
> >          return git.GitRepository(
> >              target=target, url=self.remote, clean=clean, rev="aristainetos-denx"
> >          )
> > 
> > class aristainetosUBoot(lab.UBootMachine):
> >      name = "ari-ub"
> >      prompt = "=> "
> >      autoboot_prompt = None
> >      build = aristainetosUBootBuilder()
> > 
> > 
> > 
> > in your lab config (just example) you need:
> > 
> > class UBootBuilder(uboot.UBootBuilder):
> >      if tbot.selectable.LabHost.name in ["pollux", "hercules"]:
> >          remote = "/home/git/u-boot.git"
> > 
> >      def do_configure(self, bh: BH, repo: git.GitRepository[BH]) -> None:
> >          super().do_configure(bh, repo)
> > 
> >          tbot.log.message("Patching U-Boot config ...")
> > 
> >          # Add local-version tbot
> >          kconfig.set_string_value(repo / ".config", "CONFIG_LOCALVERSION", "-tbot")
> > 
> >          # Tab completion
> >          kconfig.enable(repo / ".config", "CONFIG_AUTO_COMPLETE")
> >          [...]
> > 
> > 
> > class PolluxLab(connector.ParamikoConnector, linux.Bash, linux.Lab, linux.Builder):
> > [...]
> >      def toolchains(self) -> typing.Dict[str, linux.build.Toolchain]:
> >          return {
> >              "bootlin-armv5-eabi": linux.build.EnvSetBootlinToolchain(
> >                  arch = "armv5-eabi",
> >                  libc = "glibc",
> >                  typ = "stable",
> >                  date = "2018.11-1",
> >                  ),
> >              "linaro-gnueabi": linux.build.EnvSetLinaroToolchain(
> >                  host_arch = "i686",
> >                  arch = "arm-linux-gnueabi",
> >                  date = "2018.05",
> >                  gcc_vers = "7.3",
> >                  gcc_subvers = "1",
> >                  ),
> >              "generic-armv7a": linux.build.EnvScriptToolchain(
> >                  linux.Path(
> >                      self,
> >                      "/home/hs/toolchain/linaro/gcc-linaro-7.2.1-2017.11-i686_arm-linux-gnueabi",
> >                  )
> >              ),
> > 
> > for using a toolchain from bootlin or linaro, please add the attached
> > patch to tbot. If you have not installed the toolchain, this class
> > downloads it. ! This patch is development state only !
> > 
> > 
> > Now, if you want to build on the same machine add simply:
> > 
> >      def build(self) -> linux.Builder:
> >          return self
> > 
> > If you want to build on other linux machines:
> > 
> >      def build(self) -> linux.Builder:
> >          if "pollux-build" in tbot.flags:
> >              return builders.PolluxSSH(self)
> >          elif "xpert-build" in tbot.flags:
> >              return builders.XpertSSH(self)
> >          elif "hercules-build" in tbot.flags:
> >              return builders.HerculesSSH(self)
> >          elif "threadripper-build" in tbot.flags:
> >              return builders.ThreadripperSSH(self)
> > 
> > and define for example in labs/builders.py this build machines, and you
> > can select through tbot commandline flags, which build machine you
> > want to use ...
> > 
> > May we should such an example to our doc ...
> > --
> > DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> > Phone: +49-8142-66989-52   Fax: +49-8142-66989-80   Email: hs at denx.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-toolchain-and-builder-for-pcduino3.patch
Type: text/x-patch
Size: 3755 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200224/70a18188/attachment.bin>