From mboxrd@z Thu Jan  1 00:00:00 1970
From: Simon Glass <sjg@chromium.org>
Date: Sun, 23 Feb 2020 19:34:05 -0700
Subject: [U-Boot] Sharing a hardware lab
In-Reply-To: <9868edb1-b906-8b1f-1482-355a0728fa82@denx.de>
References: <CAPnjgZ3e0RgRx+CsHUna1S4yM_Zdn1LG27DzwDGBrLVpMRH3WA@mail.gmail.com>
 <20191204223038.GC9549@bill-the-cat>
 <CAPnjgZ3UXZXf2hwOLMuxhsXBf4A59zczj8C87WPF9cdCCSO=7A@mail.gmail.com>
 <eaa5d7af-32d2-8f34-0baa-2a2324649a33@denx.de>
 <CAPnjgZ3st6hLy5fP5kzP1jf5PriZLHcoM_wFaozhpjERagV_fw@mail.gmail.com>
 <9868edb1-b906-8b1f-1482-355a0728fa82@denx.de>
Message-ID: <CAPnjgZ2peOt1PjF=WoOQeYM70HGxSFzOFOvUrY5sOkaV4K8xeg@mail.gmail.com>
List-Id: <u-boot.lists.denx.de>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
To: u-boot@lists.denx.de

Hi Heiko,

Thanks for the hints! I pushed your things here:

https://github.com/sglass68/tbot/tree/simon

Then I try:
tbot -l kea.py -b pcduino3.py uboot_build

and get an error:

tbot starting ...
type <class 'module'>
??Calling uboot_build ...
?   ??Fail. (0.000s)
??Exception:
?   Traceback (most recent call last):
?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/main.py",
line 318, in main
?       func(**params)
?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/decorators.py",
line 103, in wrapped
?       result = tc(*args, **kwargs)
?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/tc/uboot/build.py",
line 271, in _build
?       builder = UBootBuilder._get_selected_builder()
?     File "/home/sglass/.local/lib/python3.6/site-packages/tbot-0.8.0-py3.6.egg/tbot/tc/uboot/build.py",
line 160, in _get_selected_builder
?       builder = getattr(tbot.selectable.UBootMachine, "build")
?   AttributeError: type object 'UBootMachine' has no attribute 'build'

I'm a bit lost in all the classes and functions. A working example or
a patch would be great!

I've pushed all my scripts here:

https://github.com/sglass68/ubtest

The top commit is your patches.

Regards,
Simon

On Wed, 12 Feb 2020 at 22:49, Heiko Schocher <hs@denx.de> wrote:
>
> Hello Simon,
>
> Am 12.02.2020 um 18:14 schrieb Simon Glass:
> > Hi Heiko,
> >
> > On Wed, 12 Feb 2020 at 01:50, Heiko Schocher <hs@denx.de> wrote:
> >>
> >> Hello Simon,
> >>
> >> Am 05.02.2020 um 15:10 schrieb Simon Glass:
> >>> Hi Tom,
> >>>
> >>> On Wed, 4 Dec 2019 at 15:30, Tom Rini <trini@konsulko.com> wrote:
> >>>>
> >>>> On Fri, Nov 29, 2019 at 09:23:43PM -0700, Simon Glass wrote:
> >>>>
> >>>>> Hi Tom,
> >>>>>
> >>>>> I have been meaning to have a crack at setting up a little hardware
> >>>>> lab for a while.
> >>>>>
> >>>>> I made some progress recently and hooked up a rpi_3 with sdwire for
> >>>>> USB/SD, ykush for power and a little computer to control it. It builds
> >>>>> U-Boot, sticks it on the SD card and runs pytest.
> >>>>>
> >>>>> I pushed a tree here and hopefully you can see the 'hwlab' thing at the end:
> >>>>>
> >>>>> https://gitlab.denx.de/u-boot/custodians/u-boot-dm/pipelines/148
> >>>>>
> >>>>> So far it is just running the 'help' test. It seems to hang with
> >>>>> serial console problems if I try to do more. It is not 100% reliable
> >>>>> yet. I based it on Stephen's test hooks:
> >>>>>
> >>>>> https://github.com/sglass68/uboot-test-hooks
> >>>>>
> >>>>> Is it possible to share this so that others can use the lab when they
> >>>>> push trees? Is it as simple as adding to the .gitlab-ci.yml file as I
> >>>>> have done here?
> >>>>>
> >>>>> https://gitlab.denx.de/u-boot/custodians/u-boot-dm/blob/gitlab-working/.gitlab-ci.yml
> >>>>>
> >>>>> I also got tbot going in a similar way, to test booting into Linux.
> >>>>> Should we look at integrating that at the same time? It should be
> >>>>> fairly easy to do.
> >>>>>
> >>>>> I have quite a lot of random boards and in principle it should not be
> >>>>> too hard to hook up some more of them, with sufficient SDwires, hubs
> >>>>> and patience.
> >>>
> >>> Bumping this thread as I have now hooked up about about 8 mostly ARM
> >>> and x86 boards and have tbot and pytest automation mostly working for
> >>> them.
> >>
> >> Great news!
> >>
> >> added Harald Seiler to cc, as he did the tbot port to python3.6.
> >>
> >> Do you have somewhere your tbot integration online?
> >
> > https://github.com/sglass68/ubtest
> >
> > But it is tbot only.  It would be good if there were a way to upstream
> > this stuff.
> >
> > For pytest I am sending upstream to:
> >
> > https://github.com/swarren/uboot-test-hooks
> >
> > BTW I have not yet got tbot to build U-Boot and write it onto the
> > board. Do you have examples for that?
>
> We have them on our gitlab server, but only private as I know.
>
> @Harald: Do you have some good examples somewhere online?
>
> May the doc help here too:
>
> http://tbot.tools/modules/tc.html#u-boot
>
> and see [1]
>
> >> I ask because on our ToDo list is to integrate pytest into tbot and may
> >> we can share work?
> >>
> >>>> There's two parts of this.  The first part I think is that we need some
> >>>> good examples of how to have one private CI job poll / monitor other
> >>>> public jobs and run.  I believe some labs do this today.  This would be
> >>>> helpful as at least personally I'm kicking my hardware tests manually.
> >>>> This is because as best I can tell there isn't a way to include an
> >>>> optional stage/portion of a CI job.
> >>>
> >>> So the model here is that people with a lab 'watch' various repos? I
> >>> think that would be useful. Stephen Warren does this I think, but I'm
> >>> not sure how the builds are kicked off.
> >>>
> >>> But what about a full public lab? E.g. is it possible to add some of
> >>> the boards I have here to every build that people do?
> >>
> >> Here begins the hard game I think, because what happens if 2 builds
> >> triggered in parallel and want to test a board in the same lab
> >> at the same time?
> >
> > The gitlab-runner thing seems to handle that.
>
> Ah, so all gitlabs need to use the same gitlab runner, ok.
>
> >> If you trigger the test with tbot, it should be easy to add a locking
> >> mechanism into tbots lab specific function power_check() [1]
> >>
> >> May in this power_check() function tbot waits until it get the board...
> >> The locking mechanism itself is lab specific.
> >>
> >>>> The second part is that long term, we need to most likely develop some
> >>>> LAVA experience as that will get us easier access to various kernelci
> >>>> labs and in turn be included in kernelci labs, when the overall SoC and
> >>>> lab support being able to test firmware.
> >>>
> >>> I wonder if these are set up for replacing firmware? It specifically
> >>> mentions boards getting bricked, so I suspect not.
> >>
> >> Unfortunately I had not yet time for looking into LAVA or kernelci.
> >
> > I haven't much, but I do wonder if we could add firmware testing to it.
>
> bye,
> Heiko
>
> [1] tbot u-boot build example for the aristainetos board
>
> add in your board config:
>
> class aristainetosUBootBuilder(lab.UBootBuilder):
>      name = "aristainetos"
>      defconfig = "aristainetos2_defconfig"
>      toolchain = "linaro-gnueabi"
>      remote = "git at gitlab.denx.de:abb/aristainetos-uboot.git"
>
>      def do_checkout(self, target: linux.Path, clean: bool) -> git.GitRepository:
>          return git.GitRepository(
>              target=target, url=self.remote, clean=clean, rev="aristainetos-denx"
>          )
>
> class aristainetosUBoot(lab.UBootMachine):
>      name = "ari-ub"
>      prompt = "=> "
>      autoboot_prompt = None
>      build = aristainetosUBootBuilder()
>
>
>
> in your lab config (just example) you need:
>
> class UBootBuilder(uboot.UBootBuilder):
>      if tbot.selectable.LabHost.name in ["pollux", "hercules"]:
>          remote = "/home/git/u-boot.git"
>
>      def do_configure(self, bh: BH, repo: git.GitRepository[BH]) -> None:
>          super().do_configure(bh, repo)
>
>          tbot.log.message("Patching U-Boot config ...")
>
>          # Add local-version tbot
>          kconfig.set_string_value(repo / ".config", "CONFIG_LOCALVERSION", "-tbot")
>
>          # Tab completion
>          kconfig.enable(repo / ".config", "CONFIG_AUTO_COMPLETE")
>          [...]
>
>
> class PolluxLab(connector.ParamikoConnector, linux.Bash, linux.Lab, linux.Builder):
> [...]
>      def toolchains(self) -> typing.Dict[str, linux.build.Toolchain]:
>          return {
>              "bootlin-armv5-eabi": linux.build.EnvSetBootlinToolchain(
>                  arch = "armv5-eabi",
>                  libc = "glibc",
>                  typ = "stable",
>                  date = "2018.11-1",
>                  ),
>              "linaro-gnueabi": linux.build.EnvSetLinaroToolchain(
>                  host_arch = "i686",
>                  arch = "arm-linux-gnueabi",
>                  date = "2018.05",
>                  gcc_vers = "7.3",
>                  gcc_subvers = "1",
>                  ),
>              "generic-armv7a": linux.build.EnvScriptToolchain(
>                  linux.Path(
>                      self,
>                      "/home/hs/toolchain/linaro/gcc-linaro-7.2.1-2017.11-i686_arm-linux-gnueabi",
>                  )
>              ),
>
> for using a toolchain from bootlin or linaro, please add the attached
> patch to tbot. If you have not installed the toolchain, this class
> downloads it. ! This patch is development state only !
>
>
> Now, if you want to build on the same machine add simply:
>
>      def build(self) -> linux.Builder:
>          return self
>
> If you want to build on other linux machines:
>
>      def build(self) -> linux.Builder:
>          if "pollux-build" in tbot.flags:
>              return builders.PolluxSSH(self)
>          elif "xpert-build" in tbot.flags:
>              return builders.XpertSSH(self)
>          elif "hercules-build" in tbot.flags:
>              return builders.HerculesSSH(self)
>          elif "threadripper-build" in tbot.flags:
>              return builders.ThreadripperSSH(self)
>
> and define for example in labs/builders.py this build machines, and you
> can select through tbot commandline flags, which build machine you
> want to use ...
>
> May we should such an example to our doc ...
> --
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: +49-8142-66989-52   Fax: +49-8142-66989-80   Email: hs at denx.de