All of lore.kernel.org
 help / color / mirror / Atom feed
From: Harald Seiler <hws@denx.de>
To: Simon Glass <sjg@chromium.org>, tbot List <tbot@lists.denx.de>
Cc: U-Boot Mailing List <u-boot@lists.denx.de>
Subject: Re: Question on running uboot_testpy with tbot
Date: Thu, 27 Jan 2022 15:53:50 +0100	[thread overview]
Message-ID: <e758265ca7842356b0fd245017c529eedfaa1c5f.camel@denx.de> (raw)
In-Reply-To: <CAPnjgZ0=HtTy5Q_+kqVF3C3WOROd6DZbjQ80tC9oTbsZ0TEhCQ@mail.gmail.com>

Hi Simon,

first of all, sorry for the super late response to this :/  I had it on
my list for a long time but had trouble finding enough time to revisit
the test/py integration properly.  You see, the existing test/py
integration has bitrotten quite a lot and I couldn't even get the
existing code working in my environment anymore...

I've now tried reworking all this code for a more robust solution [1].
It has reached a state where it works well for me and if you are still
interested, I'd be grateful for some feedback on how it fares for your
needs.  That said, if your lab and board config haven't changed, the old
integration might still work for you.

The new one is not yet in a tbot release but will be shortly.  In the
meantime you could check it out from the master branch and documentation
can be found here:

        https://tbot.tools/contrib/uboot.html#tbot_contrib.uboot.testpy

More regarding your actual problem below.

[1]: https://github.com/Rahix/tbot/pull/64

On Sat, 2021-10-30 at 13:34 -0600, Simon Glass wrote:
> Hi,
> 
> I am trying to run the pytests on my unit. The documentation doesn't really explain how it works.
> 
> This is what I am trying:
> 
> $ tbot -vv -l kea.py -b rpi3.py -T tbot/contrib  -p 'testpy_args=["-k", "help", "-vv"]' uboot_testpy
> tbot starting ...
> ├─Parameters:
> │     testpy_args = ['-k', 'help', '-vv']
> ├─Calling uboot_testpy ...
> │   ├─Logging in on sglass@kea:22 ...
> │   ├─[sglass] bash --norc --noprofile
> │   ├─Calling uboot_setup_testhooks ...
> │   │   ├─[sglass] echo " ${HOME}"
> │   │   │    ##  /home/sglass
> │   │   ├─[sglass] mkdir -p /home/sglass/tbot-workdir
> │   │   ├─[sglass] test -d /home/sglass/tbot-workdir/uboot-testpy-tbot
> │   │   ├─Creating FIFOs ...
> │   │   ├─[sglass] rm -rf /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_send
> │   │   ├─[sglass] mkfifo /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_send
> │   │   ├─[sglass] rm -rf /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_recv
> │   │   ├─[sglass] mkfifo /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_console_recv
> │   │   ├─[sglass] rm -rf /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_commands
> │   │   ├─[sglass] mkfifo /home/sglass/tbot-workdir/uboot-testpy-tbot/fifo_commands
> │   │   ├─[sglass] cat /home/sglass/tbot-workdir/uboot-testpy-tbot/tbot-scripts.sha256
> │   │   │    ## 2d30892b61eb713ce9413e06c4f2a0cd00d2a74b6b8c2ac6624e1e49909b1897
> │   │   ├─Hooks are up to date, skipping deployment ...
> │   │   ├─Adding hooks to $PATH ...
> │   │   ├─[sglass] echo " ${PATH}"
> │   │   │    ##
>  /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/cosarm/depot_tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/l
> ocal/games:/snap/bin:/cosarm/depot_tools:/home/sglass/.local/bin:/vid/software/devel/ubtest/u-boot-test-hooks/bin:/vid/software/devel/ubtest/standalone-hdctools
> │   │   ├─[sglass] export PATH=/home/sglass/tbot-workdir/uboot-testpy-
> tbot:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/cosarm/depot_tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/u
> sr/local/games:/snap/bin:/cosarm/depot_tools:/home/sglass/.local/bin:/vid/software/devel/ubtest/u-boot-test-hooks/bin:/vid/software/devel/ubtest/standalone-hdctools
> │   │   ├─Open console & command channels ...
> │   │   ├─[sglass] /home/sglass/tbot-workdir/uboot-testpy-tbot/tbot-console
> │   │   ├─[sglass] /home/sglass/tbot-workdir/uboot-testpy-tbot/tbot-commands
> │   │   └─Done. (0.028s)
> │   ├─Calling uboot_checkout ...
> │   │   ├─Builder: rpi3
> │   │   ├─[sglass] test -d /home/sglass/tbot-workdir/uboot-rpi3/.git
> │   │   ├─[sglass] git -C /home/sglass/tbot-workdir/uboot-rpi3 fetch
> │   │   └─Done. (0.121s)
> │   ├─[sglass] test -e /home/sglass/tbot-workdir/uboot-rpi3/.config
> │   ├─[sglass] test -e /home/sglass/tbot-workdir/uboot-rpi3/include/autoconf.mk
> │   ├─[sglass] picocom -q -b 115200 /dev/ttyusb_port1
> │   ├─POWERON (rpi3)
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --dut
> │   ├─[sglass] ykushcmd -s YK17698 -g 1
> │   │    ## 
> │   │    ## 
> │   │    ## Downstream port 1 is OFF
> │   │    ## 
> │   ├─[sglass] ykushcmd -s YK17698 -u 1
> │   ├─UBOOT (rpi3-u-boot)
> │   │    <> 
> │   │    <> 
> │   │    <> U-Boot 2020.10-rc2-00140-g392aa09f310 (Oct 30 2021 - 12:38:07 -0600)
> │   │    <> 
> │   │    <> DRAM:  992 MiB
> │   │    <> RPI 3 Model B (0xa22082)
> │   │    <> MMC:   mmc@7e202000: 0, sdhci@7e300000: 1
> │   │    <> Loading Environment from FAT... *** Warning - bad CRC, using default environment
> │   │    <> 
> │   │    <> In:    serial
> │   │    <> Out:   vidconsole
> │   │    <> Err:   vidconsole
> │   │    <> Net:   No ethernet found.
> │   │    <> Hit any key to stop autoboot:  0 
> │   │    <> U-Boot> 
> │   ├─[sglass] cd /home/sglass/tbot-workdir/uboot-rpi3
> │   ├─[sglass] ./test/py/test.py --build-dir . --board-type unknown -k help -vv
> │   │    ## +u-boot-test-flash unknown na
> │   │    ## ============================= test session starts ==============================
> │   │    ## platform linux -- Python 3.6.9, pytest-5.2.1, py-1.8.0, pluggy-0.13.0 -- /usr/bin/python3
> │   │    ## cachedir: .pytest_cache
> │   │    ## rootdir: /home/sglass/tbot-workdir/uboot-rpi3/test/py, inifile: pytest.ini
> │   │    ## collected 324 items / 322 deselected / 2 selected                              
> │   │    ## 
> │   │    ## test/py/tests/test_fpga.py::test_fpga_help SKIPPED                       [ 50%]│   ├─[sglass] ykushcmd -s YK17698 -d 1
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --ts
> │   ├─Exiting poweroff
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --dut
> │   ├─[sglass] ykushcmd -s YK17698 -g 1
> │   │    ## 
> │   │    ## 
> │   │    ## Downstream port 1 is OFF
> │   │    ## 
> │   ├─[sglass] ykushcmd -s YK17698 -u 1
> 
> │   ├─POWEROFF (rpi3)
> │   ├─[sglass] ykushcmd -s YK17698 -d 1
> │   ├─[sglass] sd-mux-ctrl --device-serial sdwire-18 --ts
> │   ├─Exiting poweroff
> │   └─Fail. (38.376s)
> ├─Exception:
> │   Traceback (most recent call last):
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/main.py", line 345, in main
> │       func(**params)
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/decorators.py", line 92, in wrapped
> │       return tc(*args, **kwargs)
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/decorators.py", line 157, in wrapped
> │       return tc(lh, *args, **kwargs)
> │     File "/home/sglass/.local/lib/python3.8/site-packages/tbot-0.9.2-py3.8.egg/tbot/tc/uboot/testpy.py", line 321, in testpy
> │       os.write(chan_console.fileno(), data)
> │   OSError: [Errno 9] Bad file descriptor
> ├─────────────────────────────────────────
> └─FAILURE (38.488s)
> 
> 
> I don't see the output appearing and I'm not sure how pytest is
> sending the commands to the board?

The challenge here is that tbot is in control of the board (because we
need seamless integration of tbot's board config) while test/py directs
what should happen.  To facilitate this, IO if funneled through tbot by
means of some fifos/named pipes on the host where test/py is running.
Roughly speaking, the setup looks like this (details may differ,
depending on config):


                          +---------------------------------------------------------+
                          |                  "build host"                           |
   +---------------+      |                  has u-boot sources                     |
   | tbot host     |      |                                            +----------+ |
   |  //====================<bh channel>==[captures stdout/stderr]======test/py   | |
   |  |  //=================<m_command channel>==[powercycle trigger]===          | |
   |  |  |    //============<m_console channel>==[board rx/tx console]==invocation| |
   |  |  |    |    |      |                                            +----------+ |
   | +----+   |    |      +---------------------------------------------------------+
   | |tbot|   |    |
   | |runs|   |    |      +------------------------------------+
   | |here|   |    |      | lab host                           |
   | +----+   |    |      | from here, we                      |
   |     |    |    |      | can connect to                     |
   |     |    |    |      | the board                          |
   |     |    |    |      |                                    |     +----------------+
   |     |    \\============<board channel>==[board console]==========    DUT         |
   |     \\=================<lh channel>==[trigger powercycle] |     |  the device    |
   |               |      +------------------------------------+     | that is tested |
   +---------------+                                                 +----------------+

The board console is essentially forwared as is between test/py and the
board.  This is technically handled by tbot, but data is passed through
verbatim.

The m_command channel is used to notify when test/py requests a
powercycle of the board.  tbot then calls the relevant methods from the
board config.

The "bh" channel is where test/py is actually executed and from where
pytest stdout is captured.

Lastly, we usually have an lh channel where the command to actually do a
powercycle runs.

------

That all said, I am not really sure what is going wrong in your case.
The traceback frame

> │     File "tbot/tc/uboot/testpy.py", line 321, in testpy
> │       os.write(chan_console.fileno(), data)
> │   OSError: [Errno 9] Bad file descriptor

indicates that writing to the board's console failed (chan_console is
the <board channel> from above). To me, this means either the terminal
emulator you're using shut down (unlikely).  Or the channel you are
using does not support `.fileno()` correctly/returns some wrong file
descriptor?

Maybe try a different connector if at all possible?  E.g. using
`SSHConnector` instead of `ParamikoConnector` or using
`ConsoleConnector` instead of `PyserialConnector`...

If you can share you configs, I can take a closer look.

Regards,
-- 
Harald

  parent reply	other threads:[~2022-01-27 14:54 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-30 19:34 Question on running uboot_testpy with tbot Simon Glass
2021-10-30 19:47 ` Simon Glass
2021-12-24 12:47   ` Simon Glass
2022-01-27 14:53 ` Harald Seiler [this message]
2022-01-27 16:05   ` Simon Glass
2022-02-01  9:35     ` Harald Seiler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e758265ca7842356b0fd245017c529eedfaa1c5f.camel@denx.de \
    --to=hws@denx.de \
    --cc=sjg@chromium.org \
    --cc=tbot@lists.denx.de \
    --cc=u-boot@lists.denx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.