Re: [Fuego] First steps with fuego

["Bird, Timothy" <Tim.Bird@sony.com>]

> -----Original Message-----
> From: Rafael Gago Castano on Friday, May 12, 2017 2:39 AM
>
> For some reason I didn't CC the previous mail to the mailing list but just to
> you.
> 
> > OK.  cmd *should* return the error code from the remote command.
> > If it's not, then that's a bug.
> 
> Yes, I was in a rush yesterday and I had no time to do the things properly, but
> I wrote a terse loop with while using "test -f" to poll for file existance on the
> DUT and it failed, so I wronlgy assumed that was a "cmd" error.
> 
> If I write this in the body of a test, the test gets early interrupted and never
> prints "after" (I'm using the ssh transport):
> 
>     echo "before"
>     cmd "test -f /tmp/nonexistant-file"
>     echo "after"
> 
> On the logs I see:
> 
> ++ echo before
> before
> ++ cmd 'test -f /tmp/nonexistant'
> ++ report_devlog 'cmd: test -f /tmp/nonexistant'
> ++ echo 'cmd: test -f /tmp/nonexistant'
> ++ ov_transport_cmd 'test -f /tmp/nonexistant'
> ++ case "$TRANSPORT" in
> ++ sshpass -e ssh -o ServerAliveInterval=30 -o StrictHostKeyChecking=no -o
> UserKnownHostsFile=/dev/null -o ConnectTimeout=30 -p 22
> root@192.168.1.2 'test -f /tmp/nonexistant'
> Warning: Permanently added '192.168.1.2' (ECDSA) to the list of known hosts.
> + signal_handler
> + echo 'in signal_handler'
> in signal_handler

The way Fuego core operates now, the test scripts are run with bash set -e
in effect.  That is, any command error (outside of an if, assignment, or
expression) will issue an error trap and move directly to post_test.
A command failure is interpreted as a hard error.  Possibly this is too aggressive,
but it is the same as the default behavior of Jenkins.

If you have a command that whose failure does not indicate test failure
(something optional, or a test), then you can wrap it like so:

  set +e
  cmd "this might fail, but don't stop the test"
  set -e

We have been contemplating changing this, but that's how it works right now.

> > Just FYI - I experimented a bit, and was able to do some conditional
> > waiting on the target with:
> >  ftc wait-for -t 60 "/bin/bash -c \"source $LOGDIR/prolog.sh ;
> ov_transport_cmd test -f /tmp/sync_file\""
> 
> > This could be improved syntactically, but it did work.  I envision extended
> 'ftc wait-for' to support
> > something like:
> >    ftc wait-for "target: test -f /tmp/sync_file"
> > for this style of board-side conditional test.
> 
> As fuego is writing the test for running in the host, the host is issuing shell
> comands to the DUT (kind of master-slave), so the HOST to DUT
> synchronization is already implicit. There already is a clear "happens before"
> relationship in place.
> 
> It's the other way around, the HOST syncing to the DUT that I couldn't fix. I
> was attempting something similar to this (untested):
> 
> FUEGO_SYNC_PREFIX=/tmp/fuego-sync/
> 
> # Just clear all the sync files from previous runs
> function on_test_start() {
>     cmd rm -rf $FUEGO_SYNC_PREFIX && mkdir $FUEGO_SYNC_PREFIX
>     #error handling
> }
> 
> function wait_for_dut() {
>     local FILE=$FUEGO_SYNC_PREFIX$1
>     local SEC=999999
>     if [[ ! -z $2 ]]; then
>         SEC=$2
>     fi
>     local ELAPSED=0
>     while [[ -n cmd "test -f $FILE" ]]; do
I'm not sure I follow the use of '-n' here.  Is 'test -f' noisy in one case
and silent in the other?  I would think that maybe it should be:
   while cmd "test -f $FILE" ; do

>         if [[ $ELAPSED -lt $SEC ]]; then
>             return 1
>         fi
>         sleep 1
>         ELAPSED=$((ELAPSED + 1))
>     done
>     cmd "rm $FILE"
>     return 0
> }
>

This looks like a good way to provide a generalized sync solution from target
to host, based on files.  I hope you don't mind if I copy parts of this if I implement
a synchronize function for the core. :-)

> > This is probably something that would be good to add to the board file.
> 
> I hand't thought that the board files are just sourced and that I can add
> variables there. That's powerful. In our case we have more than one serial
> port on each device though.
> 
> > That looks great.  I don't know the failure modes for the serial port
> transfers
> > so I can't say how robust this is, but it looks OK to me.  As long as stty
> observes
> > the timeouts, then things shouldn't get stuck.
> >
> > I have a minor preference for using TAP format for the test output for
> these types
> > of simple test  - basically formatting the output line as:
> > ok <testnum> test description
> > But that's not required, and I think this looks like a nice test.
> >
> > I'm trying to think how I can try this here.  I have my test board here with
> the
> > serial port connected to the serial console, but maybe I could wire up a
> second
> > one for testing.
> 
> Don't bother, I should had posted this as an RFC, I just found easier to share
> it as a patch than by copying snippets. As the test is done now it isn't
> guaranteed to work correctly, for doing this test properly the sender (HOST)
> would need to start sending (echo) after making sure that the DUT is
> listening (after the launched "dut_rx.sh" script is past the "cat" command =>
> cat launched as a subprocess). The test is failing for me today.

OK - but it's our first serial port, multi-node test, and I was excited to see it.
:-)
 -- Tim