Re: [Fuego] Executing test on slave machine connected to target board via serial port

* Re: [Fuego] Executing test on slave machine connected to target board via serial port
@ 2019-02-06 18:55 dhinakar k
  2019-02-13  3:02 ` Tim.Bird
  0 siblings, 1 reply; 45+ messages in thread
From: dhinakar k @ 2019-02-06 18:55 UTC (permalink / raw)
  To: fuego

[-- Attachment #1: Type: text/plain, Size: 18765 bytes --]

Hi Tim,

Thanks for your prompt response.
I think your second approach will be very good (SSH pass through to serial
port) as I don't need to setup any jenkins slave on the PC connected to the
target PC. Also since the PCs are on LAN I can provide SSH connectivity to
slave PCs.

I am very much interested in this -> 'Let me know what you think of this,
and maybe I can cobble together a proof of concept for you to try out.' Iam
eagerly waiting for it :)

Thanks again.
Regards,
Dhinakar

On Wed, 6 Feb 2019, 1:06 pm <fuego-request@lists.linuxfoundation.org wrote:

> Send Fuego mailing list submissions to
>         fuego@lists.linuxfoundation.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://lists.linuxfoundation.org/mailman/listinfo/fuego
> or, via email, send a message with subject or body 'help' to
>         fuego-request@lists.linuxfoundation.org
>
> You can reach the person managing the list at
>         fuego-owner@lists.linuxfoundation.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Fuego digest..."
>
>
> Today's Topics:
>
>    1. Re: [PATCH v2] rpm: Fix test case rpm02. (Tim.Bird@sony.com)
>    2. Re: Executing test on slave machine connected to target board
>       via serial port (Tim.Bird@sony.com)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 5 Feb 2019 18:47:33 +0000
> From: <Tim.Bird@sony.com>
> To: <zhengrq.fnst@cn.fujitsu.com>, <fuego@lists.linuxfoundation.org>
> Subject: Re: [Fuego] [PATCH v2] rpm: Fix test case rpm02.
> Message-ID:
>         <ECADFF3FD767C149AD96A924E7EA6EAF8E0E93A8@USCULXMSG01.am.sony.com>
> Content-Type: text/plain; charset="utf-8"
>
> I'm not sure I understand the issue here.
>
> Please see questions inline below.
>
> > -----Original Message-----
> > From: Zheng Ruoqin
> >
> > In some user's boards, the rpm provided in this suite is not fit for
> them, and
> > rpm can't be installed.
> >
> > This patch provide an srpm file for user to rebuild it to produce the
> > rpm file fit for them.
> >
> > Signed-off-by: Zheng Ruoqin <zhengrq.fnst@cn.fujitsu.com>
> > Signed-off-by: Qiu Tingting <qiutt@cn.fujitsu.com>
> > ---
> >  .../{ => data}/test-manual-1.2.3.noarch.rpm          | Bin
> >  .../Functional.rpm/data/test-manual-1.2.3.src.rpm    | Bin 0 -> 2335
> bytes
> >  engine/tests/Functional.rpm/fuego_test.sh            |  19
> > +++++++++++++++++--
> >  engine/tests/Functional.rpm/tests/rpm_02.sh          |   2 +-
> >  4 files changed, 18 insertions(+), 3 deletions(-)
> >  rename engine/tests/Functional.rpm/{ => data}/test-manual-
> > 1.2.3.noarch.rpm (100%)
> >  create mode 100644 engine/tests/Functional.rpm/data/test-manual-
> > 1.2.3.src.rpm
> >
> > diff --git a/engine/tests/Functional.rpm/test-manual-1.2.3.noarch.rpm
> > b/engine/tests/Functional.rpm/data/test-manual-1.2.3.noarch.rpm
> > similarity index 100%
> > rename from engine/tests/Functional.rpm/test-manual-1.2.3.noarch.rpm
> > rename to engine/tests/Functional.rpm/data/test-manual-1.2.3.noarch.rpm
> > diff --git a/engine/tests/Functional.rpm/data/test-manual-1.2.3.src.rpm
> > b/engine/tests/Functional.rpm/data/test-manual-1.2.3.src.rpm
> > new file mode 100644
> > index
> > 0000000000000000000000000000000000000000..9c0f45da76baed63ab88be262
> > 18c90f813ab260f
> > GIT binary patch
> > literal 2335
> > zcma)+c~DbV6o+2|Dn?OSsiGwiTo6=V-
> > %DOzlBx(55drs|#W8QgSdu^%tEjc&0xlp3
> > zZle~pfICH0MC%`nOJzivx@%o)s|7|-sue|%o*Qo++L_Mu&Yb-6-E-
> > c#=iGbWYuMj-
> > z+l?S(gv=q@9U*3;#c50qp-
> > DBV2~ktDo~CFu@qdSm=<x5*?1Qy3JcoQ!z(T<~fkl9M
> > z!P1_f9l_9mShreGcd#LtpMaux3Jl`io`NEO7?_ObC|0Ct9VZ%iLDDg-
> > PSofPtVW=9
> > zyg;k9T7lBB5~qb&7#^LX?Lmzv@1)x{24)xJgbpczRA<29s;RB5trY&l8U)d<Zm2^
> > p
> > zz|;sZs6&o=Krv{}0}PG7>qdRZG1q}2?uNM@vo~hcS7$IK<_65cm>WSq^ojdM`
> > 695t
> > z3Hv8tehG^Fs6JHBD^Qew2Idw}RIh}&6%@rU!2AyLlK1S6)<%{I20gi>Shy-
> > g4rY`a
> > zaXw}RD2m^VxeF*-ZwKbC?^)ItGdg#Om-
> > WZo9s6OwUF)GehUaC&Fn<Dy;$t!UU`G9-
> > z`g((+_yo*-Ft5cN1PVRKwt>R_xa&btzCz3fQ1tvB%%5Yvf;k-
> > XL(Gw&C|@1s(V&nI
> > zN_k&T6lTcjs&5P^{FkG?P`#*56p!u$;zaEC#XJjh7$~Y&0EPABs1C#;W>g1a33JS
> > Q
> > z?vV$I^1JTGNTS{QO}O4~;=}vh-WBb%J5)v$no`+P%qqLn<bajpE2|Qct-
> > LXLj@9mf
> > z#BlT@M^A_)BCNdAELt2!hskPbzXs_r+Q_6dBH3hd&L=EZqm7^AdQQF!q;lzzr
> > AH7#
> > z-
> > uU>+)0ah_;W)iUt2anmMoZ~<o@FGSlO#?fQnbM!NIcE5YBkT&YKB#7MM2N
> > <46Wfb
> > zdQK;3IYQFujf|EN1ihMN8CuPAf<S4EdY!<*SAo(}ypE?RJtwg=XW-
> > RFJ<W(ZJ*Cmm
> > zMxN)@_yeIrHKolFElIZ2|5W%+xCU&x0vT*mvWW|dG$o4>rX<)ndoY3LMq1
> > 4&HqmYu
> > z1>_xPG>hP4NS35RXhK6$Bm>HjG)-
> > #IUaYN*Lu;!hIjyNCi)t9Igp@3HM;LD*CFfj|
> > z!|o(`t2vYiQH`;f9IAPu&5rgSZ}EpiQqfwCLBmpdmO@Rr?g+XkE^}Q{*DU<BJAT
> > rh
> > zBJ%&3z{g1NDJCnClx9jH(0hUgy4S96Vq05l82tD0UxTLL#<^upPK@%N=>2VN5
> > gG78
> > zigJeZEH8L{+yATFD(}g$sb<eN{pP-GynA==>*qDw-
> > ^N7m$t>J7rT${y%)$)63#Zlx
> > z9jq<dF)6(;-
> > A^H>!~F*(Y`vA}%<^pYYrLbmZ@6!0PRaK2lBWk1=C*V^n>oJ9^OAqK
> > zT+z}mxVp!S;w9&H^}hGKpmf3FrCeUwzQUR7Pi^TK*>&>Zy%j!W-
> > )c8zHh2C+dDxn5
> > z8PC(HX{7?$ae+G>q}bIde`eXoAp?SPcQT1(o)fHl_eI35wu^iEpAdzj&u-
> > )vaaqw<
> > zYr4!8mLEM3yTPg5G3%JesV_)vcJ%T!$K(%Ib3Hz~6}9Tg;aEjczJLFfim@@pH($~
> > o
> > z>^IeCuBuhHJ!X0aI<5uut<eyXos}#1nd7g=4hmM>RBX*uDsq<wTnkti{P1GWJ
> > Z{6l
> > zx{al5hpUq(ZV9{IbE>iOX+}wTzUq?m=bE9x^LKmSBUg+~42!(q)NA#k(2D%xiY
> > =>s
> > z&BfJSWWCC&e<CIZ#Bech&1B%<n~u01c|ij#1FwzCi1ANPtY>p#=Ita#j|ecO9;i
> > (%
> > zRZjEjz0zyNu7O(FvnctOsXpZ|B4;l9Ax^o5vC(f*7v}tU_CTIm8`7<=-cosL(AeT<
> > zj<l0RKjG;&Nu!27$dV*w=fQVm))HmZ_^v!}|Gm%SyH!`dyHLOC()igvj|=kV`|f
> > $8
> > zz1jTB;&ZJfTi$s->a?)SdEL0?f~x(SnPYd3H$DI3!o{N<hToW*;a78FMBA&-sqxN=
> > z%?-gV70X)F#I_YTk{XA9`owq113zWe8~cHSe>VO0>S@`bm&f0ZzOnoI_glBm-
> > *B>}
> > zVN^~*qw=`3e5`5CEoPWuVe_KP$zR9Y4xPB&C+%Ky;F2fRMU7Qq(`_Ynj~b5u
> > 8nC}#
> > zzV9Go&(zv6;XZ-
> > WX^Uj)zUg#y>$_3HJDVyYZt7uO!i*WU`Axk}uA8;_417HQ0{fs{
> > Ak^lez
> >
> > literal 0
> > HcmV?d00001
> >
> > diff --git a/engine/tests/Functional.rpm/fuego_test.sh
> > b/engine/tests/Functional.rpm/fuego_test.sh
> > index 2a56a6d..b7856d8 100755
> > --- a/engine/tests/Functional.rpm/fuego_test.sh
> > +++ b/engine/tests/Functional.rpm/fuego_test.sh
> > @@ -1,19 +1,34 @@
> >  NEED_ROOT=1
> >
> > +NEED_BUILD_RPM=false
> >  function test_pre_check {
> >      is_on_target_path rpm PROGRAM_RPM
> >      assert_define PROGRAM_RPM
> > +    is_on_target_path rpmbuild PROGRAM_RPMBUILD
> > +    if [ -n $PROGRAM_RPMBUILD ];
> > +    then
> > +        NEED_BUILD_RPM=true
> > +    fi
>
> OK - just to clarify.  The logic here is that if a board has 'rpm' but not
> 'rpmbuild',
> then just use the default test rpm package for the test.  But if a board
> has
> the command 'rpmbuild' in the PATH, then use that to rebuild the
> test rpm package, and use that instead of the default test rpm package.
>
> Is that right?
>
> What is deficient about the default test rpm package, that it cannot
> be used in some circumstances?
>
> I see that you are manipulating '_topdir' in the rebuild.  Is there some
> problem
> with the test package touching things outside the Fuego test directory?
>
> Can you provide an error message or log for when using the default test rpm
> package fails on some board - that this fix is trying to address?
>
> This is a fairly heavy-weight fix.  Not many embedded products will
> have rpmbuild installed IMHO, so I hesitate to make that a requirement.
> But it seems like the test still attempts to work even if 'rpmbuild' is
> missing.
>
> But I want to get a sense for how many systems will have problems
> with this test, in the case that rpmbuild is not present.
>
> And I'm wondering if there's another way to resolve whatever problem
> the test rpm package has (like maybe using --relocate option with 'rpm'
> if there is some directory or path problem).
>
> Thanks,
> (Not applied yet)
>  -- Tim
>
>
> >  }
> >
> >  function test_deploy {
> >      put $TEST_HOME/rpm_test.sh $BOARD_TESTDIR/fuego.$TESTDIR/
> > -    put $TEST_HOME/test-manual-1.2.3.noarch.rpm
> > $BOARD_TESTDIR/fuego.$TESTDIR/
> > +    put -r $TEST_HOME/data $BOARD_TESTDIR/fuego.$TESTDIR/
> >      put -r $TEST_HOME/tests $BOARD_TESTDIR/fuego.$TESTDIR/
> >  }
> >
> >  function test_run {
> > +    if [ $NEED_BUILD_RPM ];
> > +    then
> > +        cmd "cd $BOARD_TESTDIR/fuego.$TESTDIR; \
> > +            mv /etc/rpm/platform /etc/rpm/platform.old; \
> > +            rpmbuild --define \"_topdir \${dir:-
> > $BOARD_TESTDIR/fuego.$TESTDIR/rpm}\" --rebuild data/test-manual-
> > 1.2.3.src.rpm; \
> > +            mv /etc/rpm/platform.old /etc/rpm/platform; \
> > +            mv rpm/RPMS/noarch/test-manual-1.2.3-20181012.noarch.rpm
> > data/test-manual-1.2.3.noarch.rpm"
> > +    fi
> >      report "cd $BOARD_TESTDIR/fuego.$TESTDIR; \
> > -        sh -v rpm_test.sh"
> > +        sh -v rpm_test.sh; \
> > +        rpm -e test-manual"
> >  }
> >
> >  function test_processing {
> > diff --git a/engine/tests/Functional.rpm/tests/rpm_02.sh
> > b/engine/tests/Functional.rpm/tests/rpm_02.sh
> > index 9b0f411..4a7de3b 100644
> > --- a/engine/tests/Functional.rpm/tests/rpm_02.sh
> > +++ b/engine/tests/Functional.rpm/tests/rpm_02.sh
> > @@ -11,7 +11,7 @@ then
> >      rpm -e $test_manual
> >  fi
> >
> > -rpm -ivh ${test_manual}.rpm --nodeps
> > +rpm -ivh data/${test_manual}.rpm --nodeps
> >
> >  if rpm -ql $test_manual | grep '.*/home/test/rpm-test/text1.txt.*'
> >  then
> > --
> > 1.8.3.1
> >
> >
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
> ------------------------------
>
> Message: 2
> Date: Wed, 6 Feb 2019 06:21:15 +0000
> From: <Tim.Bird@sony.com>
> To: <dhinakar.k@samsung.com>
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] Executing test on slave machine connected to
>         target board via serial port
> Message-ID:
>         <ECADFF3FD767C149AD96A924E7EA6EAF8E0E9606@USCULXMSG01.am.sony.com>
> Content-Type: text/plain; charset="utf-8"
>
>
>
> > -----Original Message-----
> > From: Dhinakar Kalyanasundaram
> > Hi Tim,
> >
> > Thanks for the detailed response.
> >
> > Sorry for the confusion.
> >
> > I have provided below a better explanation of my proposal/requirement.
> >
> > Proposal/Requirement:
> >
> > I have setup Fuego on a powerful DELL server to cater to multiple
> > projects/teams.
> >
> > The server is kept in a lab at a remote location, hence target boards
> won't be
> > directly connected to it.
> >
> > In case of projects where 'ethernet' IP is supported, I can connect the
> target
> > board to corporate LAN, create a board file ~fuego/fuego-ro/boards and
> run
> > tests on it remotely.
> >
> > So multiple users can connect their target board similar way, execute
> their
> > tests and all results will be available on the Fuego server (Dell
> server).
> >
> > The issue is in case of projects where 'ethernet' IP is not supported,
> say only
> > UART or Serial connectivity is supported.
> >
> >
> >
> > As mentioned in this link you sent
> https://www.quora.com/What-is-the-use-
> > of-master-and-slave-in-Jenkins , Fuego running on Dell Server will be the
> > 'Jenkins Master'.
> >
> > I thought of setting up Jenkins slave on the user pc (Linux/windows) and
> > connecting the target boards to it and trigger tests from Fuego
> dashboard.
> >
> > There can be multiple Jenkins slaves one for each board (connected via
> micro
> > usb or uart or serial port).
> >
> >
> >
> > I have done similar Jenkins Master & Slave setups successfully before but
> > without Fuego (I mean just with Jenkins).
> >
> > In case of  Fuego, since Jenkins master runs inside the container I was
> > checking out what will be best way to establish connection between,
> >
> > Jenkins Master (running inside fuego container) and Jenkins Slave(s)
> which
> > could be just any other pc (Linux/windows) on the network attached to
> > target board.
> >
> > Jenkins Master need to trigger the test on the Jenkins Slave to which
> target
> > board is connected via serial/uart port.
> >
> > How to create a board file in such cases? and I will also have to
> install/setup
> > 'Jenkins Slave' on those PCs where target board is attached.
> >
> > It is here I want your advice/suggestions.
> >
> > I hope the above explanation clarifies my requirement, else please let me
> > know.
>
>
> OK - this clarifies things a lot.  Thanks.
>
> There are, IMO, two possible approaches here.  One would be to put a Fuego
> docker
> container on the slave, and use a headless jenkins slave to execute the
> job on the board.
> This would include building the test, deploying it, gathering the logs,
> analyzing them,
> and populating the run.json results for the test.    This requires that
> the slave machine
> have toolchains installed in the docker container there, and that you work
> out some
> mechanism to transfer the run data back to the master machine.  This
> sounds like a lot
> of work.
>
> The second approach would be to just put together a new Fuego 'transport',
> that talks
> from the master to the slave, that the slave can then relay to the board
> using the serial
> port.  Fuego really only requires 4 operations to communicate with the
> board:
>  put, cmd, get, reboot
>
> So what you could do would be to make board definition on the master
> machine, and
> then define the transport routines for the boards connected to the slave,
> that utilize
> some ssh pass-through to the serial port.  I presume that the master has
> ssh access to
> the slave machine.  If not, then it's still possible.  We toyed with the
> idea of creating a
> web-based transport for Fuego, where the host would deposit materials on a
> web site,
> and the target machine would retrieve them (operations and data) and
> perform them.
> Originally I envisioned the target board itself doing this (for boards
> that have network access
> but not ssh access (only http or something)).  However, you could use the
> same approach
> with a slave machine to which boards are connected via serial port.
>
> This does not require any Jenkins code on the slave machine, and indeed it
> could be
> quite a lightweight machine (like a raspberry pi or something), since it's
> basically just
> serving as a network to serial gateway for a few operations.
>
> Let me know what you think of this, and maybe I can cobble together a
> proof of concept
> for you to try out.
>
> The big question is: does the master have ssh access to the slave
> machine?  If so, then
> I think that simplifies things considerably.
>  -- Tim
>
> > --------- Original Message ---------
> >
> > Sender : Tim.Bird@sony.com <Tim.Bird@sony.com>
> >
> > Date : 2019-02-05 01:05 (GMT+5:30)
> >
> > Title : RE: [Fuego] Executing test on slave machine connected to target
> board
> > via serial port
> >
> >
> >
> > > -----Original Message-----
> > > From: Dhinakar Kalyanasundaram
> > >
> > > Dear Fuego Users,
> > >
> > > I am checking out the feasibility of executing tests on a slave machine
> > which
> > > is connected to target board via serial port.
> > >
> > > The idea is to setup Fuego on a power server and creates nodes in it.
> > I don't know what this means.
> >
> > >
> > > Each node will resemble a host pc on the network which is connected to
> > the
> > > target board via serial port.
> > >
> > > I am not sure how to setup master/slave in such a scenario.
> > >
> > > The Jenkins running inside Fuego container will be the master and
> slaves
> > can
> > > be PCs (Linux/windows) with Jenkins agent installed?
> > I don't know what you mean by  'Jenkins agent'.
> > Are you talking about /fuego-core/engine/scripts/slave.jar or
> > /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar, or something else?
> >
> > Are you trying to create Jenkins slaves on other nodes?
> >
> > Specifically, are you talking about Jenkins masters and slaves
> > as described here:?
> > https://www.quora.com/What-is-the-use-of-master-and-slave-in-Jenkins
> >
> > > How can the Jenkins running inside container connect to Jenkins slave
> on
> > > other PC? Any plugins required like 'docker plugin' to achieve this.
> >
> > >
> > > Please let me know if any of you had a similar requirement and
> > implemented
> > > it.
> >
> > I'm not following what you are trying to do here, or what problem you
> > are trying to solve.
> >
> > Fuego is fairly heavily geared towards an environment where a single host
> > is connected to (or has network access to) one or more target boards.
> > It is possible, though, to split up test operations between different
> machines
> > (and that will likely be key to integrating better with LAVA in the near
> future).
> >
> > Here are some letters to indicate where operations are performed during
> > test operation:
> > Jh = performed by Jenkins in docker container on Fuego host
> > Fh = performed by Fuego core code on the Fuego host
> > Th = performed by Test code on Fuego host
> > Tb = performed by test code on target board
> >
> > Below is a list of test operations, and where they take place now.
> > Can you indicate which of these you would like to move to a separate
> > machine,
> > and why?  I think this will help me understand whether (and how) Fuego
> > and Jenkins can support your proposed architecture.
> >
> >  - trigger (detection that a test should be run) - Jh
> >  - job scheduling (deciding which node should execute a job, and waiting
> until
> > the node is available) - Jh
> >  - build software under test (left as exercise for the user)
> >  - install software under test to the target board (left as an exercise
> for the
> > user)
> >  - build test software - Fh/Th
> >  - deploy test software to target board - Fh/Th/Tb
> >  - execute test software - Tb
> >  - collect logs and results - Fh/Th/Tb
> >  - parse test log - Fh/Th
> >  - format results - Fh
> >  - display results - Jh
> >
> > Depending on the test, it might also do:
> >  - reboot the target board - could be Fh or Th or Tb.
> >
> > I believe you are proposing that some of these operations should be moved
> > to other machines,
> > or have their execution or control split between a primary Fuego host and
> > some secondary
> > machines.  Can you describe in greater detail what you want to do?
> >
> > Thanks,
> >  -- Tim
> >
> >
> >
> >
> > <http://ext.samsung.net/mail/ext/v1/external/status/update?userid=dhina
> > kar.k&do=bWFpbElEPTIwMTkwMjA1MTM0MzAxZXBjbXM1cDVkYmQ2Yzc3M
> > TM1OTY2N2RjYjk3YzY0ODI0YjdiODRlMiZyZWNpcGllbnRBZGRyZXNzPVRpbS5C
> > aXJkQHNvbnkuY29t>
>
> ------------------------------
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
>
> End of Fuego Digest, Vol 31, Issue 8
> ************************************
>

[-- Attachment #2: Type: text/html, Size: 25065 bytes --]

^ permalink raw reply	[flat|nested] 45+ messages in thread