From mboxrd@z Thu Jan 1 00:00:00 1970 From: Simon Glass Date: Thu, 14 Jan 2016 16:12:24 -0700 Subject: [U-Boot] [PATCH V2 1/7] test/py: Implement pytest infrastructure In-Reply-To: <56724BFB.5070502@denx.de> References: <1449094708-14784-1-git-send-email-swarren@wwwdotorg.org> <56685784.9040907@wwwdotorg.org> <56717F09.8020407@monstr.eu> <567190E2.5010308@wwwdotorg.org> <56724BFB.5070502@denx.de> Message-ID: List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: u-boot@lists.denx.de Hi Heiko, On 16 December 2015 at 22:45, Heiko Schocher wrote: > Hello Stephen, > > > Am 16.12.2015 um 17:27 schrieb Stephen Warren: >> >> On 12/16/2015 08:11 AM, Michal Simek wrote: >>> >>> On 9.12.2015 17:32, Stephen Warren wrote: >>>> >>>> On 12/02/2015 03:18 PM, Stephen Warren wrote: >>>>> >>>>> This tool aims to test U-Boot by executing U-Boot shell commands using >>>>> the >>>>> console interface. A single top-level script exists to execute or >>>>> attach >>>>> to the U-Boot console, run the entire script of tests against it, and >>>>> summarize the results. Advantages of this approach are: >>>>> >>>>> - Testing is performed in the same way a user or script would interact >>>>> with U-Boot; there can be no disconnect. >>>>> - There is no need to write or embed test-related code into U-Boot >>>>> itself. >>>>> It is asserted that writing test-related code in Python is simpler >>>>> and >>>>> more flexible that writing it all in C. >>>>> - It is reasonably simple to interact with U-Boot in this way. >>>>> >>>>> A few simple tests are provided as examples. Soon, we should convert as >>>>> many as possible of the other tests in test/* and test/cmd_ut.c too. >>>>> >>>>> In the future, I hope to publish (out-of-tree) the hook scripts, relay >>>>> control utilities, and udev rules I will use for my own HW setup. >>>> >>>> >>>> I finally got permission to publish these. Examples are at: >>>> https://github.com/swarren/uboot-test-hooks >>> >>> >>> Interesting. What's the normal setup which you have for the board? >>> I see from your description that you use numato usb relay - I expect one >>> with more channels for reset. >>> Then you are able to control boot mode. Is it also via the same relay? >>> How do you power up the board? >> >> >> In my current setup I leave the board on all the time (or rather, manually >> turn on the power when >> I'm about to run the tests). Automating control of the power source is a >> step I'll take later. > > > Maybe you give tbot (I mentioned it in this thread) a chance? > > There, this things are automated, also you can do linux (and other console > based) > tests ... Currently added a testcase [1] which adds patches from patchwork, > which are in a users ToDo list, to a git tree! In this case it is a u-boot > git > tree ... checks them with checkpatch, compiles it, and tries the new image > on > the board and calls testcases ... fully automated in a now weekly build [2] > ... > (only weekly, but thats a setup parameter in buildbot, as I have tbot and > buildbot > running on a raspberry pi) > and don;t forget, I have the board not where I run tbot, the boards are > ~1000km > from me .. So, it is possible to add a U-Boot Testsetup on a server, > and test boards all over the world ... This sounds like a great development. How can we get this so that it can be used by U-Boot people? Do you think you could add a README to the mainline, or some scripts to aid setting it up? I would be interested in setting up a few boards that run continuous testing, and I suspect others would also if it were easier. > >> For Tegra, there are two important signals: reset and "force recovery". >> Each of these has a separate >> relay, so the system currently uses 2 relays per target board. The numato >> relay board I own has 8 >> relays, although there are a number of different models. >> >> On Tegra, when reset is pulsed: >> >> - If force-recovery is connected, the SoC enters USB recovery mode. In >> this state, SW can be >> downloaded over USB into RAM and executed. >> >> - If force-recovery is not connected, the SoC boots normally, from SW >> stored in flash (eMMC, SPI, ...) >> >> The example scripts always use recovery mode to download U-Boot into RAM >> rather than writing it to >> flash first and then resetting. This saves wear cycles on the flash, but >> does mean the download >> happens in the "reset" rather than "flash" script, which may make the >> examples a bit different than >> for some other SoCs. >> >> Finally, the example scripts support two boards; my home/laptop dev setup >> that uses a Numato relay >> board to control the signals to the board I use there, and my work desktop >> dev setup that uses our >> "PM342" debug board to controll the signals. The latter works logically >> the same as the numato relay >> board, except it contains electronic switches driven by an FTDI chip. > > > I seperated such functions into a seperate python class, so such setup > specific things should be easy to add for other (in tbot called lab) > setups ... > > Currently I have only a bdi testcase, which flashes a new image into > the board, when it is broken ... but such relay things can be added > of course ... > > bye, > Heiko > [1] get patchlist from patchwork > > https://github.com/hsdenx/tbot/commit/0bcaf4877e7aad4df2039913dcb6e85303a0b15f > apply them to git tree: > > https://github.com/hsdenx/tbot/commit/b7e2de3731252b518754cc3f71dc782559b0cad6 > and use this on the smartweb board: > > https://github.com/hsdenx/tbot/commit/56a1ac18e5730ae9ffa7686acfc52877272be91d > > [2] weekly started testcase for the smartweb board: > http://xeidos.ddns.net/buildbot/builders/smartweb_dfu > log with the testcases from [1] > > http://xeidos.ddns.net/buildbot/builders/smartweb_dfu/builds/32/steps/shell/logs/tbotlog > > -- > DENX Software Engineering GmbH, Managing Director: Wolfgang Denk > HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany > _______________________________________________ > U-Boot mailing list > U-Boot at lists.denx.de > http://lists.denx.de/mailman/listinfo/u-boot Regards, Simon