From mboxrd@z Thu Jan 1 00:00:00 1970 From: Simon Glass Date: Thu, 26 Nov 2015 18:52:06 -0800 Subject: [U-Boot] [PATCH] Implement pytest-based test infrastructure In-Reply-To: <5654D67B.2090307@wwwdotorg.org> References: <1447570381-1361-1-git-send-email-swarren@wwwdotorg.org> <564E0030.9090808@wwwdotorg.org> <564E1E6B.7010708@wwwdotorg.org> <5651FBB0.9010202@wwwdotorg.org> <5653EB13.60506@wwwdotorg.org> <5654D67B.2090307@wwwdotorg.org> Message-ID: List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: u-boot@lists.denx.de Hi Stephen, On 24 November 2015 at 13:28, Stephen Warren wrote: > On 11/24/2015 12:04 PM, Simon Glass wrote: >> >> Hi Stephen, >> >> On 23 November 2015 at 21:44, Stephen Warren >> wrote: >>> >>> On 11/23/2015 06:45 PM, Simon Glass wrote: >>>> >>>> On 22 November 2015 at 10:30, Stephen Warren >>>> wrote: >>>>> >>>>> On 11/21/2015 09:49 AM, Simon Glass wrote: > > >>>>>> OK I got it working thank you. It is horribly slow though - do you >>>>>> know what is holding it up? For me to takes 12 seconds to run the >>>>>> (very basic) tests. > > .. > >>> I put a bit of time measurement into run_command() and found that on my >>> system at work, for p.send("the shell command to execute") was actually >>> (marginally) slower on sandbox than on real HW, despite real HW being a >>> 115200 baud serial port, and the code splitting the shell commands into >>> chunks that are sent and waited for synchronously to avoid overflowing >>> UART FIFOs. I'm not sure why this is. Looking at U-Boot's console, it >>> seems to be non-blocking, so I don't think termios VMIN/VTIME come into >>> play (setting them to 0 made no difference), and the two raw modes took >>> the same time. I meant to look into pexpect's termios settings to see if >>> there was anything to tweak there, but forgot today. >>> >>> I did do one experiment to compare expect (the Tcl version) and pexpect. >>> If I do roughly the following in both: >>> >>> spawn u-boot (sandbox) >>> wait for prompt >>> 100 times: >>> send "echo $foo\n" >>> wait for "echo $foo" >>> wait for shell prompt >>> send "reset" >>> wait for "reset" >>> send "\n" >>> >>> ... then Tcl is about 3x faster on my system (IIRC 0.5 vs. 1.5s). If I >>> remove all the "wait"s, then IIRC Tcl was about 15x faster or more. >>> That's a pity. Still, I'm sure as heck not going to rewrite all this in >>> Tcl:-( I wonder if something similar to pexpect but more targetted at >>> simple "interactive shell" cases would remove any of that overhead. >> >> >> It is possible that we should use sandbox in 'cooked' mode so that >> lines an entered synchronously. The -t option might help here, or we >> may need something else. > > > I don't think cooked mode will work, since I believe cooked is > line-buffered, yet when U-Boot emits the shell prompt there's no \n printed > afterwards. Do you mean we need fflush() after writing the prompt? If so, that should be easy to arrange. We have a similar problem with the LCD, and added lcd_sync(). > > FWIW, I hacked out pexpect and replaced it with some custom code. That > reduced by sandbox execution time from ~5.1s to ~2.3s. Execution time > against real HW didn't seem to be affected at all. Some features like > timeouts and complete error handling are still missing, but I don't think > that would affect the execution time. See my github tree for the WIP patch. Interesting, that's a big improvement. I wonder if we should look at building U-Boot with SWIG to remove all these overheads? Then the U-Boot command line (and any other feature we want) could become a Python class. Of course that would only work for sandbox. Regards, Simon