From mboxrd@z Thu Jan 1 00:00:00 1970 From: Rob Clark Date: Tue, 5 Sep 2017 19:48:32 -0400 Subject: [U-Boot] [PATCH 00/23] efi_loader implement missing functions In-Reply-To: References: <20170826225110.7381-1-xypron.glpk@gmx.de> <8dd266b9-9595-e8b7-5b92-5b37b019de49@gmx.de> <0d42bc48-9e8f-ab5e-8e15-c6fbc7829f00@suse.de> <20170829125755.4unqrcv6zbgc4jvz@bivouac.eciton.net> <20170901144549.GY17058@bill-the-cat> Message-ID: List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: u-boot@lists.denx.de On Tue, Sep 5, 2017 at 4:55 AM, Simon Glass wrote: > Hi, > > On 1 September 2017 at 22:45, Tom Rini wrote: >> On Wed, Aug 30, 2017 at 04:16:34AM +0800, Simon Glass wrote: >>> Hi, >>> >>> On 29 August 2017 at 22:16, Rob Clark wrote: >>> > On Tue, Aug 29, 2017 at 8:57 AM, Leif Lindholm wrote: >>> >> On Tue, Aug 29, 2017 at 02:26:48PM +0200, Alexander Graf wrote: >>> >>> > > > I would add command >>> >>> > > > bootefi selftest.efi >>> >>> > > > to run the tests and provide the python wrapper code to add it to the >>> >>> > > > test suite. >>> >>> > > >>> >>> > > I think that's a great idea, yes. >>> >>> > I wonder how far we are from running UEFI tests (either the official >>> >>> > ones, or I seem to remember hearing about some other test suite which >>> >>> > didn't require UEFI shell)? >>> >>> >>> >>> Let's ask Leif, Ard and Dong. >>> >>> >>> >>> The official test suite definitely needs the UEFI Shell. Is the suite >>> >>> publicly available by now? >>> >> >>> >> In binary form, you can access it already from the links on >>> >> http://uefi.org/testtools >>> >> >>> >> Yes, 2.5 is latest release. No this is not a restriction ... the SCT >>> >> releases have been lagging the specification releases a fair bit. >>> >> >>> >> The 2.5a package contains AARCH64, IA32 and X64 support (not ARM). >>> >> Not that it couldn't contain ARM, it just hasn't been packaged. >>> >> >>> >>> > That seems more useful long term than re-inventing comprehensive UEFI >>> >>> > test suite. (Also, beyond just running shim/fallback/grub, I don't >>> >>> > really have time to invent new tests for the stack of efi_loader >>> >>> > patches I have.) >>> >>> >>> >>> Yes and no - it depends on the availability of the official suite :/. >>> >> >>> >> UEFI SCT is not yet open source/free software. Obviously, this is >>> >> something Linaro has been lobbying for since day one of our >>> >> involvement. There used to be little understanding for this, but that >>> >> attitude has shifted substantially. >>> > >>> > So, if/until UEFI SCT is not an option, what about: >>> > >>> > https://01.org/linux-uefi-validation >>> > >>> > (thx to pjones for pointing that out to me) >>> >>> Well in any case I'm not looking for a large functional test suite at >>> this stage. It certainly could be useful, but not as a replacement for >>> unit tests. The latter is for fast verification (so that everyone can >>> run it as part of 'make tests') and easy identification of the >>> location of bugs. >> >> I want to chime in here. Unless we're talking hours of time, if "make >> tests" takes 5 minutes to run, but that includes doing some sort of full >> validation suite to the relevant EFI code, that seems like a win to me. >> And if someone else is responsible for the contents of the tests and we >> just have to confirm our expected results, that's an even bigger win. > > Yes it certainly is a win. But I'm already upset with how long the > tests take to run so I would not be keen on anything that increases it > much. How long does this test suite take to run normally? > I'm not sure how long SCT would take to run until we get it working (I have a growing stack of patches on my wip-enough-uefi-for-shell branch, and have Shell.efi running but still debugging sct.efi).. maybe someone who has run SCT on a real UEFI implementation knows.. But that said, there seems to be some potential to run a subset of the tests. Either way, it seems like most of the time that travis runs take is building a million variants of u-boot. I'm not sure the time spent running tests is really the issue. BR, -R