[LTP] LTP release

[Bird, Tim <Tim.Bird@sony.com>]

> -----Original Message-----
> From: Cyril Hrubis <chrubis@suse.cz>
> 
> Hi!
> > > What else should go in?
> > How about removing runtest/quickhit?
> > https://patchwork.ozlabs.org/project/ltp/patch/20200831094617.7764-1-chrubis@suse.cz/
> 
> I guess that we can do that if we replace it with something Tim can use
> for infrastructure tests.
> 
> Tim what are your expectation from such runtest file? How many tests
> should be in?

I don't think that it should be very many.  Otherwise, people will get
the idea that it is useful for actual device testing. ;-)

The current 'quickhit' runtest file has 107 tests.  With the exception of
qmm01 (which calls mmap001 with '-n 1') and a bunch of tests that do sub-tests
of symlink01 (12 of them), the remaining test definitions
just consist of the test name and the test executable (with the same names).
I only see binary programs - no tests using shell scripts.  So there's really not much variety
here.

Apparently a pipe is allowed in the command invocation line for a test, but there's only
one example of this in all of the runtests, in syscalls:
splice02 seq 1 2000 | splice02

Really, as an infrastructure test, I only need to run a few testcases to validate that Fuego's
plumbing around runltp (and ltp-pan) works properly.  And it would be nice if
the run was very short, so I could do the check quickly.

Do you want me to create a runtest for a framework/LTP integration test, by picking a
few different "representative" tests, as a replacement for quickhit?

IMHO, the selected tests should behave the same on all possible systems, to avoid
getting results that are inconsistent due to the system under test, rather than a
problem with the integration between the framework and LTP.

Should I add some outlier cases:
 - something that times out
 - something that always fails
 - tests that return TBROK, TCONF, TWARN, etc.
 - something where the command doesn't exist

This would be helpful for checking that my parsing for different results works.

Do you have a preferred name for the runtest file?  My proposal, just off the top
of my head, is: "ltp-selftest-quick", but I'm open to other suggestions.

I'm also open to suggestions for possible tests.  I'd like a shell script command
to add to the list of binary programs.  Here is what I've chosen so far:
access01 access01
chdir01 chdir01
fork01 fork01
time01 time01
wait02 wait02
write01 write01
symlink01 symlink01
stat04 symlink01 -T stat04
utime01A symlink01 -T utime01
rename01A symlink01 -T rename01
splice02 seq 1 20 | splice02

This takes about 5 seconds on one of my test machines.

 -- Tim

P.S. Maybe, if you're moving away from runltp and ltp-pan, it's a little late to be
adding some selftests to make sure they work correctly.  But Fuego is using them.
I don't know what other frameworks use when they invoke LTP to perform
tests.

P.P.S How come some tests produce TPASS and some produce just PASS?