From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bird, Tim Date: Mon, 14 Sep 2020 22:30:35 +0000 Subject: [LTP] LTP release In-Reply-To: <20200914111543.GE5524@yuki.lan> References: <20200908073138.GB2475@yuki.lan> <20200910084554.GA21741@dell5510> <20200914111543.GE5524@yuki.lan> Message-ID: List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: ltp@lists.linux.it > -----Original Message----- > From: Cyril Hrubis > > Hi! > > > What else should go in? > > How about removing runtest/quickhit? > > https://patchwork.ozlabs.org/project/ltp/patch/20200831094617.7764-1-chrubis@suse.cz/ > > I guess that we can do that if we replace it with something Tim can use > for infrastructure tests. > > Tim what are your expectation from such runtest file? How many tests > should be in? I don't think that it should be very many. Otherwise, people will get the idea that it is useful for actual device testing. ;-) The current 'quickhit' runtest file has 107 tests. With the exception of qmm01 (which calls mmap001 with '-n 1') and a bunch of tests that do sub-tests of symlink01 (12 of them), the remaining test definitions just consist of the test name and the test executable (with the same names). I only see binary programs - no tests using shell scripts. So there's really not much variety here. Apparently a pipe is allowed in the command invocation line for a test, but there's only one example of this in all of the runtests, in syscalls: splice02 seq 1 2000 | splice02 Really, as an infrastructure test, I only need to run a few testcases to validate that Fuego's plumbing around runltp (and ltp-pan) works properly. And it would be nice if the run was very short, so I could do the check quickly. Do you want me to create a runtest for a framework/LTP integration test, by picking a few different "representative" tests, as a replacement for quickhit? IMHO, the selected tests should behave the same on all possible systems, to avoid getting results that are inconsistent due to the system under test, rather than a problem with the integration between the framework and LTP. Should I add some outlier cases: - something that times out - something that always fails - tests that return TBROK, TCONF, TWARN, etc. - something where the command doesn't exist This would be helpful for checking that my parsing for different results works. Do you have a preferred name for the runtest file? My proposal, just off the top of my head, is: "ltp-selftest-quick", but I'm open to other suggestions. I'm also open to suggestions for possible tests. I'd like a shell script command to add to the list of binary programs. Here is what I've chosen so far: access01 access01 chdir01 chdir01 fork01 fork01 time01 time01 wait02 wait02 write01 write01 symlink01 symlink01 stat04 symlink01 -T stat04 utime01A symlink01 -T utime01 rename01A symlink01 -T rename01 splice02 seq 1 20 | splice02 This takes about 5 seconds on one of my test machines. -- Tim P.S. Maybe, if you're moving away from runltp and ltp-pan, it's a little late to be adding some selftests to make sure they work correctly. But Fuego is using them. I don't know what other frameworks use when they invoke LTP to perform tests. P.P.S How come some tests produce TPASS and some produce just PASS?