[SPDK] New Test Utility for SPDK per check-in and nightly tests

[SPDK] New Test Utility for SPDK per check-in and nightly tests

[Luse, Paul E]

[-- Attachment #1: Type: text/plain, Size: 2414 bytes --]

Hi All,

For those adding per check-in or nightly tests this is good stuff to read/understand.  A change was just merged that adds the following capability:

New tests that run some sort of CLI tool or existing tests that are running a CLI tool (most everything we do) can all benefit from this as most of the time our test scripts simply count on the CLI "not exploding" or returning with an exist code of 0/1 to determine if the test passed or not.  With this new utility we can capture the expected output from a CLI, fix it up so that it will work on any platform, and then confirm any level of detail that we want after the tool runs.  Please read through this and yell if there are any questions either here or on IRC...

Thanks!
Paul

From https://review.gerrithub.io/#/c/397027/ : (check out the match source for more info, you can find it in /test/apps/match)


The 'match' util was borrowed from the PMDK folks after first

adding the "ignore" feature upstream to make it easier to use

in SPDK.  It works like this:



When the developer checks in a test they create and check in

the output of the test with two different file extensions:



.ignore: should include a string per line for output lines

that we want to totally ignore typically because they're

platform specific so the output could be different from

machine to machine.  In this case I'm ignoring all output

lines with 'DPDK' or 'EAL' or '...' in them. The first

few are obvious, the last is because the test tool will

print a varying number of these as progress indicators.



.match: this is a copy of the output that the developer

'fixes' up by replacing platform specific output strings

with replicable tokens as described in the 'match' help.

This is where you'd want to match an entire line minus

something like a CPU count or free block count or

something. The 'ignore' feature was added simply so we

wouldn't have to edit every single line of an output

file that had DPDK or EAL in it.



Then you modify the test script to save the output and

simply run the match util providing the name of the

match file and if it fails to match the actual output

with the saved output that's been token'ized the script

will error.



The obvious advantage here is that now we can confirm all

of the output from a test executable is as we expect.



[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 5536 bytes --]