Re: [LSF/MM TOPIC] improving storage testing

[Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>]

Hi, Bart

In this way, we may end up modifying probably most of the common tools,
which in long run can create a bunch of the code for the tests. If everyone
(test contributors and tools maintainers) agrees to have such a "test" mode
to all the tools, we can go for this approach.

-Chaitanya

From: linux-block-owner@vger.kernel.org <linux-block-owner@vger.kernel.org> on behalf of Bart Van Assche <bvanassche@acm.org>
Sent: Thursday, February 14, 2019 3:26 PM
To: Johannes Thumshirn; Theodore Y. Ts'o
Cc: lsf-pc@lists.linux-foundation.org; linux-block@vger.kernel.org; linux-fsdevel@vger.kernel.org
Subject: Re: [LSF/MM TOPIC] improving storage testing
  
 
On Thu, 2019-02-14 at 11:55 +0100, Johannes Thumshirn wrote:
> On Wed, Feb 13, 2019 at 01:07:54PM -0500, Theodore Y. Ts'o wrote:
> > Also, there are expectations about minimum versions of bash that can
> > be supported; but there aren't necessarily for other components such
> > as nvme-cli, and I suspect that it is due to the use of a overly new
> > version of nvme-cli from its git tree.  Is that supposed to work, or
> > should I constrain myself to whatever version is being shipped in
> > Fedora or some other reference distribution?  More generally, what is
> > the overall expectations that should be expected?  xfstests has some
> > extremely expansive set of sed scripts to normalize shell script
> > output to make xfstests extremely portable; will patches along similar
> > lines something that we should be doing for blktests?
> 
> I think this is the root cause of the problems you've sent out mails for this
> week. A lot of blktests test need filtering. See [1] as an example.
> 
> [1] https://github.com/osandov/blktests/pull/34

 https://avatars0.githubusercontent.com/u/5319409?s=400&v=4 

Add filter function for nvme discover by frankenmichl · Pull Request #34 · osandov/blktests
github.com
Several NVMe tests (002, 016, 017) used a pipe to a sed call filtering the output. This call is moved to a new filter function nvme/rc and the calls to sed are replaced by this function. Additional...


Hi Johannes,

Output of tools like nvme-cli is not an ABI although an ABI is what is
required to make blktests work reliably. One possible approach is to modify
nvme-cli such that it has two output formats: one output format that is
intended for humans and another that is easy to parse by software. I think
we should consider that approach and compare it to using sed scripts.

Bart.