linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Darrick J. Wong" <djwong@kernel.org>
To: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Cc: Theodore Ts'o <tytso@mit.edu>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Matthew Wilcox <willy@infradead.org>,
	"Leonidas P. Papadakos" <papadakospan@gmail.com>,
	"zajec5@gmail.com" <zajec5@gmail.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Hans de Goede <hdegoede@redhat.com>,
	linux-fsdevel <linux-fsdevel@vger.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Al Viro <viro@zeniv.linux.org.uk>
Subject: Re: [GIT PULL] vboxsf fixes for 5.14-1
Date: Tue, 10 Aug 2021 00:02:55 -0700	[thread overview]
Message-ID: <20210810070255.GJ3601405@magnolia> (raw)
In-Reply-To: <0e175373cef24e2abe76e203bb36d260@paragon-software.com>

On Thu, Aug 05, 2021 at 03:48:36PM +0000, Konstantin Komarov wrote:
> > From: Darrick J. Wong <djwong@kernel.org>
> > Sent: Wednesday, August 4, 2021 4:04 AM
> > To: Theodore Ts'o <tytso@mit.edu>
> > Cc: Linus Torvalds <torvalds@linux-foundation.org>; Matthew Wilcox <willy@infradead.org>; Leonidas P. Papadakos
> > <papadakospan@gmail.com>; Konstantin Komarov <almaz.alexandrovich@paragon-software.com>; zajec5@gmail.com; Greg Kroah-
> > Hartman <gregkh@linuxfoundation.org>; Hans de Goede <hdegoede@redhat.com>; linux-fsdevel <linux-fsdevel@vger.kernel.org>;
> > Linux Kernel Mailing List <linux-kernel@vger.kernel.org>; Al Viro <viro@zeniv.linux.org.uk>
> > Subject: Re: [GIT PULL] vboxsf fixes for 5.14-1
> > 
> > On Tue, Aug 03, 2021 at 08:49:28PM -0400, Theodore Ts'o wrote:
> > > On Tue, Aug 03, 2021 at 05:10:22PM -0700, Linus Torvalds wrote:
> > > > The user-space FUSE thing does indeed work reasonably well.
> > > >
> > > > It performs horribly badly if you care about things like that, though.
> > > >
> > > > In fact, your own numbers kind of show that:
> > > >
> > > >   ntfs/default: 670 tests, 55 failures, 211 skipped, 34783 seconds
> > > >   ntfs3/default: 664 tests, 67 failures, 206 skipped, 8106 seconds
> > > >
> > > > and that's kind of the point of ntfs3.
> > >
> > > Sure, although if you run fstress in parallel ntfs3 will lock up, the
> > > system hard, and it has at least one lockdep deadlock complaints.
> > > It's not up to me, but personally, I'd feel better if *someone* at
> > > Paragon Software responded to Darrrick and my queries about their
> > > quality assurance, and/or made commitments that they would at least
> > > *try* to fix the problems that about 5 minutes of testing using
> > > fstests turned up trivially.
> > 
> > <cough> Yes, my aim was to gauge their interest in actively QAing the
> > driver's current problems so that it doesn't become one of the shabby
> > Linux filesystem drivers, like <cough>ntfs.
> > 
> > Note I didn't even ask for a particular percentage of passing tests,
> > because I already know that non-Unix filesystems fail the tests that
> > look for the more Unix-specific behaviors.
> > 
> > I really only wanted them to tell /us/ what the baseline is.  IMHO the
> > silence from them is a lot more telling.  Both generic/013 and
> > generic/475 are basic "try to create files and read and write data to
> > them" exercisers; failing those is a red flag.
> > 
> 
> Hi Darrick and Theodore! First of all, apologies for the silence on your questions.
> Let me please clarify and summarize the QA topic for you.
> 
> The main thing to outline is that: we have the number of autotests executed
> for ntfs3 code. More specifically, we are using TeamCity as our CI tool, which
> is handling autotests. Those are being executed against each commit to the
> ntfs3 codebase.
> 
> Autotests are divided into the "promotion" levels, which are quite standard:
> L0, L1, L2. Those levels have the division from the shortest "smoke" (L0)
> to the longest set (L2). This we need to cover the ntfs3 functionality with
> tests under given amount of time (feedback loop for L0 is minutes, while for
> L2 is up to 24hrs).

Sounds comparable to my setup, which has these tiers:

fstests -g quick (~45 minutes) on fast ssds
fstests -g all (~3 hours) on fast ssds
fstests -g all (~12 hours) on slow(er) cheap(er) cloud storage
fstests -g long_soak (~7 days) on aging ssds

(There's also the fifth tier which is spawning dozens of VMs to fuzz
test, but I don't have enough spare time to run that and triage the
results on a regular basis.)

> As for suites we are using - it is the mix of open/well known suites:
> - xfstests, ltp, pjd suite, fsx, dirstress, fstorture - those are of known utilites/suites
> And number of internal autotests which were developed for covering various parts of
> fs specs, regression autotests which are introduced to the infrastructure after bugfixes
> and autotests written to test the driver operation on various data sets.
> 
> This approach is settled in Paragon for years, and ntfs3, from the first line of code written,
> is being developed this way. You may refer the artifacts linked below, where the progress/coverage
> during the last year is spoken by autotest results:
> 
> the 27th patch-series code (July'2021): 
> https://dl.paragon-software.com/ntfs3/p27_tests.tar
> 25th (March'2021):
> https://dl.paragon-software.com/ntfs3/p25_tests.tar
> 2nd (August, 2020):
> https://dl.paragon-software.com/ntfs3/p2_tests.tar
> 
> Those are results on ntfs3 ran within the 'linux-next' (the most recent one given the tests start date)
> As may be observed, we never skipped the "tests day" :)
> 
> There is a note should be provided on xfstests specifically. We have been using this suite
> as a part of our autotests for several years already. However the suite originate for Linux
> native file systems and a lot of cases are not applicable to the NTFS. This is one of the reasons
> why some of "red-flag" failures are there (e.g. generic/475) - they were excluded at some point of time
> and we've missed to enable it back when it was the time :)

generic/019, generic/317, generic/476 (and generic/521 and 522) are
supposed to be stress exercisers of standard functionality (read, write,
mkdir, creat, unlink) which means they ought to pass on any filesystem.

Hmm, we /dont/ have a tag for these generic exercisers.  Maybe we
should, I'll think about that.

(FWIW a minor correction: I meant to reference generic/476 and not 475,
because 475 is a looping test of crash recovery.  475 is notorious for
intermittent failures even on ext4/btrfs/xfs.)

> Thank you all for this effort to run and look closer on our code, on the next patchset, the
> 91, 317 and 475 should be resolved. And now we are looking up to other excluded tests to find out more of such.

Ok, thank you!

> Hope this will resolve some of your concerns.

It does. :)

--D

> > --D
> > 
> > > I can even give them patches and configsto make it trivially easy for
> > > them to run fstests using KVM or GCE....
> > >
> > > 				- Ted

  reply	other threads:[~2021-08-10  7:02 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-13 10:45 Hans de Goede
2021-07-13 19:15 ` Linus Torvalds
2021-07-13 20:14   ` Al Viro
2021-07-13 20:18     ` Al Viro
2021-07-13 20:24       ` Randy Dunlap
2021-07-13 20:32         ` Al Viro
2021-07-13 21:43           ` Randy Dunlap
2021-07-14 10:50     ` Rafał Miłecki
2021-07-14 14:13       ` Christoph Hellwig
2021-07-14 14:51       ` Greg KH
2021-07-14 15:59         ` Rafał Miłecki
2021-07-14 16:05           ` Matthew Wilcox
2021-07-14 16:18             ` Rafał Miłecki
2021-07-15 21:50               ` Neal Gompa
2021-07-16 11:46               ` Leonidas P. Papadakos
2021-07-16 18:07                 ` Linus Torvalds
2021-07-30 15:55                   ` Konstantin Komarov
2021-07-30 17:23                     ` Paragon NTFSv3 (was Re: [GIT PULL] vboxsf fixes for 5.14-1) Linus Torvalds
2021-08-13 16:11                       ` Konstantin Komarov
2021-08-15 20:32                         ` Stephen Rothwell
2021-08-16  3:00                         ` Kari Argillander
2021-09-02 21:55                       ` Linus Torvalds
2021-08-03 22:48                   ` [GIT PULL] vboxsf fixes for 5.14-1 Theodore Ts'o
2021-08-03 23:44                     ` Matthew Wilcox
2021-08-04  0:04                       ` Theodore Ts'o
2021-08-04  0:10                         ` Linus Torvalds
2021-08-04  0:49                           ` Theodore Ts'o
2021-08-04  1:03                             ` Darrick J. Wong
2021-08-04  6:38                               ` Kari Argillander
2021-08-04 16:30                                 ` Theodore Ts'o
2021-08-05 15:48                               ` Konstantin Komarov
2021-08-10  7:02                                 ` Darrick J. Wong [this message]
2021-09-02 22:09                           ` NTFS testing (was: " Szabolcs Szakacsits
2021-09-03 17:48                             ` Eric Biggers
2021-09-03 21:17                               ` Szabolcs Szakacsits
2021-07-17 16:47                 ` Pali Rohár
2021-07-14 16:13           ` Darrick J. Wong
2021-07-14 16:18             ` Christoph Hellwig
2021-07-14 16:38               ` Gao Xiang
2021-07-14 20:03               ` Eric W. Biederman
2021-07-15 22:14               ` Darrick J. Wong
2021-07-13 19:17 ` pr-tracker-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210810070255.GJ3601405@magnolia \
    --to=djwong@kernel.org \
    --cc=almaz.alexandrovich@paragon-software.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hdegoede@redhat.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=papadakospan@gmail.com \
    --cc=torvalds@linux-foundation.org \
    --cc=tytso@mit.edu \
    --cc=viro@zeniv.linux.org.uk \
    --cc=willy@infradead.org \
    --cc=zajec5@gmail.com \
    --subject='Re: [GIT PULL] vboxsf fixes for 5.14-1' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).