Re: [PATCH 12/11 TESTSUITE] audit_testsuite: Add stress test for tree watches

[Jan Kara <jack@suse.cz>]

On Mon 15-10-18 11:39:51, Paul Moore wrote:
> On Mon, Oct 15, 2018 at 6:04 AM Jan Kara <jack@suse.cz> wrote:
> > On Thu 11-10-18 19:03:53, Paul Moore wrote:
> > > On October 11, 2018 7:39:39 AM Jan Kara <jack@suse.cz> wrote:
> > > > On Wed 10-10-18 02:43:46, Paul Moore wrote:
> > > >> On Tue, Oct 9, 2018 at 3:40 AM Jan Kara <jack@suse.cz> wrote:
> > > >>> On Fri 05-10-18 17:06:22, Paul Moore wrote:
> > > >>>> On Tue, Sep 4, 2018 at 12:06 PM Jan Kara <jack@suse.cz> wrote:
> > > >>>>> Add stress test for stressing audit tree watches by adding and deleting
> > > >>>>> rules while events are generated and watched filesystems are mounted and
> > > >>>>> unmounted in parallel.
> > > >>>>>
> > > >>>>> Signed-off-by: Jan Kara <jack@suse.cz>
> > > >>>>> ---
> > > >>>>> tests/stress_tree/Makefile |   8 +++
> > > >>>>> tests/stress_tree/test     | 171 +++++++++++++++++++++++++++++++++++++++++++++
> > > >>>>> 2 files changed, 179 insertions(+)
> > > >>>>> create mode 100644 tests/stress_tree/Makefile
> > > >>>>> create mode 100755 tests/stress_tree/test
> > > >>>>
> > > >>>> No commentary on the test itself, other than perhaps it should live
> > > >>>> under test_manual/, but in running the tests in a loop today I am
> > > >>>> reliably able to panic my test kernel after ~30m or so.
> > > >>>
> > > >>> Interesting. How do you run the test?
> > > >>
> > > >> Nothing fancy, just a simple bash loop:
> > > >>
> > > >> # cd tests/stress_tree
> > > >> # while ./test; do /bin/true; done
> > > >
> > > > OK, I did succeed in reproducing some problems with my patches - once I was
> > > > able to trigger a livelock and following softlockup warning - this is
> > > > actually a problem introduced by my patches, and once a use after free
> > > > issue (not sure what that was since after I've added some debugging I
> > > > wasn't able to trigger it anymore). Anyway, I'll try more after fixing the
> > > > livelock. Do you want me to add fixes on top of my series or just fixup the
> > > > original series?
> > >
> > > Since these are pretty serious bugs, and I try to avoid merging
> > > known-broken patches which will go up to Linus, why don't you go ahead
> > > and respin the patchset with the new fixes included.  You can also use
> > > the opportunity to squash in the rename patch and fix that mid-patchset
> > > compilation problem that I fixed up during the merge.
> >
> > OK, I'm now testing a version with the softlockup fixed and some locking
> > around untag_chunk() simplified when I had to meddle with that anyway. I'll
> > see if I can hit further failures...
> 
> Thanks for the update, let me know how the testing goes ...

OK, yesterday I've finally nailed down the list corruption. Testing has ran
fine for 10 hours, after that it crashed due to independent problem in
fsnotify infrastructure. I've restarted the testing but I think patches are
good for another posting - will send in a minute.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR