Re: [LTP] [PATCH 5/5] syscalls/fanotify10: Add test cases for evictable ignore mark

[Jan Stancek <jstancek@redhat.com>]

On Wed, Aug 24, 2022 at 5:24 PM Jan Kara <jack@suse.cz> wrote:
>
> On Tue 12-07-22 10:19:19, Jan Stancek wrote:
> > On Sat, Jul 9, 2022 at 12:09 PM Amir Goldstein <amir73il@gmail.com> wrote:
> > >
> > > On Thu, Jul 7, 2022 at 3:49 PM Jan Stancek <jstancek@redhat.com> wrote:
> > > >
> > > > On Thu, Jun 30, 2022 at 10:21 AM Amir Goldstein <amir73il@gmail.com> wrote:
> > > > >
> > > > > On Thu, Jun 30, 2022 at 9:27 AM Jan Stancek <jstancek@redhat.com> wrote:
> > > > > >
> > > > > > On Mon, Jun 13, 2022 at 4:39 PM Amir Goldstein <amir73il@gmail.com> wrote:
> > > > > > >
> > > > > > > Test multiple groups with evictable mark with ignore mask
> > > > > > >
> > > > > > > Signed-off-by: Amir Goldstein <amir73il@gmail.com>
> > > > > > > ---
> > > > > > >  .../kernel/syscalls/fanotify/fanotify10.c     | 78 +++++++++++++++++++
> > > > > > >  1 file changed, 78 insertions(+)
> > > > > > >
> > > > > > > diff --git a/testcases/kernel/syscalls/fanotify/fanotify10.c b/testcases/kernel/syscalls/fanotify/fanotify10.c
> > > > > > > index b9a50672d..52277d0b7 100644
> > > > > > > --- a/testcases/kernel/syscalls/fanotify/fanotify10.c
> > > > > > > +++ b/testcases/kernel/syscalls/fanotify/fanotify10.c
> > > > > > > @@ -71,6 +71,7 @@ static char event_buf[EVENT_BUF_LEN];
> > > > > > >  static int exec_events_unsupported;
> > > > > > >  static int fan_report_dfid_unsupported;
> > > > > > >  static int filesystem_mark_unsupported;
> > > > > > > +static int evictable_mark_unsupported;
> > > > > > >
> > > > > > >  #define MOUNT_PATH "fs_mnt"
> > > > > > >  #define MNT2_PATH "mntpoint"
> > > > > > > @@ -90,6 +91,10 @@ static int filesystem_mark_unsupported;
> > > > > > >  #define FILE_EXEC_PATH2 MNT2_PATH"/"TEST_APP
> > > > > > >  #define FILE2_EXEC_PATH2 MNT2_PATH"/"TEST_APP2
> > > > > > >
> > > > > > > +#define DROP_CACHES_FILE "/proc/sys/vm/drop_caches"
> > > > > > > +#define CACHE_PRESSURE_FILE "/proc/sys/vm/vfs_cache_pressure"
> > > > > > > +
> > > > > > > +static int old_cache_pressure;
> > > > > > >  static pid_t child_pid;
> > > > > > >  static int bind_mount_created;
> > > > > > >  static unsigned int num_classes = NUM_CLASSES;
> > > > > > > @@ -98,12 +103,14 @@ enum {
> > > > > > >         FANOTIFY_INODE,
> > > > > > >         FANOTIFY_MOUNT,
> > > > > > >         FANOTIFY_FILESYSTEM,
> > > > > > > +       FANOTIFY_EVICTABLE,
> > > > > > >  };
> > > > > > >
> > > > > > >  static struct fanotify_mark_type fanotify_mark_types[] = {
> > > > > > >         INIT_FANOTIFY_MARK_TYPE(INODE),
> > > > > > >         INIT_FANOTIFY_MARK_TYPE(MOUNT),
> > > > > > >         INIT_FANOTIFY_MARK_TYPE(FILESYSTEM),
> > > > > > > +       INIT_FANOTIFY_MARK_TYPE(EVICTABLE),
> > > > > > >  };
> > > > > > >
> > > > > > >  static struct tcase {
> > > > > > > @@ -289,14 +296,59 @@ static struct tcase {
> > > > > > >                 0,
> > > > > > >                 FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > > >         },
> > > > > > > +       /* Evictable ignore mark test cases */
> > > > > > > +       {
> > > > > > > +               "don't ignore mount events created on file with evicted ignore mark",
> > > > > > > +               MOUNT_PATH, FANOTIFY_MOUNT,
> > > > > > > +               FILE_PATH, FANOTIFY_EVICTABLE,
> > > > > > > +               0,
> > > > > > > +               FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > > > +       },
> > > > > > > +       {
> > > > > > > +               "don't ignore fs events created on a file with evicted ignore mark",
> > > > > > > +               MOUNT_PATH, FANOTIFY_FILESYSTEM,
> > > > > > > +               FILE_PATH, FANOTIFY_EVICTABLE,
> > > > > > > +               0,
> > > > > > > +               FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > > > +       },
> > > > > > > +       {
> > > > > > > +               "don't ignore mount events created inside a parent with evicted ignore mark",
> > > > > > > +               MOUNT_PATH, FANOTIFY_MOUNT,
> > > > > > > +               DIR_PATH, FANOTIFY_EVICTABLE,
> > > > > > > +               FAN_EVENT_ON_CHILD,
> > > > > > > +               FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > > > +       },
> > > > > > > +       {
> > > > > > > +               "don't ignore fs events created inside a parent with evicted ignore mark",
> > > > > > > +               MOUNT_PATH, FANOTIFY_FILESYSTEM,
> > > > > > > +               DIR_PATH, FANOTIFY_EVICTABLE,
> > > > > > > +               FAN_EVENT_ON_CHILD,
> > > > > > > +               FILE_PATH, FAN_OPEN, FAN_OPEN
> > > > > > > +       },
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > we are seeing some sporadic failures from this last testcase, with
> > > > > > recent upstream kernels (v5.19-rc4-14-g941e3e791269).
> > > > > > Has anyone also ran into it and knows if it's bug on test side or kernel?
> > >
> > > Hi Jan,
> > >
> > > I am traveling so I cannot work on solving the problem for the next week,
> > > but I can explain the problem and offer suggestions for short term solutions
> > > and maybe a longer term solution.
> > >
> > > > > >
> > > > > > fanotify10.c:496: TINFO: Test #27: don't ignore fs events created
> > > > > > inside a parent with evicted ignore mark
> > > > > > fanotify10.c:338: TPASS: No fanotify inode marks as expected
> > > > > > fanotify10.c:338: TPASS: No fanotify inode marks as expected
> > > > > > fanotify10.c:338: TPASS: No fanotify inode marks as expected
> > > > > > fanotify10.c:340: TFAIL: Unexpected inode mark (mflags=240,
> > > > > > mask=8000020 ignored_mask=20)
> > > > >
> > > > > It is a test bug.
> > > > > The problem is that we want to evict an inode, but there is no
> > > > > reliable mechanism to do that.
> > > > >
> > > > > This is the reason for this workaround in fanotify23:
> > > > >
> > > > >         /* Shrinkers on other fs do not work reliably enough to
> > > > > guarantee mark eviction on drop_caches */
> > > > >         .dev_fs_type = "ext2",
> > > > >
> > > > > I did not encounter the problem with fanotify10 myself, but it should
> > > > > be the same.
> > > > > fanotify10 is not filesystem dependent, so if you can apply the same workaround
> > > > > from fanotify23 and it works on your systems we can do that.
> > > >
> > > > Test is using default fs type, which should already be ext2.
> > >
> > > I see. You can try xfs on your system to see if it behaves better,
> > > because xfs has a specialized inode shriker.
> > >
> > > > Here's a more complete log from failed test:
> > > > https://s3.us-east-1.amazonaws.com/arr-cki-prod-datawarehouse-public/datawarehouse-public/2022/06/30/redhat:576928171/build_ppc64le_redhat:576928171_ppc64le/tests/1/results_0001/job.01/recipes/12221009/tasks/8/logs/syscalls.fail.log
> > > >
> > >
> > > Are the failures only in the first test iteration?
> >
> > I think so. It runs in automated environment, where we see failures
> > about once a week. I haven't managed to reproduce it by hand yet.
> >
> > >
> > > As a long shot, I would try to remove mount_cycle() from setup().
> > > I see that fanotify23 does not have it and it is not really needed, so
> > > it may help
> > > fanotify10 pass the first iteration.
> > >
> > > The problem is that the drop_caches knob is not a reliable way to evict inodes,
> > > so when a test like fanotify10 needs to evict an inode, random factors are mixed
> > > into the test run.
> > >
> > > As a quick band aid, I suggest to replace the TFAIL, "Unexpected inode mark"
> > > in show_fanotify_marks() with TCONF, because the test failed to setup an
> > > "evictable ignored mark that gets evicted", propagate a failure return
> > > value from
> > > create_fanotify_groups() => show_fanotify_marks() and skip the test case
> > > instead of failing it.
> >
> > Thanks for suggestion.
>
> Is this still an issue?

It still rarely fails - two instances in last 3 weeks:
6.0.0-0.rc1.13.test.eln
https://s3.us-east-1.amazonaws.com/arr-cki-prod-datawarehouse-public/datawarehouse-public/2022/08/15/redhat:613661619/build_x86_64_redhat:613661619_x86_64/tests/3/results_0001/job.01/recipes/12445146/tasks/10/logs/syscalls.fail.log

5.20.0-0.rc0.aea23e7c464b
https://s3.us-east-1.amazonaws.com/arr-cki-prod-datawarehouse-public/datawarehouse-public/2022/08/14/redhat:613063904/build_x86_64_redhat:613063904_x86_64_debug/tests/2/results_0001/job.01/recipes/12440847/tasks/9/logs/syscalls.fail.log

> I didn't see anything happening in the fanotify10
> test upstream. If the issue still happens, maybe something like the
> attached patch may improve the situation? Jan, do you have a chance to test
> it?

I can't reproduce failure on demand. If the patch doesn't have any
side-effects, then let's apply it, and we'll see in couple weeks.

Regards,
Jan

>
>                                                                 Honza
> --
> Jan Kara <jack@suse.com>
> SUSE Labs, CR


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp