Linux-kselftest Archive on lore.kernel.org
 help / color / Atom feed
From: Rafael Aquini <aquini@redhat.com>
To: Michal Hocko <mhocko@kernel.org>
Cc: Shakeel Butt <shakeelb@google.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-kselftest@vger.kernel.org, shuah@kernel.org
Subject: Re: [PATCH] tools/testing/selftests/vm/mlock2-tests: fix mlock2 false-negative errors
Date: Mon, 23 Mar 2020 11:02:59 -0400
Message-ID: <20200323150259.GD23364@optiplex-lnx> (raw)
In-Reply-To: <20200323145106.GM7524@dhcp22.suse.cz>

On Mon, Mar 23, 2020 at 03:51:06PM +0100, Michal Hocko wrote:
> On Mon 23-03-20 10:42:40, Rafael Aquini wrote:
> > On Mon, Mar 23, 2020 at 08:52:08AM +0100, Michal Hocko wrote:
> > > On Sun 22-03-20 09:36:49, Shakeel Butt wrote:
> > > > On Sat, Mar 21, 2020 at 9:31 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Sat, 21 Mar 2020 22:03:26 -0400 Rafael Aquini <aquini@redhat.com> wrote:
> > > > >
> > > > > > > > + * In order to sort out that race, and get the after fault checks consistent,
> > > > > > > > + * the "quick and dirty" trick below is required in order to force a call to
> > > > > > > > + * lru_add_drain_all() to get the recently MLOCK_ONFAULT pages moved to
> > > > > > > > + * the unevictable LRU, as expected by the checks in this selftest.
> > > > > > > > + */
> > > > > > > > +static void force_lru_add_drain_all(void)
> > > > > > > > +{
> > > > > > > > + sched_yield();
> > > > > > > > + system("echo 1 > /proc/sys/vm/compact_memory");
> > > > > > > > +}
> > > > > > >
> > > > > > > What is the sched_yield() for?
> > > > > > >
> > > > > >
> > > > > > Mostly it's there to provide a sleeping gap after the fault, whithout
> > > > > > actually adding an arbitrary value with usleep().
> > > > > >
> > > > > > It's not a hard requirement, but, in some of the tests I performed
> > > > > > (whithout that sleeping gap) I would still see around 1% chance
> > > > > > of hitting the false-negative. After adding it I could not hit
> > > > > > the issue anymore.
> > > > >
> > > > > It's concerning that such deep machinery as pagevec draining is visible
> > > > > to userspace.
> > > > >
> > > > 
> > > > We already have other examples like memcg stats where the
> > > > optimizations like batching per-cpu stats collection exposes
> > > > differences to the userspace. I would not be that worried here.
> > > 
> > > Agreed! Tests should be more tolerant for counters imprecision.
> > > Unevictable LRU is an optimization and transition to that list is a
> > > matter of an internal implementation detail.
> > >
> > > > > I suppose that for consistency and correctness we should perform a
> > > > > drain prior to each read from /proc/*/pagemap.  Presumably this would
> > > > > be far too expensive.
> > > > >
> > > > > Is there any other way?  One such might be to make the MLOCK_ONFAULT
> > > > > pages bypass the lru_add_pvecs?
> > > > >
> > > > 
> > > > I would rather prefer to have something similar to
> > > > /proc/sys/vm/stat_refresh which drains the pagevecs.
> > > 
> > > No, please don't. Pagevecs draining is by far not the only batching
> > > scheme we use and an interface like this would promise users to
> > > effectivelly force flushing all of them.
> > > 
> > > Can we simply update the test to be more tolerant to imprecisions
> > > instead?
> > > 
> > 
> > I don't think, thouhg, that this particular test case can be entirely
> > reduced as "counter imprecison".
> > 
> > The reason I think this is a different beast, is that having the page
> > being flagged as PG_unevictable is expected part of the aftermath of
> > a mlock* call. This selftest is, IMO, correctly verifying that fact,
> > as it checks the functionality correctness.
> > 
> > The problem boils down to the fact that the page would immediately
> > be flagged as PG_unevictable after the mlock (under MCL_FUTURE|MCL_ONFAULT
> > semantics) call, and the test was expecting it, and commit 9c4e6b1a7027f
> > changed that by "delaying" that flag setting.
> 
> As I've tried to explain in other email in this email thread. The test
> was exploiting a certain user visible side effect. The unevictable flag
> or the placement on the unevictable LRU list is are not really needed
> for the user contract correctness. That means that the test is not
> really correct. Working around that by trying to enforce kernel to
> comply with the test expectations is just plain wrong at least for two
> reasons 1) you cannot expect or event do not want userspace to do the
> same because the behavior might change in the future 2) the test is not
> really testing for correctness in the first place.
>

Sorry, Michal, it seems we keep going back and forth (I just replied to
your comment on the other thread)

The selftest also checks the kernel visible effect, via
/proc/kpageflags, and that's where it fails after 9c4e6b1a7027f.

As I mentioned before, I think it is a reasonable check, given this
is a kernel selftest, although we need to compensate it for the
differences between its expectations and what the kernel is doing
currently.

-- Rafael


  reply index

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-22  1:35 Rafael Aquini
2020-03-22  1:43 ` Andrew Morton
2020-03-22  2:03   ` Rafael Aquini
2020-03-22  4:31     ` Andrew Morton
2020-03-22  5:41       ` Rafael Aquini
2020-03-22 16:40         ` Shakeel Butt
2020-03-22 16:36       ` Shakeel Butt
2020-03-23  7:52         ` Michal Hocko
2020-03-23 14:42           ` Rafael Aquini
2020-03-23 14:51             ` Michal Hocko
2020-03-23 15:02               ` Rafael Aquini [this message]
2020-03-23 15:12                 ` Michal Hocko
2020-03-23 15:41                   ` Rafael Aquini
2020-03-23 15:51                     ` Michal Hocko
2020-03-23 15:54                       ` Rafael Aquini
2020-03-24 15:42                         ` Michal Hocko
2020-03-24 15:49                           ` Rafael Aquini
2020-03-26  0:49                             ` Andrew Morton
2020-03-26  2:22                               ` Rafael Aquini
2020-03-26  6:49                               ` Michal Hocko
2020-03-26 19:58                                 ` Andrew Morton
2020-03-26 20:16                                   ` Rafael Aquini
2020-03-26 20:19                                   ` Rafael Aquini
2020-03-22 16:31 ` Shakeel Butt
2020-03-23 14:16   ` Rafael Aquini
2020-03-23 14:29     ` Michal Hocko
2020-03-23 14:55       ` Rafael Aquini
2020-03-23 15:01       ` Michal Hocko
2020-03-23 15:07         ` Rafael Aquini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200323150259.GD23364@optiplex-lnx \
    --to=aquini@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=mhocko@kernel.org \
    --cc=shakeelb@google.com \
    --cc=shuah@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-kselftest Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-kselftest/0 linux-kselftest/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kselftest linux-kselftest/ https://lore.kernel.org/linux-kselftest \
		linux-kselftest@vger.kernel.org
	public-inbox-index linux-kselftest

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kselftest


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git