Re: [PATCH] mm/oom_kill: Ensure MMU notifier range_end() is paired with range_start()

From: Sean Christopherson <seanjc@google.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Andrew Morton" <akpm@linux-foundation.org>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	"David Rientjes" <rientjes@google.com>,
	"Ben Gardon" <bgardon@google.com>,
	"Michal Hocko" <mhocko@suse.com>,
	"Jérôme Glisse" <jglisse@redhat.com>,
	"Andrea Arcangeli" <aarcange@redhat.com>,
	"Johannes Weiner" <hannes@cmpxchg.org>,
	"Dimitri Sivanich" <dimitri.sivanich@hpe.com>
Subject: Re: [PATCH] mm/oom_kill: Ensure MMU notifier range_end() is paired with range_start()
Date: Wed, 10 Mar 2021 23:22:09 -0800	[thread overview]
Message-ID: <YEnFIZde1t+TF4y6@google.com> (raw)
In-Reply-To: <20210311015013.GS444867@ziepe.ca>

On Wed, Mar 10, 2021, Jason Gunthorpe wrote:
> On Wed, Mar 10, 2021 at 05:20:01PM -0800, Sean Christopherson wrote:
> 
> > > Which I believe is fatal to kvm? These notifiers certainly do not only
> > > happen at process exit.
> > 
> > My point about the process dying is that the existing bug that causes
> > mmu_notifier_count to become imbalanced is benign only because the process is
> > being killed, and thus KVM will stop running its vCPUs.
> 
> Are you saying we only call non-blocking invalidate during a process
> exit event?? 

Yes?  __oom_reap_task_mm() is the only user of _nonblock(), if that's what you're
asking.

> > > So, both of the remaining _end users become corrupted with this patch!
> > 
> > I don't follow.  mn_hlist_invalidate_range_start() iterates over all
> > notifiers, even if a notifier earlier in the chain failed.  How will
> > KVM become imbalanced?
> 
> Er, ok, that got left in a weird way. There is another "bug" where end
> is not supposed to be called if the start failed.

Ha, the best kind of feature.  :-)

> > The existing _end users never fail their _start.  If KVM started failing its
> > start, then yes, it could get corrupted.  
> 
> Well, maybe that is the way out of this now. If we don't permit a
> start to fail if there is an end then we have no problem to unwind it
> as we can continue to call everything. This can't be backported too
> far though, the itree notifier conversions are what made the WARN_ON
> safe today.
> 
> Something very approximately like this is closer to my preference:

Makes sense.  I don't have a strong preference, I'll give this a spin tomorrow.

> diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c
> index 61ee40ed804ee5..6d5cd20f81dadc 100644
> --- a/mm/mmu_notifier.c
> +++ b/mm/mmu_notifier.c
> @@ -501,10 +501,25 @@ static int mn_hlist_invalidate_range_start(
>  						"");
>  				WARN_ON(mmu_notifier_range_blockable(range) ||
>  					_ret != -EAGAIN);
> +				/*
> +				 * We call all the notifiers on any EAGAIN,
> +				 * there is no way for a notifier to know if
> +				 * its start method failed, thus a start that
> +				 * does EAGAIN can't also do end.
> +				 */
> +				WARN_ON(ops->invalidate_range_end);
>  				ret = _ret;
>  			}
>  		}
>  	}
> +
> +	if (ret) {
> +		/* Must be non-blocking to get here*/
> +		hlist_for_each_entry_rcu (subscription, &subscriptions->list,
> +					  hlist, srcu_read_lock_held(&srcu))
> +			subscription->ops->invalidate_range_end(subscription,
> +								range);
> +	}
>  	srcu_read_unlock(&srcu, id);
>  
>  	return ret;