From: Peter Zijlstra <peterz@infradead.org>
To: Avi Kivity <avi@redhat.com>
Cc: "Nikunj A. Dadhania" <nikunj@linux.vnet.ibm.com>,
mingo@elte.hu, jeremy@goop.org, mtosatti@redhat.com,
kvm@vger.kernel.org, x86@kernel.org, vatsa@linux.vnet.ibm.com,
linux-kernel@vger.kernel.org, hpa@zytor.com
Subject: Re: [RFC PATCH v1 3/5] KVM: Add paravirt kvm_flush_tlb_others
Date: Tue, 01 May 2012 12:57:07 +0200 [thread overview]
Message-ID: <1335869827.13683.133.camel@twins> (raw)
In-Reply-To: <4F9FBF38.2060903@redhat.com>
On Tue, 2012-05-01 at 13:47 +0300, Avi Kivity wrote:
> On 05/01/2012 12:39 PM, Peter Zijlstra wrote:
> > On Sun, 2012-04-29 at 15:23 +0300, Avi Kivity wrote:
> > > On 04/27/2012 07:24 PM, Nikunj A. Dadhania wrote:
> > > > flush_tlb_others_ipi depends on lot of statics in tlb.c. Replicated
> > > > the flush_tlb_others_ipi as kvm_flush_tlb_others to further adapt to
> > > > paravirtualization.
> > > >
> > > > Use the vcpu state information inside the kvm_flush_tlb_others to
> > > > avoid sending ipi to pre-empted vcpus.
> > > >
> > > > * Do not send ipi's to offline vcpus and set flush_on_enter flag
> > >
> > > get_user_pages_fast() depends on the IPI to hold off page table teardown
> > > while they are locklessly walked with interrupts disabled. If a vcpu
> > > were to be preempted while in this critical section, another vcpu
> > > tearing down page tables would go ahead and destroy them. when the
> > > preempted vcpu resumes it then touches the freed pages.
> > >
> > > We could try to teach kvm and get_user_pages_fast() about this, but this
> > > is intrusive. Another option is to replace the cpu_relax() loop with
> > > something that sleeps and is then woken up by the TLB IPI handler if needed.
> >
> > I think something like
> >
> > select HAVE_RCU_TABLE_FREE if PARAVIRT
> >
> > or somesuch is just about all it takes.
> >
> > A slightly better option would be to wrap all that tlb_*table* goo into
> > paravirt stuff and only do the RCU free when paravirt is indeed enabled,
> > but other than that you're there.
>
> I infer from this that there is a cost involved with rcu freeing. Any
> idea how much?
No idea, so far that code has only been used on platforms that required
it so they didn't have a choice in the matter.
> Looks like this increases performance for the overcommitted case, and
> also for the case where many vcpus are sleeping, while reducing
> performance for the uncontended, high duty cycle case.
Sounds backwards if you put it like that ;-)
> > This should work because the preempted vcpu's RCU state would also be
> > stalled and thus avoids the actual page-table from going away.
>
> It can be unstalled at any moment. But spin_lock_irq() > rcu_read_lock().
Right, but since gup_fast has IRQs disabled the RCU state machine (as
driven by the tick) won't actually do anything until its done.
To be clear, the case was where the gup_fast() performing vcpu was
preempted in the middle of gup_fast(), on wakeup it would perform the
TLB flush on the virt-enter hook, but meanwhile a sibling vcpu might
have free'd the page-tables.
By using call_rcu_sched() to free the page-tables you'd need to receive
and process at least one tick on the woken up cpu after the freeing, but
since the in-progress gup_fast() will have IRQs disabled this will be
delayed.
Anyway, I don't have any idea about the costs involved with
HAVE_RCU_TABLE_FREE, but I don't think its much.. otherwise these other
platforms (PPC,SPARC) wouldn't have used it, gup_fast() is a very
specific case, whereas mmu-gather is something affecting pretty much all
tasks.
But mostly my comment was due to you saying modifying gup_fast() would
be difficult.. I was thinking the one Kconfig line wasn't as onerous ;-)
next prev parent reply other threads:[~2012-05-01 10:57 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-04-27 16:23 [RFC PATCH v1 0/5] KVM paravirt remote flush tlb Nikunj A. Dadhania
2012-04-27 16:23 ` [RFC PATCH v1 1/5] KVM Guest: Add VCPU running/pre-empted state for guest Nikunj A. Dadhania
2012-05-01 1:03 ` Raghavendra K T
2012-05-01 3:25 ` Nikunj A Dadhania
2012-04-27 16:23 ` [RFC PATCH v1 2/5] KVM-HV: " Nikunj A. Dadhania
2012-04-27 16:24 ` [RFC PATCH v1 3/5] KVM: Add paravirt kvm_flush_tlb_others Nikunj A. Dadhania
2012-04-29 12:23 ` Avi Kivity
2012-05-01 3:34 ` Nikunj A Dadhania
2012-05-01 9:39 ` Peter Zijlstra
2012-05-01 10:47 ` Avi Kivity
2012-05-01 10:57 ` Peter Zijlstra [this message]
2012-05-01 10:59 ` Peter Zijlstra
2012-05-01 22:49 ` Jeremy Fitzhardinge
2012-05-03 14:09 ` Stefano Stabellini
2012-05-01 12:12 ` Avi Kivity
2012-05-01 14:59 ` Peter Zijlstra
2012-05-01 15:31 ` Avi Kivity
2012-05-01 15:36 ` Peter Zijlstra
2012-05-01 15:39 ` Avi Kivity
2012-05-01 15:42 ` Peter Zijlstra
2012-05-01 15:11 ` Peter Zijlstra
2012-05-01 15:33 ` Avi Kivity
2012-05-01 15:14 ` Peter Zijlstra
2012-05-01 15:36 ` Avi Kivity
2012-05-01 16:16 ` Peter Zijlstra
2012-05-01 16:43 ` Paul E. McKenney
2012-05-01 16:18 ` Peter Zijlstra
2012-05-01 16:20 ` Peter Zijlstra
2012-05-02 8:51 ` Nikunj A Dadhania
2012-05-02 10:20 ` Peter Zijlstra
2012-05-02 13:53 ` Nikunj A Dadhania
2012-05-04 4:32 ` Nikunj A Dadhania
2012-05-04 11:44 ` Srivatsa Vaddagiri
2012-05-07 3:10 ` Nikunj A Dadhania
2012-04-27 16:26 ` [RFC PATCH v1 4/5] KVM: get kvm_kick_vcpu out for pv_flush Nikunj A. Dadhania
2012-04-27 16:27 ` [RFC PATCH v1 5/5] KVM: Introduce PV kick in flush tlb Nikunj A. Dadhania
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1335869827.13683.133.camel@twins \
--to=peterz@infradead.org \
--cc=avi@redhat.com \
--cc=hpa@zytor.com \
--cc=jeremy@goop.org \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=mtosatti@redhat.com \
--cc=nikunj@linux.vnet.ibm.com \
--cc=vatsa@linux.vnet.ibm.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).