From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: Kevin Tian <kevin.tian@intel.com>, Keir Fraser <keir@xen.org>,
Andrew Cooper <andrew.cooper3@citrix.com>,
Dario Faggioli <dario.faggioli@citrix.com>,
"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
Jan Beulich <jbeulich@suse.com>, Feng Wu <feng.wu@intel.com>
Subject: Re: Ideas Re: [PATCH v14 1/2] vmx: VT-d posted-interrupt core logic handling
Date: Mon, 7 Mar 2016 10:53:05 -0500 [thread overview]
Message-ID: <20160307155304.GD5402@localhost.localdomain> (raw)
In-Reply-To: <CAFLBxZaZ6qgaTwVUT99tvHXQBhacGdfJ94GLhz6HtYWdqU4zZg@mail.gmail.com>
On Mon, Mar 07, 2016 at 11:21:33AM +0000, George Dunlap wrote:
> On Fri, Mar 4, 2016 at 10:00 PM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> >> +/* Handle VT-d posted-interrupt when VCPU is blocked. */
> >> +static void pi_wakeup_interrupt(struct cpu_user_regs *regs)
> >> +{
> >> + struct arch_vmx_struct *vmx, *tmp;
> >> + spinlock_t *lock = &per_cpu(vmx_pi_blocking, smp_processor_id()).lock;
> >> + struct list_head *blocked_vcpus =
> >> + &per_cpu(vmx_pi_blocking, smp_processor_id()).list;
> >> +
> >> + ack_APIC_irq();
> >> + this_cpu(irq_count)++;
> >> +
> >> + spin_lock(lock);
> >> +
> >> + /*
> >> + * XXX: The length of the list depends on how many vCPU is current
> >> + * blocked on this specific pCPU. This may hurt the interrupt latency
> >> + * if the list grows to too many entries.
> >> + */
> >> + list_for_each_entry_safe(vmx, tmp, blocked_vcpus, pi_blocking.list)
> >> + {
> >
> >
> > My recollection of the 'most-horrible' case of this being really bad is when
> > the scheduler puts the vCPU0 and VCPU1 of the guest on the same pCPU (as an example)
> > and they round-robin all the time.
> >
> > <handwaving>
> > Would it be perhaps possible to have an anti-affinity flag to deter the
> > scheduler from this? That is whichever struct vcpu has 'anti-affinity' flag
> > set - the scheduler will try as much as it can _to not_ schedule the 'struct vcpu'
> > if the previous 'struct vcpu' had this flag as well on this pCPU?
>
> Well having vcpus from the same guest on the same pcpu is problematic
> for a number of reasons -- spinlocks first and foremost. So in
> general trying to avoid that would be useful for most guests.
PV ticketlocks in HVM and PV guests make this "manageable".
>
> The thing with scheduling is that it's a bit like economics: it seems
> simple but it's actually not at all obvious what the emergent behavior
> will be from adding a simple rule. :-)
<nods>
>
> On the whole it seems unlikely that having two vcpus on a single pcpu
> is a "stable" situation -- it's likely to be pretty transient, and
> thus not have a major impact on performance.
Except that we are concerned with it - in fact we are disabling this
feature because it may happen. How do we make sure it does not happen
all the time? Or at least do some back-off if things do get
in this situation.
>
> That said, the load balancing code from credit2 *should*, in theory,
> make it easier to implement this sort of thing; it has the concept of
> a "cost" that it's trying to minimize; so you could in theory add a
> "cost" to configurations where vcpus from the same processor share the
> same pcpu. Then it's not a hard-and-fast rule: if you have more vcpus
> than pcpus, the scheduler will just deal. :-)
>
> But I think some profiling is in order before anyone does serious work on this.
I appreciate your response being 'profiling' instead of 'Are you
NUTS!?' :-)
>
> -George
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
next prev parent reply other threads:[~2016-03-07 15:53 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-02-29 3:00 [PATCH v14 0/2] Add VT-d Posted-Interrupts support Feng Wu
2016-02-29 3:00 ` [PATCH v14 1/2] vmx: VT-d posted-interrupt core logic handling Feng Wu
2016-02-29 13:33 ` Jan Beulich
2016-02-29 13:52 ` Dario Faggioli
2016-03-01 5:39 ` Wu, Feng
2016-03-01 9:24 ` Jan Beulich
2016-03-01 10:16 ` George Dunlap
2016-03-01 13:06 ` Wu, Feng
2016-03-01 5:24 ` Tian, Kevin
2016-03-01 5:39 ` Wu, Feng
2016-03-04 22:00 ` Ideas " Konrad Rzeszutek Wilk
2016-03-07 11:21 ` George Dunlap
2016-03-07 15:53 ` Konrad Rzeszutek Wilk [this message]
2016-03-07 16:19 ` Dario Faggioli
2016-03-07 20:23 ` Konrad Rzeszutek Wilk
2016-03-08 12:02 ` George Dunlap
2016-03-08 13:10 ` Wu, Feng
2016-03-08 14:42 ` George Dunlap
2016-03-08 15:42 ` Jan Beulich
2016-03-08 17:05 ` George Dunlap
2016-03-08 17:26 ` Jan Beulich
2016-03-08 18:38 ` George Dunlap
2016-03-09 5:06 ` Wu, Feng
2016-03-09 13:39 ` Jan Beulich
2016-03-09 16:01 ` George Dunlap
2016-03-09 16:31 ` Jan Beulich
2016-03-09 16:23 ` On setting clear criteria for declaring a feature acceptable (was "vmx: VT-d posted-interrupt core logic handling") George Dunlap
2016-03-09 16:58 ` On setting clear criteria for declaring a feature acceptable Jan Beulich
2016-03-09 18:02 ` On setting clear criteria for declaring a feature acceptable (was "vmx: VT-d posted-interrupt core logic handling") David Vrabel
2016-03-10 1:15 ` Wu, Feng
2016-03-10 9:30 ` George Dunlap
2016-03-10 5:09 ` Tian, Kevin
2016-03-10 8:07 ` vmx: VT-d posted-interrupt core logic handling Jan Beulich
2016-03-10 8:43 ` Tian, Kevin
2016-03-10 9:05 ` Jan Beulich
2016-03-10 9:20 ` Tian, Kevin
2016-03-10 10:05 ` Tian, Kevin
2016-03-10 10:18 ` Jan Beulich
2016-03-10 10:35 ` David Vrabel
2016-03-10 10:46 ` George Dunlap
2016-03-10 11:16 ` David Vrabel
2016-03-10 11:49 ` George Dunlap
2016-03-10 13:24 ` Jan Beulich
2016-03-10 11:00 ` George Dunlap
2016-03-10 11:21 ` Dario Faggioli
2016-03-10 13:36 ` Wu, Feng
2016-05-17 13:27 ` Konrad Rzeszutek Wilk
2016-05-19 7:22 ` Wu, Feng
2016-03-10 10:41 ` George Dunlap
2016-03-09 5:22 ` Ideas Re: [PATCH v14 1/2] " Wu, Feng
2016-03-09 11:25 ` George Dunlap
2016-03-09 12:06 ` Wu, Feng
2016-02-29 3:00 ` [PATCH v14 2/2] Add a command line parameter for VT-d posted-interrupts Feng Wu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160307155304.GD5402@localhost.localdomain \
--to=konrad.wilk@oracle.com \
--cc=George.Dunlap@eu.citrix.com \
--cc=andrew.cooper3@citrix.com \
--cc=dario.faggioli@citrix.com \
--cc=feng.wu@intel.com \
--cc=jbeulich@suse.com \
--cc=keir@xen.org \
--cc=kevin.tian@intel.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).