From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Zhenzhong Duan <zhenzhong.duan@oracle.com>,
"xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
Yuval Shaia <yuval.shaia@oracle.com>,
Feng Jin <joe.jin@oracle.com>, Chien Yen <chien.yen@oracle.com>
Subject: Re: [PATCH] xen: reuse the same pirq allocated when driver load first time
Date: Mon, 13 May 2013 12:17:14 -0400 [thread overview]
Message-ID: <20130513161714.GC10401@phenom.dumpdata.com> (raw)
In-Reply-To: <alpine.DEB.2.02.1305131540020.4799@kaball.uk.xensource.com>
On Mon, May 13, 2013 at 03:50:52PM +0100, Stefano Stabellini wrote:
> On Mon, 13 May 2013, Konrad Rzeszutek Wilk wrote:
> > On Mon, May 13, 2013 at 12:06:43PM +0100, Stefano Stabellini wrote:
> > > On Fri, 10 May 2013, Konrad Rzeszutek Wilk wrote:
> > > > On Wed, May 08, 2013 at 04:18:24PM +0800, Zhenzhong Duan wrote:
> > > > > When driver load and unload in a loop, pirq will exhaust finally.
> > > > > Try to use the same pirq which was already mapped and binded at first time
> > > >
> > > > So what happens if I unload and reload two drivers in random order?
> > > >
> > > > > when driver loaded.
> > > > >
> > > > > Read pirq from msix entry and test if data is XEN_PIRQ_MSI_DATA
> > > > > xen_irq_from_pirq(pirq) < 0 checking is wrong as irq will be freed
> > > > > when driver unload, it's always true in second load.
> > > >
> > > > If my understanding is right the issue at hand is that the caching
> > > > information about the pirq disappears once the driver has been
> > > > unloaded b/c the event's irq-info is removed (as the driver is
> > > > unloaded and free_irq is called).
> > > >
> > > > Stefano,
> > > > Is there a specific write to the MSI structure that would cause the
> > > > hypervisor to drop the PIRQ? Or a nice hypercall to "free" an
> > > > PIRQ in usage?
> > >
> > > We already have a "free PIRQ" hypercall, it's called
> > > PHYSDEVOP_unmap_pirq and should be called by QEMU.
> >
> > Considering that we call function that allocates (PHYSDEVOP_get_free_pirq)
> > it in the Linux kernel (and not in QEMU), perhaps that should be done in the
> > Linux kernel as part of xen_destroy_irq()? Or would that confuse QEMU?
>
> I think it would confuse QEMU. It is probably better to let the unmap
> being handled by it.
>
>
> > It looks like QEMU only does that hypercall (via xc_physdev_unmap_pirq)
> > unregister_real_device which is only called during pci unplug?
>
> You are right! I would think that this behaviour is erroneous unless it
> was done on purpose to avoid allocating MSIs twice.
> If that is the case we would need to do something similar in Linux too.
>
> I think that the issue is the mismatch between QEMU's and Linux's
> behaviours: either both should be allocating MSIs once, or they should
> both be allocating and deallocating MSIs every time the driver is loaded
> and unloaded.
Right. But we also have the scenario that QEMU and Linux are going to
be out of sync. So we need fixes in both places - I think.
>
>
> > > Linux should disable the MSI bit in the PCI config options of the
> > > device:
> > >
> > > pci_disable_msi -> pci_msi_shutdown -> msi_set_enable(0)
> >
> > Zhenzhong, does it do that? Looking at the driver it certainly seems that way.
> > >
> > > That should cause QEMU to issue a xc_physdev_unmap_pirq that actually
> > > unmaps the pirq. If it doesn't, it's a bug :)
> >
> > <sigh> It doesn't do that. So two bugs:
> > - QEMU doing that
> > - Linux kernel doing the hypercall as well.
>
> At first sight I also thought that the Linux kernel was issuing
> PHYSDEVOP_unmap_pirq too but actually Linux is only doing it if
> xen_initial_domain().
That seems like an easy fix. Just do 'if (xen_initial_domain()
|| xen_hvm_domain())'. I think the only one we cannot do it for
is 'xen_pv_domain()' (so PCI in PV guests) as the "owner" of the PIRQ is
actually dom0.
next prev parent reply other threads:[~2013-05-13 16:17 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-08 8:18 [PATCH] xen: reuse the same pirq allocated when driver load first time Zhenzhong Duan
2013-05-10 18:53 ` Konrad Rzeszutek Wilk
2013-05-13 7:44 ` Zhenzhong Duan
2013-05-13 11:06 ` Stefano Stabellini
2013-05-13 14:07 ` Konrad Rzeszutek Wilk
2013-05-13 14:50 ` Stefano Stabellini
2013-05-13 16:17 ` Konrad Rzeszutek Wilk [this message]
2013-05-13 17:24 ` Stefano Stabellini
2013-05-13 18:20 ` Konrad Rzeszutek Wilk
2013-05-14 13:49 ` Stefano Stabellini
2013-05-14 14:20 ` Konrad Rzeszutek Wilk
2013-05-15 9:41 ` Stefano Stabellini
2013-05-15 14:18 ` Zhenzhong Duan
2013-05-17 2:22 ` Zhenzhong Duan
2013-05-20 10:24 ` Stefano Stabellini
2013-05-20 15:24 ` Konrad Rzeszutek Wilk
2013-05-20 17:57 ` Konrad Rzeszutek Wilk
2013-05-20 20:38 ` Konrad Rzeszutek Wilk
2013-05-21 10:07 ` [Xen-devel] " David Vrabel
2013-05-21 13:40 ` Konrad Rzeszutek Wilk
2013-05-21 16:51 ` Stefano Stabellini
2013-05-21 20:42 ` Konrad Rzeszutek Wilk
2013-05-21 21:50 ` Stefano Stabellini
2013-05-21 22:41 ` Konrad Rzeszutek Wilk
2013-05-22 9:37 ` Jan Beulich
2013-05-22 15:14 ` Konrad Rzeszutek Wilk
2013-05-22 15:25 ` Jan Beulich
2013-05-22 16:41 ` Konrad Rzeszutek Wilk
2013-05-23 6:31 ` Jan Beulich
2013-05-29 17:50 ` Stefano Stabellini
2013-05-30 17:48 ` Konrad Rzeszutek Wilk
[not found] ` <51AECC3A.7060803@oracle.com>
2013-06-05 12:50 ` Stefano Stabellini
2013-06-20 2:57 ` Zhenzhong Duan
2013-06-20 14:21 ` Stefano Stabellini
2013-06-24 7:19 ` Zhenzhong Duan
2013-06-24 17:18 ` Stefano Stabellini
2013-06-25 5:33 ` DuanZhenzhong
2013-06-25 17:51 ` Stefano Stabellini
2013-06-26 4:00 ` Zhenzhong Duan
2013-06-26 18:08 ` Stefano Stabellini
2013-06-27 4:01 ` Zhenzhong Duan
2013-06-27 11:52 ` Stefano Stabellini
2013-06-28 2:33 ` Zhenzhong Duan
2013-06-28 11:12 ` Stefano Stabellini
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130513161714.GC10401@phenom.dumpdata.com \
--to=konrad.wilk@oracle.com \
--cc=chien.yen@oracle.com \
--cc=hpa@zytor.com \
--cc=joe.jin@oracle.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=stefano.stabellini@eu.citrix.com \
--cc=tglx@linutronix.de \
--cc=xen-devel@lists.xensource.com \
--cc=yuval.shaia@oracle.com \
--cc=zhenzhong.duan@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).