From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Jan Beulich" Subject: Re: [BUG] win2008 guest cannot get ip through sriov Date: Thu, 02 Jun 2016 06:08:33 -0600 Message-ID: <57503DE102000078000F0F13@prv-mh.provo.novell.com> References: <854CA60B22EFA249948B09AB135B283E047E7FED@shsmsx102.ccr.corp.intel.com> <20160527103931.GG22076@citrix.com> <574856BE02000078000EF342@prv-mh.provo.novell.com> <20160527133404.GI22076@citrix.com> <5748706B02000078000EF41E@prv-mh.provo.novell.com> <945CA011AD5F084CBEA3E851C0AB28894B8C4A64@SHSMSX101.ccr.corp.intel.com> <20160602102222.GR5160@citrix.com> <575028D702000078000F0DCE@prv-mh.provo.novell.com> <20160602110300.GU5160@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=__Part9FA995D1.2__=" Return-path: Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1b8RQU-0007Iu-Kw for xen-devel@lists.xenproject.org; Thu, 02 Jun 2016 12:08:38 +0000 In-Reply-To: <20160602110300.GU5160@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" To: Wei Liu , Quan Xu Cc: Xen-devel , Xudong Hao , PengtaoX Zhang List-Id: xen-devel@lists.xenproject.org This is a MIME message. If you are reading this text, you may want to consider changing to a mail reader or gateway that understands how to properly handle MIME multipart messages. --=__Part9FA995D1.2__= Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Disposition: inline >>> On 02.06.16 at 13:03, wrote: > On Thu, Jun 02, 2016 at 04:38:47AM -0600, Jan Beulich wrote: >> >>> On 02.06.16 at 12:22, wrote: >> > On Thu, Jun 02, 2016 at 07:31:06AM +0000, Xu, Quan wrote: >> >> On May 27, 2016 10:06 PM, Jan Beulich wrote: >> >> > >>> On 27.05.16 at 15:34, wrote: >> >> > > On Fri, May 27, 2016 at 06:16:30AM -0600, Jan Beulich wrote: >> >> > >> >>> On 27.05.16 at 12:39, wrote: >> >> > >> > Is this a regression? Does it work on previous versions of = Xen? >> >> > >> >> >> > >> I think this is what was already reported by other Intel = people, see >> >> > >> e.g. Quan's most recent reply: >> >> > >> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg0= 1896. >> >> > >> html It is not clear where the problem is, and not seeing the = issue >> >> > >> myself makes it hard to analyze. In any event this quite likely = is a >> >> > >> regression. >> >> > >> >> >> > > >> >> > > My reading of that email thread and all relevant links (including= the >> >> > > KVM bug report) is that there is a regression vf driver, but not = in Xen. >> >> >=20 >> >> > Just from reading that I would tend to agree. But the report here = is about >> >> > Win2K8. >> >>=20 >> >> Do you know which commit is a regression one? I try to find out = the=20 >> > regression commit. That may be helpful to find out the root cause. >> >>=20 >> >> Btw, some feedback from QA team, rhel 6.4 VM doesn't work, but rhel = 7.2 VM=20 > does. >> >=20 >> > Isn't this at least an indication that the guest could be buggy here? >> > It could also be both the hypervsior and guest have bugs. But we're = just >> > not sure at this point. >>=20 >> Indeed, and (with the many fixes that went in already) I really >> suspect a combination of both, or some of the involved hypervisor >> changes having unmasked some guest issue. Regardless, I'm >> afraid this ought to be treated as a blocker for the release at >> least until we understand what the issue is. But otoh making it a >> blocker probably makes sense only if we can expect progress >> (which we haven't really made for quite long a time). >>=20 >=20 > This issue is on my list, but the information gathered so far isn't > convincing enough to make it a blocker. >=20 > And yes, we need meaningful progress to make it a blocker. To make it > so, commitment from various parties is needed. Let's start with setting > out things to look at, who is going to investigate what, and a possible > timeline for each item. >=20 > Jan, can you come up with a list of what sort of information you need? Well, I had hoped to avoid that. But now that you ask for it, providing an initial debugging patch seems better than a description which may get misunderstood. Attached both a hypervisor and a qemu patch. Their plus debug key M and i output is what I'd like to start with. Jan > And then maybe Quan and Pengtao can give an estimation on how long it > takes to gather all necessary information and move on to next stage. >=20 > Wei. >=20 >> Jan >>=20 --=__Part9FA995D1.2__= Content-Type: text/plain; name="W2K8-MSI-X.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="W2K8-MSI-X.patch" --- a/xen/arch/x86/hvm/vmsi.c=0A+++ b/xen/arch/x86/hvm/vmsi.c=0A@@ -276,6 = +276,7 @@ static int msixtbl_write(struct vcpu *v,=0A if ( !entry )=0A = goto out;=0A nr_entry =3D (address - entry->gtable) / = PCI_MSIX_ENTRY_SIZE;=0A+printk("%pv: write MSI-X#%u: [%lx]=3D%0*lx\n", v, = nr_entry, address, (int)len * 2, val);//temp=0A =0A offset =3D address = & (PCI_MSIX_ENTRY_SIZE - 1);=0A if ( offset !=3D PCI_MSIX_ENTRY_VECTOR_= CTRL_OFFSET )=0A@@ -321,7 +322,17 @@ static int msixtbl_write(struct vcpu = *v,=0A =0A ASSERT(msi_desc =3D=3D desc->msi_desc);=0A =0A+{//temp=0A= + bool_t h =3D msi_desc->msi_attrib.host_masked;=0A+ bool_t g =3D = msi_desc->msi_attrib.guest_masked;=0A+ bool_t ha =3D entry->pdev->msix->hos= t_maskall;=0A+ bool_t ga =3D entry->pdev->msix->guest_maskall;=0A = guest_mask_msi_irq(desc, !!(val & PCI_MSIX_VECTOR_BITMASK));=0A+ printk("%p= v: MSI-X#%u %d(%d) / %d(%d) -> %d(%d) / %d(%d)\n",=0A+ v, nr_entry, = h, ha, g, ga,=0A+ msi_desc->msi_attrib.host_masked, entry->pdev->msi= x->host_maskall,=0A+ msi_desc->msi_attrib.guest_masked, entry->pdev-= >msix->guest_maskall);=0A+}=0A =0A unlock:=0A spin_unlock_irqrestore(&d= esc->lock, flags);=0A@@ -330,6 +341,7 @@ unlock:=0A =0A out:=0A = rcu_read_unlock(&msixtbl_rcu_lock);=0A+printk("%pv: write MSI-X [%lx] -> = %d\n", v, address, r);//temp=0A return r;=0A }=0A =0A--- a/xen/arch/x86= /msi.c=0A+++ b/xen/arch/x86/msi.c=0A@@ -438,6 +438,10 @@ static bool_t = msi_set_mask_bit(struct ir=0A if ( likely(control & PCI_MSIX_FL= AGS_ENABLE) )=0A break;=0A =0A+if(pdev->info.is_virtfn) = {//temp=0A+ printk("%04x:%02x:%02x.%o#%u: %d/%d >> %d/%d [%p]\n", seg, = bus, slot, func, entry->msi_attrib.entry_nr,=0A+ entry->msi_attrib.h= ost_masked, entry->msi_attrib.guest_masked, host, guest, __builtin_return_a= ddress(0));=0A+}=0A entry->msi_attrib.host_masked =3D host;=0A = entry->msi_attrib.guest_masked =3D guest;=0A =0A@@ -1305,6 = +1309,11 @@ int pci_msi_conf_write_intercept(struct=0A = pdev->msix->guest_maskall =3D !!(*data & PCI_MSIX_FLAGS_MASKALL);=0A = if ( pdev->msix->host_maskall )=0A *data |=3D = PCI_MSIX_FLAGS_MASKALL;=0A+if(pdev->info.is_virtfn) {//temp=0A+ printk("%04= x:%02x:%02x.%o: ctrl -> %04x (d%d:%lx,d%d)\n", seg, bus, slot, func,=0A+ = (uint16_t)*data, current->domain->domain_id, guest_cpu_user_regs()->ei= p,=0A+ pdev->domain ? pdev->domain->domain_id : -1);=0A+}=0A =0A = return 1;=0A }=0A --=__Part9FA995D1.2__= Content-Type: text/plain; name="W2K8-MSI-X-qemuu.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="W2K8-MSI-X-qemuu.patch" --- a/hw/xen/xen_pt.h=0A+++ b/hw/xen/xen_pt.h=0A@@ -10,6 +10,7 @@ void = xen_pt_log(const PCIDevice *d, cons=0A =0A #define XEN_PT_ERR(d, _f, = _a...) xen_pt_log(d, "%s: Error: "_f, __func__, ##_a)=0A =0A+#define = XEN_PT_LOGGING_ENABLED//temp=0A #ifdef XEN_PT_LOGGING_ENABLED=0A # define = XEN_PT_LOG(d, _f, _a...) xen_pt_log(d, "%s: " _f, __func__, ##_a)=0A # = define XEN_PT_WARN(d, _f, _a...) \=0A--- a/hw/xen/xen_pt_config_init.c=0A++= + b/hw/xen/xen_pt_config_init.c=0A@@ -1474,6 +1474,8 @@ static int = xen_pt_msixctrl_reg_write(Xen=0A writable_mask =3D reg->emu_mask & = ~reg->ro_mask & valid_mask;=0A *data =3D XEN_PT_MERGE_VALUE(*val, = *data, writable_mask);=0A =0A+XEN_PT_LOG(&s->dev, "MSI-X ctrl %04x = (%04x/%04x)\n",//temp=0A+ *val, *data, XEN_PT_MERGE_VALUE(*val, = dev_value, throughable_mask));//temp=0A /* create value for writing to = I/O device register */=0A *val =3D XEN_PT_MERGE_VALUE(*val, dev_value, = throughable_mask);=0A =0A@@ -2027,7 +2029,7 @@ int xen_pt_config_init(XenPC= IPassthrough=0A = reg_grp_offset,=0A = ®_grp_entry->size);=0A if (rc < 0) {=0A- = XEN_PT_LOG(&s->dev, "Failed to initialize %d/%ld, type=3D0x%x, rc:%d\n",=0A= + XEN_PT_LOG(&s->dev, "Failed to initialize %d/%zd, = type=3D0x%x, rc:%d\n",=0A i, ARRAY_SIZE(xen_pt_e= mu_reg_grps),=0A xen_pt_emu_reg_grps[i].grp_type= , rc);=0A xen_pt_config_delete(s);=0A@@ -2044,7 +2046,7 @@ = int xen_pt_config_init(XenPCIPassthrough=0A /* = initialize capability register */=0A rc =3D xen_pt_conf= ig_reg_init(s, reg_grp_entry, regs);=0A if (rc < 0) = {=0A- XEN_PT_LOG(&s->dev, "Failed to initialize = %d/%ld reg 0x%x in grp_type=3D0x%x (%d/%ld), rc=3D%d\n",=0A+ = XEN_PT_LOG(&s->dev, "Failed to initialize %d/%zd reg 0x%x in = grp_type=3D0x%x (%d/%zd), rc=3D%d\n",=0A = j, ARRAY_SIZE(xen_pt_emu_reg_grps[i].emu_regs),=0A = regs->offset, xen_pt_emu_reg_grps[i].grp_type,=0A = i, ARRAY_SIZE(xen_pt_emu_reg_grps), rc);=0A--- = a/hw/xen/xen_pt_msi.c=0A+++ b/hw/xen/xen_pt_msi.c=0A@@ -444,6 +444,7 @@ = static void pci_msix_write(void *opaque,=0A entry =3D &msix->msix_entry= [entry_nr];=0A offset =3D addr % PCI_MSIX_ENTRY_SIZE;=0A =0A+XEN_PT_LOG= (&s->dev, "[%"PRIx64"]=3D%08"PRIx64" (%u,%x)\n", addr, val, entry_nr, = offset);//temp=0A if (offset !=3D PCI_MSIX_ENTRY_VECTOR_CTRL) {=0A = if (get_entry_value(entry, offset) =3D=3D val=0A && = entry->pirq !=3D XEN_PT_UNASSIGNED_PIRQ) {=0A --=__Part9FA995D1.2__= Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KWGVuLWRldmVs IG1haWxpbmcgbGlzdApYZW4tZGV2ZWxAbGlzdHMueGVuLm9yZwpodHRwOi8vbGlzdHMueGVuLm9y Zy94ZW4tZGV2ZWwK --=__Part9FA995D1.2__=--