* PCI passthrough issue @ 2011-01-12 15:38 Jean Baptiste Favre 2011-01-12 15:43 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 15:38 UTC (permalink / raw) To: xen-devel Hello, I'm facing a strange issue with network card PCI passthrough on my openwrt test domU. - With network PCI passthrough, DNS lookup failed for some domains (exemple, google.com) but not for other (free.fr my ISP, or my domain jbfavre.org). I can ping an IP address without any problem. - Starting domU as a "normal" (ie without PCI passthrough), no problem. As far as I can say, domU is not the root cause. I really think this is related to PCI passsthrough. This seems to be related to packet length. Did not see anything strange in dom0 logs. Is there any incompatibility between 2.6.32 dom0 kernel with Xen 4.0.1 and 2.6.37 domU kernel ? Regards, JB Dom0: Debian Squeeze # uname -a Linux remus 2.6.32-5-xen-amd64 # dpkg -l|grep hyper ii xen-hypervisor-4.0-amd64 4.0.1-1 DomU: Openwrt backfire # uname -a Linux OpenWrt 2.6.37 # lsmod Module Size Used by Not tainted sky2 35744 0 All Xen related stuff is integrated into the kernel, not compiled as module. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 15:38 PCI passthrough issue Jean Baptiste Favre @ 2011-01-12 15:43 ` Konrad Rzeszutek Wilk 2011-01-12 15:53 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-12 15:43 UTC (permalink / raw) To: xen-devel On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote: > Hello, > I'm facing a strange issue with network card PCI passthrough on my > openwrt test domU. > > - With network PCI passthrough, DNS lookup failed for some domains > (exemple, google.com) but not for other (free.fr my ISP, or my domain > jbfavre.org). I can ping an IP address without any problem. Do you have "both" (so PCI passthrough and the Xen network driver) in the guest? If so, have you tried eliminating the xen network driver to see if it is just a routing issue? What does your routing table look like? Your IP table? > > - Starting domU as a "normal" (ie without PCI passthrough), no problem. > > > As far as I can say, domU is not the root cause. I really think this is > related to PCI passsthrough. This seems to be related to packet length. Then that would imply the MTU is not set right. > > Did not see anything strange in dom0 logs. > > Is there any incompatibility between 2.6.32 dom0 kernel with Xen 4.0.1 > and 2.6.37 domU kernel ? No. > > Regards, > JB > > Dom0: Debian Squeeze > # uname -a > Linux remus 2.6.32-5-xen-amd64 > # dpkg -l|grep hyper > ii xen-hypervisor-4.0-amd64 4.0.1-1 > > DomU: Openwrt backfire > # uname -a > Linux OpenWrt 2.6.37 > # lsmod > Module Size Used by Not tainted > sky2 35744 0 > All Xen related stuff is integrated into the kernel, not compiled as module. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 15:43 ` Konrad Rzeszutek Wilk @ 2011-01-12 15:53 ` Jean Baptiste Favre 2011-01-12 16:09 ` Jean Baptiste Favre 2011-01-12 16:36 ` Konrad Rzeszutek Wilk 0 siblings, 2 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 15:53 UTC (permalink / raw) To: xen-devel Hello Konrad, Le 12/01/2011 16:43, Konrad Rzeszutek Wilk a écrit : > On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote: >> Hello, >> I'm facing a strange issue with network card PCI passthrough on my >> openwrt test domU. >> >> - With network PCI passthrough, DNS lookup failed for some domains >> (exemple, google.com) but not for other (free.fr my ISP, or my domain >> jbfavre.org). I can ping an IP address without any problem. > > Do you have "both" (so PCI passthrough and the Xen network driver) > in the guest? If so, have you tried eliminating the xen network driver > to see if it is just a routing issue? Have not tried to eliminate xen network driver. Think I have both drivers. My kernel .config looks like: $ grep XEN build_dir/linux-x86_xen_domu/linux-2.6.37/.config CONFIG_XEN=y # CONFIG_XEN_PRIVILEGED_GUEST is not set CONFIG_XEN_PVHVM=y CONFIG_XEN_MAX_DOMAIN_MEMORY=128 CONFIG_XEN_SAVE_RESTORE=y CONFIG_PCI_XEN=y CONFIG_XEN_PCIDEV_FRONTEND=y CONFIG_XEN_BLKDEV_FRONTEND=y CONFIG_XEN_NETDEV_FRONTEND=y CONFIG_HVC_XEN=y CONFIG_XEN_BALLOON=y CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_DEV_EVTCHN=y CONFIG_XENFS=y CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y # CONFIG_XEN_PLATFORM_PCI is not set CONFIG_SWIOTLB_XEN=y So, I should remove CONFIG_XEN_NETDEV_FRONTEND ? > What does your routing table look like? Your IP table? My routing table is pretty clean, nothing strange here # route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 br-wan 0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 br-wan >> - Starting domU as a "normal" (ie without PCI passthrough), no problem. >> >> >> As far as I can say, domU is not the root cause. I really think this is >> related to PCI passsthrough. This seems to be related to packet length. > > Then that would imply the MTU is not set right. Already checked it: 1500 :) >> Did not see anything strange in dom0 logs. >> >> Is there any incompatibility between 2.6.32 dom0 kernel with Xen 4.0.1 >> and 2.6.37 domU kernel ? > > No. So, will try to remove Xen Network driver and see what happen. Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 15:53 ` Jean Baptiste Favre @ 2011-01-12 16:09 ` Jean Baptiste Favre 2011-01-12 16:36 ` Konrad Rzeszutek Wilk 1 sibling, 0 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 16:09 UTC (permalink / raw) To: xen-devel Hello again, Just tried without Xen network driver and got the same behaviour. Would say hopefully because removing xen network driver prevent me to use my domU as firewall for dom0 as I intented to do. BTW, have no idea left :-/ Regards, JB Le 12/01/2011 16:53, Jean Baptiste Favre a écrit : > Hello Konrad, > > Le 12/01/2011 16:43, Konrad Rzeszutek Wilk a écrit : >> On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote: >>> Hello, >>> I'm facing a strange issue with network card PCI passthrough on my >>> openwrt test domU. >>> >>> - With network PCI passthrough, DNS lookup failed for some domains >>> (exemple, google.com) but not for other (free.fr my ISP, or my domain >>> jbfavre.org). I can ping an IP address without any problem. >> >> Do you have "both" (so PCI passthrough and the Xen network driver) >> in the guest? If so, have you tried eliminating the xen network driver >> to see if it is just a routing issue? > Have not tried to eliminate xen network driver. Think I have both drivers. > > My kernel .config looks like: > $ grep XEN build_dir/linux-x86_xen_domu/linux-2.6.37/.config > CONFIG_XEN=y > # CONFIG_XEN_PRIVILEGED_GUEST is not set > CONFIG_XEN_PVHVM=y > CONFIG_XEN_MAX_DOMAIN_MEMORY=128 > CONFIG_XEN_SAVE_RESTORE=y > CONFIG_PCI_XEN=y > CONFIG_XEN_PCIDEV_FRONTEND=y > CONFIG_XEN_BLKDEV_FRONTEND=y > CONFIG_XEN_NETDEV_FRONTEND=y > CONFIG_HVC_XEN=y > CONFIG_XEN_BALLOON=y > CONFIG_XEN_SCRUB_PAGES=y > CONFIG_XEN_DEV_EVTCHN=y > CONFIG_XENFS=y > CONFIG_XEN_COMPAT_XENFS=y > CONFIG_XEN_SYS_HYPERVISOR=y > # CONFIG_XEN_PLATFORM_PCI is not set > CONFIG_SWIOTLB_XEN=y > > So, I should remove CONFIG_XEN_NETDEV_FRONTEND ? > >> What does your routing table look like? Your IP table? > My routing table is pretty clean, nothing strange here > # route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use > Iface > 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 > br-wan > 0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 > br-wan > >>> - Starting domU as a "normal" (ie without PCI passthrough), no problem. >>> >>> >>> As far as I can say, domU is not the root cause. I really think this is >>> related to PCI passsthrough. This seems to be related to packet length. >> >> Then that would imply the MTU is not set right. > Already checked it: 1500 :) > >>> Did not see anything strange in dom0 logs. >>> >>> Is there any incompatibility between 2.6.32 dom0 kernel with Xen 4.0.1 >>> and 2.6.37 domU kernel ? >> >> No. > So, will try to remove Xen Network driver and see what happen. > > Regards, > JB > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 15:53 ` Jean Baptiste Favre 2011-01-12 16:09 ` Jean Baptiste Favre @ 2011-01-12 16:36 ` Konrad Rzeszutek Wilk 2011-01-12 16:56 ` Jean Baptiste Favre 1 sibling, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-12 16:36 UTC (permalink / raw) To: xen-devel On Wed, Jan 12, 2011 at 04:53:51PM +0100, Jean Baptiste Favre wrote: > Hello Konrad, > > Le 12/01/2011 16:43, Konrad Rzeszutek Wilk a écrit : > > On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote: > >> Hello, > >> I'm facing a strange issue with network card PCI passthrough on my > >> openwrt test domU. > >> > >> - With network PCI passthrough, DNS lookup failed for some domains > >> (exemple, google.com) but not for other (free.fr my ISP, or my domain > >> jbfavre.org). I can ping an IP address without any problem. > > > > Do you have "both" (so PCI passthrough and the Xen network driver) > > in the guest? If so, have you tried eliminating the xen network driver > > to see if it is just a routing issue? > Have not tried to eliminate xen network driver. Think I have both drivers. > > My kernel .config looks like: > $ grep XEN build_dir/linux-x86_xen_domu/linux-2.6.37/.config > CONFIG_XEN=y > # CONFIG_XEN_PRIVILEGED_GUEST is not set > CONFIG_XEN_PVHVM=y > CONFIG_XEN_MAX_DOMAIN_MEMORY=128 > CONFIG_XEN_SAVE_RESTORE=y > CONFIG_PCI_XEN=y > CONFIG_XEN_PCIDEV_FRONTEND=y > CONFIG_XEN_BLKDEV_FRONTEND=y > CONFIG_XEN_NETDEV_FRONTEND=y > CONFIG_HVC_XEN=y > CONFIG_XEN_BALLOON=y > CONFIG_XEN_SCRUB_PAGES=y > CONFIG_XEN_DEV_EVTCHN=y > CONFIG_XENFS=y > CONFIG_XEN_COMPAT_XENFS=y > CONFIG_XEN_SYS_HYPERVISOR=y > # CONFIG_XEN_PLATFORM_PCI is not set > CONFIG_SWIOTLB_XEN=y > > So, I should remove CONFIG_XEN_NETDEV_FRONTEND ? No. You can just do 'ifconfig <x> down' whatever your Xen netfront NIC is. .. but. > > > What does your routing table look like? Your IP table? > My routing table is pretty clean, nothing strange here > # route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use > Iface > 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 > br-wan > 0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 > br-wan Ok, then the idea that the Xen networking driver and the PCI passthrough send packets over is not the way.. So ignore about the Xen networking part. > > >> - Starting domU as a "normal" (ie without PCI passthrough), no problem. > >> > >> > >> As far as I can say, domU is not the root cause. I really think this is > >> related to PCI passsthrough. This seems to be related to packet length. > > > > Then that would imply the MTU is not set right. > Already checked it: 1500 :) Ok. Next thing, did you try to disable the rx/tx checksumming? If you connect the Ethernet cable for this 'br-wan' device to another machine (so you could set it up as bridge and just let it pass through packets and sniff the data) what do the packets look like? What happens if the PCI passthrough device is not under the ownership of a bridge? What then? You wouldn't have any bridge firewall code in? ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 16:36 ` Konrad Rzeszutek Wilk @ 2011-01-12 16:56 ` Jean Baptiste Favre 2011-01-12 17:26 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 16:56 UTC (permalink / raw) To: xen-devel Le 12/01/2011 17:36, Konrad Rzeszutek Wilk a écrit : > On Wed, Jan 12, 2011 at 04:53:51PM +0100, Jean Baptiste Favre wrote: >> Hello Konrad, >> >> Le 12/01/2011 16:43, Konrad Rzeszutek Wilk a écrit : >>> On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote: >>>> Hello, >>>> I'm facing a strange issue with network card PCI passthrough on my >>>> openwrt test domU. >>>> >>>> - With network PCI passthrough, DNS lookup failed for some domains >>>> (exemple, google.com) but not for other (free.fr my ISP, or my domain >>>> jbfavre.org). I can ping an IP address without any problem. >>> >>> Do you have "both" (so PCI passthrough and the Xen network driver) >>> in the guest? If so, have you tried eliminating the xen network driver >>> to see if it is just a routing issue? >> Have not tried to eliminate xen network driver. Think I have both drivers. >> >> My kernel .config looks like: >> $ grep XEN build_dir/linux-x86_xen_domu/linux-2.6.37/.config >> CONFIG_XEN=y >> # CONFIG_XEN_PRIVILEGED_GUEST is not set >> CONFIG_XEN_PVHVM=y >> CONFIG_XEN_MAX_DOMAIN_MEMORY=128 >> CONFIG_XEN_SAVE_RESTORE=y >> CONFIG_PCI_XEN=y >> CONFIG_XEN_PCIDEV_FRONTEND=y >> CONFIG_XEN_BLKDEV_FRONTEND=y >> CONFIG_XEN_NETDEV_FRONTEND=y >> CONFIG_HVC_XEN=y >> CONFIG_XEN_BALLOON=y >> CONFIG_XEN_SCRUB_PAGES=y >> CONFIG_XEN_DEV_EVTCHN=y >> CONFIG_XENFS=y >> CONFIG_XEN_COMPAT_XENFS=y >> CONFIG_XEN_SYS_HYPERVISOR=y >> # CONFIG_XEN_PLATFORM_PCI is not set >> CONFIG_SWIOTLB_XEN=y >> >> So, I should remove CONFIG_XEN_NETDEV_FRONTEND ? > > No. You can just do 'ifconfig <x> down' whatever your Xen netfront > NIC is. .. but. >> >>> What does your routing table look like? Your IP table? >> My routing table is pretty clean, nothing strange here >> # route -n >> Kernel IP routing table >> Destination Gateway Genmask Flags Metric Ref Use >> Iface >> 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 >> br-wan >> 0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 >> br-wan > > Ok, then the idea that the Xen networking driver and the PCI passthrough send > packets over is not the way.. > > So ignore about the Xen networking part. OK. >> >>>> - Starting domU as a "normal" (ie without PCI passthrough), no problem. >>>> >>>> >>>> As far as I can say, domU is not the root cause. I really think this is >>>> related to PCI passsthrough. This seems to be related to packet length. >>> >>> Then that would imply the MTU is not set right. >> Already checked it: 1500 :) > Ok. > > Next thing, did you try to disable the rx/tx checksumming? No. Not sure I even know how to do it, but will have a look on that. > If you connect the Ethernet cable for this 'br-wan' device to another machine > (so you could set it up as bridge and just let it pass through packets and sniff > the data) what do the packets look like? I can see packets on my gateway (which acts as DNS as well). They looks good. I can see DNS answers as well leaving the gateway, but not reaching my domU. Have only basic switches between my gateway and my domU. > What happens if the PCI passthrough device is not under the ownership of a bridge? > What then? You wouldn't have any bridge firewall code in? Removed bridge + configure eth: no change. Checked ebtables/iptables (all tables: net filter and mangle) rules: empty, Policy to ACCEPT. Tried to enable/disable ip_forwarding: no change Also disabled IPV6 support (I saw some DNS answers as AAAA but no IPv6 available at home for now): no change Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 16:56 ` Jean Baptiste Favre @ 2011-01-12 17:26 ` Konrad Rzeszutek Wilk 2011-01-12 17:44 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-12 17:26 UTC (permalink / raw) To: xen-devel On Wed, Jan 12, 2011 at 05:56:45PM +0100, Jean Baptiste Favre wrote: > Le 12/01/2011 17:36, Konrad Rzeszutek Wilk a écrit : > > On Wed, Jan 12, 2011 at 04:53:51PM +0100, Jean Baptiste Favre wrote: > >> Hello Konrad, > >> > >> Le 12/01/2011 16:43, Konrad Rzeszutek Wilk a écrit : > >>> On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote: > >>>> Hello, > >>>> I'm facing a strange issue with network card PCI passthrough on my > >>>> openwrt test domU. > >>>> > >>>> - With network PCI passthrough, DNS lookup failed for some domains > >>>> (exemple, google.com) but not for other (free.fr my ISP, or my domain > >>>> jbfavre.org). I can ping an IP address without any problem. > >>> > >>> Do you have "both" (so PCI passthrough and the Xen network driver) > >>> in the guest? If so, have you tried eliminating the xen network driver > >>> to see if it is just a routing issue? > >> Have not tried to eliminate xen network driver. Think I have both drivers. > >> > >> My kernel .config looks like: > >> $ grep XEN build_dir/linux-x86_xen_domu/linux-2.6.37/.config > >> CONFIG_XEN=y > >> # CONFIG_XEN_PRIVILEGED_GUEST is not set > >> CONFIG_XEN_PVHVM=y > >> CONFIG_XEN_MAX_DOMAIN_MEMORY=128 > >> CONFIG_XEN_SAVE_RESTORE=y > >> CONFIG_PCI_XEN=y > >> CONFIG_XEN_PCIDEV_FRONTEND=y > >> CONFIG_XEN_BLKDEV_FRONTEND=y > >> CONFIG_XEN_NETDEV_FRONTEND=y > >> CONFIG_HVC_XEN=y > >> CONFIG_XEN_BALLOON=y > >> CONFIG_XEN_SCRUB_PAGES=y > >> CONFIG_XEN_DEV_EVTCHN=y > >> CONFIG_XENFS=y > >> CONFIG_XEN_COMPAT_XENFS=y > >> CONFIG_XEN_SYS_HYPERVISOR=y > >> # CONFIG_XEN_PLATFORM_PCI is not set > >> CONFIG_SWIOTLB_XEN=y > >> > >> So, I should remove CONFIG_XEN_NETDEV_FRONTEND ? > > > > No. You can just do 'ifconfig <x> down' whatever your Xen netfront > > NIC is. .. but. > >> > >>> What does your routing table look like? Your IP table? > >> My routing table is pretty clean, nothing strange here > >> # route -n > >> Kernel IP routing table > >> Destination Gateway Genmask Flags Metric Ref Use > >> Iface > >> 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 > >> br-wan > >> 0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 > >> br-wan > > > > Ok, then the idea that the Xen networking driver and the PCI passthrough send > > packets over is not the way.. > > > > So ignore about the Xen networking part. > > OK. > > >> > >>>> - Starting domU as a "normal" (ie without PCI passthrough), no problem. > >>>> > >>>> > >>>> As far as I can say, domU is not the root cause. I really think this is > >>>> related to PCI passsthrough. This seems to be related to packet length. > >>> > >>> Then that would imply the MTU is not set right. > >> Already checked it: 1500 :) > > Ok. > > > > Next thing, did you try to disable the rx/tx checksumming? > > No. Not sure I even know how to do it, but will have a look on that. > > > If you connect the Ethernet cable for this 'br-wan' device to another machine > > (so you could set it up as bridge and just let it pass through packets and sniff > > the data) what do the packets look like? > > I can see packets on my gateway (which acts as DNS as well). They looks > good. I can see DNS answers as well leaving the gateway, but not > reaching my domU. Have only basic switches between my gateway and my domU. Wait a minute. How are your domU's connected? You did turn on forwarding in your guest with the PCI passthrough right? How is your domain (with the PCI passthrought) connected to the other domains? > > > What happens if the PCI passthrough device is not under the ownership of a bridge? > > What then? You wouldn't have any bridge firewall code in? > > Removed bridge + configure eth: no change. > Checked ebtables/iptables (all tables: net filter and mangle) rules: > empty, Policy to ACCEPT. > Tried to enable/disable ip_forwarding: no change > Also disabled IPV6 support (I saw some DNS answers as AAAA but no IPv6 > available at home for now): no change > > Regards, > JB > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 17:26 ` Konrad Rzeszutek Wilk @ 2011-01-12 17:44 ` Jean Baptiste Favre 2011-01-12 18:32 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 17:44 UTC (permalink / raw) To: xen-devel Le 12/01/2011 18:26, Konrad Rzeszutek Wilk a écrit : > On Wed, Jan 12, 2011 at 05:56:45PM +0100, Jean Baptiste Favre wrote: >> Le 12/01/2011 17:36, Konrad Rzeszutek Wilk a écrit : >>> On Wed, Jan 12, 2011 at 04:53:51PM +0100, Jean Baptiste Favre wrote: >>>> Hello Konrad, >>>> >>>> Le 12/01/2011 16:43, Konrad Rzeszutek Wilk a écrit : >>>>> On Wed, Jan 12, 2011 at 04:38:49PM +0100, Jean Baptiste Favre wrote: >>>>>> Hello, >>>>>> I'm facing a strange issue with network card PCI passthrough on my >>>>>> openwrt test domU. >>>>>> >>>>>> - With network PCI passthrough, DNS lookup failed for some domains >>>>>> (exemple, google.com) but not for other (free.fr my ISP, or my domain >>>>>> jbfavre.org). I can ping an IP address without any problem. >>>>> >>>>> Do you have "both" (so PCI passthrough and the Xen network driver) >>>>> in the guest? If so, have you tried eliminating the xen network driver >>>>> to see if it is just a routing issue? >>>> Have not tried to eliminate xen network driver. Think I have both drivers. >>>> >>>> My kernel .config looks like: >>>> $ grep XEN build_dir/linux-x86_xen_domu/linux-2.6.37/.config >>>> CONFIG_XEN=y >>>> # CONFIG_XEN_PRIVILEGED_GUEST is not set >>>> CONFIG_XEN_PVHVM=y >>>> CONFIG_XEN_MAX_DOMAIN_MEMORY=128 >>>> CONFIG_XEN_SAVE_RESTORE=y >>>> CONFIG_PCI_XEN=y >>>> CONFIG_XEN_PCIDEV_FRONTEND=y >>>> CONFIG_XEN_BLKDEV_FRONTEND=y >>>> CONFIG_XEN_NETDEV_FRONTEND=y >>>> CONFIG_HVC_XEN=y >>>> CONFIG_XEN_BALLOON=y >>>> CONFIG_XEN_SCRUB_PAGES=y >>>> CONFIG_XEN_DEV_EVTCHN=y >>>> CONFIG_XENFS=y >>>> CONFIG_XEN_COMPAT_XENFS=y >>>> CONFIG_XEN_SYS_HYPERVISOR=y >>>> # CONFIG_XEN_PLATFORM_PCI is not set >>>> CONFIG_SWIOTLB_XEN=y >>>> >>>> So, I should remove CONFIG_XEN_NETDEV_FRONTEND ? >>> >>> No. You can just do 'ifconfig <x> down' whatever your Xen netfront >>> NIC is. .. but. >>>> >>>>> What does your routing table look like? Your IP table? >>>> My routing table is pretty clean, nothing strange here >>>> # route -n >>>> Kernel IP routing table >>>> Destination Gateway Genmask Flags Metric Ref Use >>>> Iface >>>> 10.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 >>>> br-wan >>>> 0.0.0.0 10.0.0.1 0.0.0.0 UG 0 0 0 >>>> br-wan >>> >>> Ok, then the idea that the Xen networking driver and the PCI passthrough send >>> packets over is not the way.. >>> >>> So ignore about the Xen networking part. >> >> OK. >> >>>> >>>>>> - Starting domU as a "normal" (ie without PCI passthrough), no problem. >>>>>> >>>>>> >>>>>> As far as I can say, domU is not the root cause. I really think this is >>>>>> related to PCI passsthrough. This seems to be related to packet length. >>>>> >>>>> Then that would imply the MTU is not set right. >>>> Already checked it: 1500 :) >>> Ok. >>> >>> Next thing, did you try to disable the rx/tx checksumming? >> >> No. Not sure I even know how to do it, but will have a look on that. >> >>> If you connect the Ethernet cable for this 'br-wan' device to another machine >>> (so you could set it up as bridge and just let it pass through packets and sniff >>> the data) what do the packets look like? >> >> I can see packets on my gateway (which acts as DNS as well). They looks >> good. I can see DNS answers as well leaving the gateway, but not >> reaching my domU. Have only basic switches between my gateway and my domU. > > Wait a minute. How are your domU's connected? You did turn on > forwarding in your guest with the PCI passthrough right? My OpenWRT domU with PCI passthrough is directly connected to my LAN through network card > How is your domain (with the PCI passthrought) connected to the > other domains? For the moment, other domU are not connected in any way to my OpenWRT domU. I want first to make sur PCI passthrough is working before trying more complex architecture. My OpenWRT domU network setup is: dom0 domU LAN vif ----- eth0 eth1 ----- SW ----- GW ----- NET |__ br __| All tests are done from OpenWRT domU for now (but I guess this won't work either from dom0). I also tried following setup: domU LAN eth1 ----- SW ----- GW ----- NET br __| That means no vif from dom0. But with PCI passthrough, I should reach network directly, right ? Finally, I think that my setup shall not be responsible for the fact that ping works, small DNS answers are received but not bigger ones (limit seems to be around 100 bytes). >>> What happens if the PCI passthrough device is not under the ownership of a bridge? >>> What then? You wouldn't have any bridge firewall code in? >> >> Removed bridge + configure eth: no change. >> Checked ebtables/iptables (all tables: net filter and mangle) rules: >> empty, Policy to ACCEPT. >> Tried to enable/disable ip_forwarding: no change >> Also disabled IPV6 support (I saw some DNS answers as AAAA but no IPv6 >> available at home for now): no change >> >> Regards, >> JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 17:44 ` Jean Baptiste Favre @ 2011-01-12 18:32 ` Konrad Rzeszutek Wilk 2011-01-12 20:07 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-12 18:32 UTC (permalink / raw) To: xen-devel > > Finally, I think that my setup shall not be responsible for the fact > that ping works, small DNS answers are received but not bigger ones Aah, somehow I missed that fact. > (limit seems to be around 100 bytes). What does wireshark or tcpdump tell you when you send the larger packets? ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 18:32 ` Konrad Rzeszutek Wilk @ 2011-01-12 20:07 ` Jean Baptiste Favre 2011-01-12 21:40 ` Konrad Rzeszutek Wilk 2011-01-12 21:44 ` Jean Baptiste Favre 0 siblings, 2 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 20:07 UTC (permalink / raw) To: xen-devel Le 12/01/2011 19:32, Konrad Rzeszutek Wilk a écrit : >> >> Finally, I think that my setup shall not be responsible for the fact >> that ping works, small DNS answers are received but not bigger ones > > Aah, somehow I missed that fact. Seems yes :) >> (limit seems to be around 100 bytes). > > What does wireshark or tcpdump tell you when you send the larger packets? That's one of my problem: nothing special :( I can only run tcpdump on my gateway (no SSH available on my OpenWRT domU and only xm console access is available). I see DNS requests comming in, I see DNS answers going out. ARP requests are fine. As an example: dig free.fr works dig jbfavre.org works dig amazon.fr works dig amazon.com works (quite long, but works) dig google.fr fails dig google.com fails >From another machine in the same LAN, all requests work fine. Thought this was due to some ARP restriction setted up by default in OpenWRT, but in this case, I can not explain why some DNS lookup do work and some don't. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 20:07 ` Jean Baptiste Favre @ 2011-01-12 21:40 ` Konrad Rzeszutek Wilk 2011-01-12 21:46 ` Jean Baptiste Favre 2011-01-12 21:44 ` Jean Baptiste Favre 1 sibling, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-12 21:40 UTC (permalink / raw) To: xen-devel On Wed, Jan 12, 2011 at 09:07:30PM +0100, Jean Baptiste Favre wrote: > Le 12/01/2011 19:32, Konrad Rzeszutek Wilk a écrit : > >> > >> Finally, I think that my setup shall not be responsible for the fact > >> that ping works, small DNS answers are received but not bigger ones > > > > Aah, somehow I missed that fact. > > Seems yes :) > > >> (limit seems to be around 100 bytes). > > > > What does wireshark or tcpdump tell you when you send the larger packets? > > That's one of my problem: nothing special :( > > I can only run tcpdump on my gateway (no SSH available on my OpenWRT > domU and only xm console access is available). You can install (or just compile it statically) tcpdump on your OpenWRT, or just hook it up to a beffier guest.. > I see DNS requests comming in, I see DNS answers going out. > ARP requests are fine. > > As an example: > dig free.fr works > dig jbfavre.org works > dig amazon.fr works > dig amazon.com works (quite long, but works) > dig google.fr fails > dig google.com fails Fails as in response or fails halfway? > > >From another machine in the same LAN, all requests work fine. > > Thought this was due to some ARP restriction setted up by default in > OpenWRT, but in this case, I can not explain why some DNS lookup do work > and some don't. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 21:40 ` Konrad Rzeszutek Wilk @ 2011-01-12 21:46 ` Jean Baptiste Favre 2011-01-12 22:18 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 21:46 UTC (permalink / raw) To: xen-devel Le 12/01/2011 22:40, Konrad Rzeszutek Wilk a écrit : > On Wed, Jan 12, 2011 at 09:07:30PM +0100, Jean Baptiste Favre wrote: >> Le 12/01/2011 19:32, Konrad Rzeszutek Wilk a écrit : >>>> >>>> Finally, I think that my setup shall not be responsible for the fact >>>> that ping works, small DNS answers are received but not bigger ones >>> >>> Aah, somehow I missed that fact. >> >> Seems yes :) >> >>>> (limit seems to be around 100 bytes). >>> >>> What does wireshark or tcpdump tell you when you send the larger packets? >> >> That's one of my problem: nothing special :( >> >> I can only run tcpdump on my gateway (no SSH available on my OpenWRT >> domU and only xm console access is available). > > You can install (or just compile it statically) tcpdump on your OpenWRT, > or just hook it up to a beffier guest.. tcpdump is available on my OpenWRT domU, but I can not perform tcpdump and DNS lookup with only console access and AFAIK screen is not available on OpenWRT >> I see DNS requests comming in, I see DNS answers going out. >> ARP requests are fine. >> >> As an example: >> dig free.fr works >> dig jbfavre.org works >> dig amazon.fr works >> dig amazon.com works (quite long, but works) >> dig google.fr fails >> dig google.com fails > > Fails as in response or fails halfway? Fails as in: # dig google.com ; <<>> DiG 9.7.2-P3 <<>> google.com ;; global options: +cmd ;; connection timed out; no servers could be reached >> >> >From another machine in the same LAN, all requests work fine. >> >> Thought this was due to some ARP restriction setted up by default in >> OpenWRT, but in this case, I can not explain why some DNS lookup do work >> and some don't. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 21:46 ` Jean Baptiste Favre @ 2011-01-12 22:18 ` Jean Baptiste Favre 2011-01-13 11:28 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 22:18 UTC (permalink / raw) To: xen-devel Ok, my bad. Screen IS available on OpenWRT. I've added it to my disk image and now have a tcpdump capture of network traffic of my DomU Have the same network capture from my GW so I 'll be able to investigate deeper. Let me know if you have specific points you want me to look at. Regards, JB Le 12/01/2011 22:46, Jean Baptiste Favre a écrit : > Le 12/01/2011 22:40, Konrad Rzeszutek Wilk a écrit : >> On Wed, Jan 12, 2011 at 09:07:30PM +0100, Jean Baptiste Favre wrote: >>> Le 12/01/2011 19:32, Konrad Rzeszutek Wilk a écrit : >>>>> >>>>> Finally, I think that my setup shall not be responsible for the fact >>>>> that ping works, small DNS answers are received but not bigger ones >>>> >>>> Aah, somehow I missed that fact. >>> >>> Seems yes :) >>> >>>>> (limit seems to be around 100 bytes). >>>> >>>> What does wireshark or tcpdump tell you when you send the larger packets? >>> >>> That's one of my problem: nothing special :( >>> >>> I can only run tcpdump on my gateway (no SSH available on my OpenWRT >>> domU and only xm console access is available). >> >> You can install (or just compile it statically) tcpdump on your OpenWRT, >> or just hook it up to a beffier guest.. > > tcpdump is available on my OpenWRT domU, but I can not perform tcpdump > and DNS lookup with only console access and AFAIK screen is not > available on OpenWRT > >>> I see DNS requests comming in, I see DNS answers going out. >>> ARP requests are fine. >>> >>> As an example: >>> dig free.fr works >>> dig jbfavre.org works >>> dig amazon.fr works >>> dig amazon.com works (quite long, but works) >>> dig google.fr fails >>> dig google.com fails >> >> Fails as in response or fails halfway? > > Fails as in: > # dig google.com > > ; <<>> DiG 9.7.2-P3 <<>> google.com > ;; global options: +cmd > ;; connection timed out; no servers could be reached > >>> >>> >From another machine in the same LAN, all requests work fine. >>> >>> Thought this was due to some ARP restriction setted up by default in >>> OpenWRT, but in this case, I can not explain why some DNS lookup do work >>> and some don't. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 22:18 ` Jean Baptiste Favre @ 2011-01-13 11:28 ` Jean Baptiste Favre 2011-01-13 19:18 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-13 11:28 UTC (permalink / raw) To: xen-devel Have had a look on both tcpdump captures. 1. From the OpenWRT domU: nothing special. Lot of STP traffic due to my network architecture. Don't thing this could be the cause of my problem since I'm always able to reproduce the failure whatever could be the order of DNS requests. Don't see "big" DNS answers coming in, see multiple requests before timeout. 2. From my gateway: nothing special here too. Same STP traffic (have checked GW network setup as well). See each DNS requests my domU sent and can see as well answers, which are not received by domU. No strange ARP traffic. I wanted to change the network, thinking about a bogus driver (never know). My Dom0 has crashed during reboot and I've no way to remotely reboot it (no remote power control and it seems that Magic SysRQ are not transfered through my Putty/screen connection). Will reboot it when at home and continue investigation with another network card and maybe another operating system to check if bad behaviour is still there. Regards, JB Le 12/01/2011 23:18, Jean Baptiste Favre a écrit : > Ok, my bad. Screen IS available on OpenWRT. > I've added it to my disk image and now have a tcpdump capture of network > traffic of my DomU > > Have the same network capture from my GW so I 'll be able to investigate > deeper. > > Let me know if you have specific points you want me to look at. > > Regards, > JB > > > Le 12/01/2011 22:46, Jean Baptiste Favre a écrit : >> Le 12/01/2011 22:40, Konrad Rzeszutek Wilk a écrit : >>> On Wed, Jan 12, 2011 at 09:07:30PM +0100, Jean Baptiste Favre wrote: >>>> Le 12/01/2011 19:32, Konrad Rzeszutek Wilk a écrit : >>>>>> >>>>>> Finally, I think that my setup shall not be responsible for the fact >>>>>> that ping works, small DNS answers are received but not bigger ones >>>>> >>>>> Aah, somehow I missed that fact. >>>> >>>> Seems yes :) >>>> >>>>>> (limit seems to be around 100 bytes). >>>>> >>>>> What does wireshark or tcpdump tell you when you send the larger packets? >>>> >>>> That's one of my problem: nothing special :( >>>> >>>> I can only run tcpdump on my gateway (no SSH available on my OpenWRT >>>> domU and only xm console access is available). >>> >>> You can install (or just compile it statically) tcpdump on your OpenWRT, >>> or just hook it up to a beffier guest.. >> >> tcpdump is available on my OpenWRT domU, but I can not perform tcpdump >> and DNS lookup with only console access and AFAIK screen is not >> available on OpenWRT >> >>>> I see DNS requests comming in, I see DNS answers going out. >>>> ARP requests are fine. >>>> >>>> As an example: >>>> dig free.fr works >>>> dig jbfavre.org works >>>> dig amazon.fr works >>>> dig amazon.com works (quite long, but works) >>>> dig google.fr fails >>>> dig google.com fails >>> >>> Fails as in response or fails halfway? >> >> Fails as in: >> # dig google.com >> >> ; <<>> DiG 9.7.2-P3 <<>> google.com >> ;; global options: +cmd >> ;; connection timed out; no servers could be reached >> >>>> >>>> >From another machine in the same LAN, all requests work fine. >>>> >>>> Thought this was due to some ARP restriction setted up by default in >>>> OpenWRT, but in this case, I can not explain why some DNS lookup do work >>>> and some don't. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-13 11:28 ` Jean Baptiste Favre @ 2011-01-13 19:18 ` Jean Baptiste Favre 2011-01-13 20:19 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-13 19:18 UTC (permalink / raw) To: xen-devel Hello, My dom0 is back and I performed some more tests. I told in my first mail that ping works. Indeed it works, but not always: # ping -c2 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: seq=0 ttl=64 time=0.846 ms 64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.824 ms --- 10.0.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.824/0.835/0.846 ms # ping -c2 -s60 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 60 data bytes 68 bytes from 10.0.0.1: seq=0 ttl=64 time=0.819 ms 68 bytes from 10.0.0.1: seq=1 ttl=64 time=0.807 ms --- 10.0.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.807/0.813/0.819 ms Increasing packet size is ok until this one: # ping -c2 -s85 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 85 data bytes 93 bytes from 10.0.0.1: seq=0 ttl=64 time=0.823 ms 93 bytes from 10.0.0.1: seq=1 ttl=64 time=0.816 ms --- 10.0.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.816/0.819/0.823 ms root@OpenWrt:/# ping -c2 -s86 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 86 data bytes --- 10.0.0.1 ping statistics --- 2 packets transmitted, 0 packets received, 100% packet loss As you see, packet size seems to be limited in a way. From another machine on the same LAN I can do ping -s1500 without any problem. So I think I hit a bug. Either it's Xen related, or Debian (through debian kernel version). Now the question is: how can I determine which part is responsible ? Regards, JB Le 13/01/2011 12:28, Jean Baptiste Favre a écrit : > Have had a look on both tcpdump captures. > 1. From the OpenWRT domU: nothing special. Lot of STP traffic due to my > network architecture. Don't thing this could be the cause of my problem > since I'm always able to reproduce the failure whatever could be the > order of DNS requests. Don't see "big" DNS answers coming in, see > multiple requests before timeout. > > 2. From my gateway: nothing special here too. Same STP traffic (have > checked GW network setup as well). See each DNS requests my domU sent > and can see as well answers, which are not received by domU. No strange > ARP traffic. > > I wanted to change the network, thinking about a bogus driver (never > know). My Dom0 has crashed during reboot and I've no way to remotely > reboot it (no remote power control and it seems that Magic SysRQ are not > transfered through my Putty/screen connection). > > Will reboot it when at home and continue investigation with another > network card and maybe another operating system to check if bad > behaviour is still there. > > Regards, > JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-13 19:18 ` Jean Baptiste Favre @ 2011-01-13 20:19 ` Konrad Rzeszutek Wilk 2011-01-13 20:44 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-13 20:19 UTC (permalink / raw) To: xen-devel On Thu, Jan 13, 2011 at 08:18:33PM +0100, Jean Baptiste Favre wrote: > Hello, > My dom0 is back and I performed some more tests. > > I told in my first mail that ping works. Indeed it works, but not always: > # ping -c2 10.0.0.1 > PING 10.0.0.1 (10.0.0.1): 56 data bytes > 64 bytes from 10.0.0.1: seq=0 ttl=64 time=0.846 ms > 64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.824 ms > > --- 10.0.0.1 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max = 0.824/0.835/0.846 ms > # ping -c2 -s60 10.0.0.1 > PING 10.0.0.1 (10.0.0.1): 60 data bytes > 68 bytes from 10.0.0.1: seq=0 ttl=64 time=0.819 ms > 68 bytes from 10.0.0.1: seq=1 ttl=64 time=0.807 ms > > --- 10.0.0.1 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max = 0.807/0.813/0.819 ms > > Increasing packet size is ok until this one: > # ping -c2 -s85 10.0.0.1 > PING 10.0.0.1 (10.0.0.1): 85 data bytes > 93 bytes from 10.0.0.1: seq=0 ttl=64 time=0.823 ms > 93 bytes from 10.0.0.1: seq=1 ttl=64 time=0.816 ms > > --- 10.0.0.1 ping statistics --- > 2 packets transmitted, 2 packets received, 0% packet loss > round-trip min/avg/max = 0.816/0.819/0.823 ms > root@OpenWrt:/# ping -c2 -s86 10.0.0.1 > PING 10.0.0.1 (10.0.0.1): 86 data bytes > > --- 10.0.0.1 ping statistics --- > 2 packets transmitted, 0 packets received, 100% packet loss > > As you see, packet size seems to be limited in a way. From another > machine on the same LAN I can do ping -s1500 without any problem. One thing that I just thought (which I keep on forgetting to do). You did set 'iommu=soft' in your Linux guest, right? > > So I think I hit a bug. Either it's Xen related, or Debian (through > debian kernel version). Now the question is: how can I determine which > part is responsible ? What does tcpdump tell you when you try to send it at -s86? ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-13 20:19 ` Konrad Rzeszutek Wilk @ 2011-01-13 20:44 ` Jean Baptiste Favre 2011-01-14 14:53 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-13 20:44 UTC (permalink / raw) To: xen-devel Le 13/01/2011 21:19, Konrad Rzeszutek Wilk a écrit : > On Thu, Jan 13, 2011 at 08:18:33PM +0100, Jean Baptiste Favre wrote: >> Hello, >> My dom0 is back and I performed some more tests. >> >> I told in my first mail that ping works. Indeed it works, but not always: >> # ping -c2 10.0.0.1 >> PING 10.0.0.1 (10.0.0.1): 56 data bytes >> 64 bytes from 10.0.0.1: seq=0 ttl=64 time=0.846 ms >> 64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.824 ms >> >> --- 10.0.0.1 ping statistics --- >> 2 packets transmitted, 2 packets received, 0% packet loss >> round-trip min/avg/max = 0.824/0.835/0.846 ms >> # ping -c2 -s60 10.0.0.1 >> PING 10.0.0.1 (10.0.0.1): 60 data bytes >> 68 bytes from 10.0.0.1: seq=0 ttl=64 time=0.819 ms >> 68 bytes from 10.0.0.1: seq=1 ttl=64 time=0.807 ms >> >> --- 10.0.0.1 ping statistics --- >> 2 packets transmitted, 2 packets received, 0% packet loss >> round-trip min/avg/max = 0.807/0.813/0.819 ms >> >> Increasing packet size is ok until this one: >> # ping -c2 -s85 10.0.0.1 >> PING 10.0.0.1 (10.0.0.1): 85 data bytes >> 93 bytes from 10.0.0.1: seq=0 ttl=64 time=0.823 ms >> 93 bytes from 10.0.0.1: seq=1 ttl=64 time=0.816 ms >> >> --- 10.0.0.1 ping statistics --- >> 2 packets transmitted, 2 packets received, 0% packet loss >> round-trip min/avg/max = 0.816/0.819/0.823 ms >> root@OpenWrt:/# ping -c2 -s86 10.0.0.1 >> PING 10.0.0.1 (10.0.0.1): 86 data bytes >> >> --- 10.0.0.1 ping statistics --- >> 2 packets transmitted, 0 packets received, 100% packet loss >> >> As you see, packet size seems to be limited in a way. From another >> machine on the same LAN I can do ping -s1500 without any problem. > > One thing that I just thought (which I keep on forgetting to do). > You did set 'iommu=soft' in your Linux guest, right? Tought I told it in my previou smails. Sorry to missed it: On my dom0: $ cat /proc/cmdline placeholder root=/dev/mapper/system-root ro console=hvc0 earlyprintk=xen nomodeset xen-pciback.permissive xen-pciback.hide=(04:00.0) pci=resource_alignment=04:00.0 quiet Xen hypervisor options are: # xm dmesg (XEN) Xen version 4.0.1 (Debian 4.0.1-1) (waldi@debian.org) (gcc version 4.4.5 20100824 (prerelease) (Debian 4.4.4-11) ) Fri Sep 3 15:38:12 UTC 2010 (XEN) Bootloader: GRUB 1.98+20100804-11 (XEN) Command line: placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all guest_loglvl=all com1=115200,8n1 console=com1 OpenWRT domU config file is: $ cat /etc/xen/auto/openwrt.cfg kernel = '/home/domU/wrt/openwrt-x86-xen_domu-vmlinuz' root = '/dev/xvda2 rw' memory = '256' vcpus = '1' cpus = '1' localtime = 0 serial = 'pty' disk = ['file:/home/domU/wrt/openwrt-x86-xen_domu-combined-ext4.img,xvda,w'] #vif = [ 'bridge=br-wan, mac=00:16:3E:01:00:64, vifname=wrt-100.eth1' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' extra = "iommu=soft swiotlb=force console=hvc0 xencons=tty" pci = [ '04:00.0' ] name = 'openwrt' hostname = 'openwrt.clichy.jbfavre.org' >> So I think I hit a bug. Either it's Xen related, or Debian (through >> debian kernel version). Now the question is: how can I determine which >> part is responsible ? > > What does tcpdump tell you when you try to send it at -s86? I can see echo requests coming in on my gateway, replies going back but replies are never received on the domU. Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-13 20:44 ` Jean Baptiste Favre @ 2011-01-14 14:53 ` Konrad Rzeszutek Wilk 2011-01-14 23:29 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-14 14:53 UTC (permalink / raw) To: xen-devel > Tought I told it in my previou smails. Sorry to missed it: You probably did and I missed it too. Good that you have those options, now: .. snip .. > > What does tcpdump tell you when you try to send it at -s86? > > I can see echo requests coming in on my gateway, replies going back but > replies are never received on the domU. What is the NIC you are using? Is it a Intel one? What does /proc/interrupts look like? Can you send me the full domU output? What does the Xen hypervisor mapping look like (xm debug-keys q, xm debug-keys Q, xm debug-keys i)? What is the motherboard you have? Did you look up to see if there are any errate for the motherboard or the NIC you are using? Do you have the latest firmware for the NIC and the latest BIOS for your motherboard? Is the OpenWRT kernel you are using one that you built yourself or do they package it for you? Can you try using a vaniall built one (2.6.37 vanilla is perfect). Is this a 64-bit kernel or 32-bit? Can you try using a 64-bit one (you should be able to run a 64-bit kernel alongside 32-bit userspace). ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-14 14:53 ` Konrad Rzeszutek Wilk @ 2011-01-14 23:29 ` Jean Baptiste Favre 2011-01-17 8:59 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-14 23:29 UTC (permalink / raw) To: xen-devel [-- Attachment #1: Type: text/plain, Size: 3296 bytes --] Hello, Le 14/01/2011 15:53, Konrad Rzeszutek Wilk a écrit : >> Tought I told it in my previou smails. Sorry to missed it: > > You probably did and I missed it too. Good that you have those options, now: > > .. snip .. >>> What does tcpdump tell you when you try to send it at -s86? >> >> I can see echo requests coming in on my gateway, replies going back but >> replies are never received on the domU. > > What is the NIC you are using? Is it a Intel one? What does /proc/interrupts look like? The NIC I use is "Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller", not Intel, with sky2 driver. # cat /proc/interrupts CPU0 55: 1393 xen-pirq-pcifront-msi sky2@pci:0000:00:00.0 246: 202 xen-dyn-event blkif 247: 103 xen-dyn-event hvc_console 248: 603 xen-dyn-event pcifront 249: 259 xen-dyn-event xenbus 250: 0 xen-percpu-ipi callfuncsingle0 251: 0 xen-percpu-virq debug0 252: 0 xen-percpu-ipi callfunc0 253: 0 xen-percpu-ipi resched0 254: 0 xen-percpu-ipi spinlock0 255: 246058 xen-percpu-virq timer0 NMI: 0 Non-maskable interrupts LOC: 0 Local timer interrupts SPU: 0 Spurious interrupts PMI: 0 Performance monitoring interrupts IWI: 0 IRQ work interrupts RES: 0 Rescheduling interrupts CAL: 0 Function call interrupts TLB: 0 TLB shootdowns TRM: 0 Thermal event interrupts THR: 0 Threshold APIC interrupts MCE: 0 Machine check exceptions MCP: 0 Machine check polls ERR: 0 MIS: 0 > Can you send me the full domU output? What does the Xen hypervisor mapping look like Output are attached to this mail. > (xm debug-keys q, xm debug-keys Q, xm debug-keys i)? What is the motherboard you have? dmidecode ... Handle 0x0002, DMI type 2, 15 bytes Base Board Information Manufacturer: ASUSTeK Computer INC. Product Name: M3N18L T-M3N8200 Version: Rev x.xx Serial Number: MS1C85B07000633 Asset Tag: To Be Filled By O.E.M. Features: Board is a hosting board Board is replaceable Location In Chassis: To Be Filled By O.E.M. Chassis Handle: 0x0003 Type: Motherboard Contained Object Handles: 0 ... > Did you look up to see if there are any errate for the motherboard or the NIC you are using? No, since everything was working fine (before I decided to test PCI passthrough). > Do you have the latest firmware for the NIC and the latest BIOS for your motherboard? Not sure. Will check > Is the OpenWRT kernel you are using one that you built yourself or do they package it > for you? Can you try using a vaniall built one (2.6.37 vanilla is perfect). It is packaged by openWRT Builroot. Just have to set the kernel version I want to use. Will try a vanilla kernel as soon as possible. > Is this a 64-bit kernel or 32-bit? Can you try using a 64-bit one (you should be > able to run a 64-bit kernel alongside 32-bit userspace). OK, will try it as well. Regards, JB [-- Attachment #2: xen_debug_keys_i.txt --] [-- Type: text/plain, Size: 8332 bytes --] (XEN) Guest interrupt information: (XEN) IRQ: 0 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:f0 type=IO-APIC-edge status=00000000 mapped, unbound (XEN) IRQ: 1 affinity:00000000,00000000,00000000,00000001 vec:28 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 1(----), (XEN) IRQ: 2 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:e2 type=XT-PIC status=00000000 mapped, unbound (XEN) IRQ: 3 affinity:00000000,00000000,00000000,00000001 vec:30 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 4 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:f1 type=IO-APIC-edge status=00000000 mapped, unbound (XEN) IRQ: 5 affinity:00000000,00000000,00000000,00000001 vec:38 type=IO-APIC-edge status=00000006 mapped, unbound (XEN) IRQ: 6 affinity:00000000,00000000,00000000,00000001 vec:40 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 7 affinity:00000000,00000000,00000000,00000001 vec:48 type=IO-APIC-edge status=00000006 mapped, unbound (XEN) IRQ: 8 affinity:00000000,00000000,00000000,00000001 vec:50 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 8(----), (XEN) IRQ: 9 affinity:00000000,00000000,00000000,00000001 vec:58 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 9(----), (XEN) IRQ: 10 affinity:00000000,00000000,00000000,00000001 vec:60 type=IO-APIC-edge status=00000006 mapped, unbound (XEN) IRQ: 11 affinity:00000000,00000000,00000000,00000001 vec:68 type=IO-APIC-edge status=00000006 mapped, unbound (XEN) IRQ: 12 affinity:00000000,00000000,00000000,00000001 vec:70 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 12(----), (XEN) IRQ: 13 affinity:00000000,00000000,00000000,0000000f vec:78 type=IO-APIC-edge status=00000002 mapped, unbound (XEN) IRQ: 14 affinity:00000000,00000000,00000000,00000001 vec:88 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 14(----), (XEN) IRQ: 15 affinity:00000000,00000000,00000000,00000001 vec:90 type=IO-APIC-edge status=00000010 in-flight=0 domain-list=0: 15(----), (XEN) IRQ: 16 affinity:00000000,00000000,00000000,0000000f vec:b0 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 17 affinity:00000000,00000000,00000000,0000000f vec:a8 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 18 affinity:00000000,00000000,00000000,00000001 vec:a0 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 18(----), (XEN) IRQ: 19 affinity:00000000,00000000,00000000,00000001 vec:98 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 19(----), (XEN) IRQ: 20 affinity:00000000,00000000,00000000,00000001 vec:41 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 20(----), (XEN) IRQ: 21 affinity:00000000,00000000,00000000,00000001 vec:39 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 21(----), (XEN) IRQ: 22 affinity:00000000,00000000,00000000,0000000f vec:29 type=IO-APIC-level status=00000002 mapped, unbound (XEN) IRQ: 23 affinity:00000000,00000000,00000000,00000001 vec:d8 type=IO-APIC-level status=00000010 in-flight=0 domain-list=0: 23(----), (XEN) IRQ: 24 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:b8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 25 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c0 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 26 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:c8 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 27 affinity:ffffffff,ffffffff,ffffffff,ffffffff vec:d0 type=PCI-MSI status=00000002 mapped, unbound (XEN) IRQ: 28 affinity:00000000,00000000,00000000,00000001 vec:59 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:275(----), (XEN) IRQ: 29 affinity:00000000,00000000,00000000,00000001 vec:31 type=PCI-MSI status=00000014 in-flight=0 domain-list=0:274(----), (XEN) IRQ: 30 affinity:00000000,00000000,00000000,00000001 vec:69 type=PCI-MSI status=00000010 in-flight=0 domain-list=0:273(----), (XEN) IRQ: 31 affinity:00000000,00000000,00000000,00000002 vec:d9 type=PCI-MSI status=00000010 in-flight=0 domain-list=21: 55(----), (XEN) IO-APIC interrupt information: (XEN) IRQ 0 Vec240: (XEN) Apic 0x00, Pin 2: vector=240, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:255 (XEN) IRQ 1 Vec 40: (XEN) Apic 0x00, Pin 1: vector=40, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 3 Vec 48: (XEN) Apic 0x00, Pin 3: vector=48, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 4 Vec241: (XEN) Apic 0x00, Pin 4: vector=241, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:255 (XEN) IRQ 5 Vec 56: (XEN) Apic 0x00, Pin 5: vector=56, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 6 Vec 64: (XEN) Apic 0x00, Pin 6: vector=64, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 7 Vec 72: (XEN) Apic 0x00, Pin 7: vector=72, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 8 Vec 80: (XEN) Apic 0x00, Pin 8: vector=80, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 9 Vec 88: (XEN) Apic 0x00, Pin 9: vector=88, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 10 Vec 96: (XEN) Apic 0x00, Pin 10: vector=96, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 11 Vec104: (XEN) Apic 0x00, Pin 11: vector=104, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 12 Vec112: (XEN) Apic 0x00, Pin 12: vector=112, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 13 Vec120: (XEN) Apic 0x00, Pin 13: vector=120, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=1, dest_id:15 (XEN) IRQ 14 Vec136: (XEN) Apic 0x00, Pin 14: vector=136, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 15 Vec144: (XEN) Apic 0x00, Pin 15: vector=144, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=0, irr=0, trigger=edge, mask=0, dest_id:1 (XEN) IRQ 16 Vec176: (XEN) Apic 0x00, Pin 16: vector=176, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:15 (XEN) IRQ 17 Vec168: (XEN) Apic 0x00, Pin 17: vector=168, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:15 (XEN) IRQ 18 Vec160: (XEN) Apic 0x00, Pin 18: vector=160, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 19 Vec152: (XEN) Apic 0x00, Pin 19: vector=152, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 20 Vec 65: (XEN) Apic 0x00, Pin 20: vector=65, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 21 Vec 57: (XEN) Apic 0x00, Pin 21: vector=57, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 (XEN) IRQ 22 Vec 41: (XEN) Apic 0x00, Pin 22: vector=41, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=1, dest_id:15 (XEN) IRQ 23 Vec216: (XEN) Apic 0x00, Pin 23: vector=216, delivery_mode=1, dest_mode=logical, delivery_status=0, polarity=1, irr=0, trigger=level, mask=0, dest_id:1 [-- Attachment #3: xen_debug_keys_q.txt --] [-- Type: text/plain, Size: 2312 bytes --] (XEN) 'q' pressed -> dumping domain info (now=0x5F00:7A49A1F4) (XEN) General information for domain 0: (XEN) refcnt=3 dying=0 nr_pages=64576 xenheap_pages=5 dirty_cpus={0} max_pages=4294967295 (XEN) handle=00000000-0000-0000-0000-000000000000 vm_assist=0000000d (XEN) Rangesets belonging to domain 0: (XEN) I/O Ports { 0-1f, 22-3f, 44-60, 62-9f, a2-3f7, 400-507, 50c-cfb, d00-ffff } (XEN) Interrupts { 0-279 } (XEN) I/O Memory { 0-febff, fec01-fedff, fee01-ffffffffffffffff } (XEN) Memory pages belonging to domain 0: (XEN) DomPage list too long to display (XEN) XenPage 000000000023fcda: caf=c000000000000002, taf=7400000000000002 (XEN) XenPage 000000000023fcd9: caf=c000000000000001, taf=7400000000000001 (XEN) XenPage 000000000023fcd8: caf=c000000000000001, taf=7400000000000001 (XEN) XenPage 000000000023fcd7: caf=c000000000000001, taf=7400000000000001 (XEN) XenPage 00000000000b7afe: caf=c000000000000002, taf=7400000000000002 (XEN) VCPU information and callbacks for domain 0: (XEN) VCPU0: CPU0 [has=T] flags=0 poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={0} cpu_affinity={0} (XEN) No periodic timer (XEN) General information for domain 21: (XEN) refcnt=3 dying=0 nr_pages=65536 xenheap_pages=5 dirty_cpus={1} max_pages=65536 (XEN) handle=a9e1d4f5-b66f-c785-aaf0-48186e5e1e20 vm_assist=0000000d (XEN) Rangesets belonging to domain 21: (XEN) I/O Ports { e800-e8ff } (XEN) Interrupts { 18, 55 } (XEN) I/O Memory { 10100-1011f, feb00-feb03 } (XEN) Memory pages belonging to domain 21: (XEN) DomPage list too long to display (XEN) XenPage 000000000022b8e1: caf=c000000000000002, taf=7400000000000002 (XEN) XenPage 000000000022bd6b: caf=c000000000000001, taf=7400000000000001 (XEN) XenPage 0000000000236521: caf=c000000000000001, taf=7400000000000001 (XEN) XenPage 000000000022bbbc: caf=c000000000000001, taf=7400000000000001 (XEN) XenPage 00000000000b7cbf: caf=c000000000000002, taf=7400000000000002 (XEN) VCPU information and callbacks for domain 21: (XEN) VCPU0: CPU1 [has=F] flags=1 poll=0 upcall_pend = 00, upcall_mask = 00 dirty_cpus={1} cpu_affinity={1} (XEN) No periodic timer (XEN) Notifying guest 0:0 (virq 1, port 4, stat 0/0/0) (XEN) Notifying guest 21:0 (virq 1, port 7, stat 0/0/0) [-- Attachment #4: xen_debug_keys_Q.txt --] [-- Type: text/plain, Size: 1032 bytes --] (XEN) ==== PCI devices ==== (XEN) 04:00.0 - dom 0 - MSIs < 31 > (XEN) 03:00.1 - dom 0 - MSIs < 30 > (XEN) 03:00.0 - dom 0 - MSIs < 28 > (XEN) 02:00.0 - dom 0 - MSIs < > (XEN) 01:05.0 - dom 0 - MSIs < > (XEN) 00:18.4 - dom 0 - MSIs < > (XEN) 00:18.3 - dom 0 - MSIs < > (XEN) 00:18.2 - dom 0 - MSIs < > (XEN) 00:18.1 - dom 0 - MSIs < > (XEN) 00:18.0 - dom 0 - MSIs < > (XEN) 00:14.0 - dom 0 - MSIs < 27 > (XEN) 00:13.0 - dom 0 - MSIs < 26 > (XEN) 00:12.0 - dom 0 - MSIs < 25 > (XEN) 00:10.0 - dom 0 - MSIs < 24 > (XEN) 00:0b.0 - dom 0 - MSIs < > (XEN) 00:09.0 - dom 0 - MSIs < 29 > (XEN) 00:08.0 - dom 0 - MSIs < > (XEN) 00:06.0 - dom 0 - MSIs < > (XEN) 00:04.1 - dom 0 - MSIs < > (XEN) 00:04.0 - dom 0 - MSIs < > (XEN) 00:02.1 - dom 0 - MSIs < > (XEN) 00:02.0 - dom 0 - MSIs < > (XEN) 00:01.4 - dom 0 - MSIs < > (XEN) 00:01.3 - dom 0 - MSIs < > (XEN) 00:01.2 - dom 0 - MSIs < > (XEN) 00:01.1 - dom 0 - MSIs < > (XEN) 00:01.0 - dom 0 - MSIs < > (XEN) 00:00.0 - dom 0 - MSIs < > [-- Attachment #5: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-14 23:29 ` Jean Baptiste Favre @ 2011-01-17 8:59 ` Jean Baptiste Favre 2011-01-17 13:58 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-17 8:59 UTC (permalink / raw) To: xen-devel Hello, Has made little tests this week-end. - Can not use full vanilla kernel with OpenWRT. Must apply some patch to make OpenWRT boot and don't had enough time to figure out which one exactly. Will work on it this week. - Have tested a Debian PV 64bits domU with 2.6.37 kernel from Debian experimental repository. Network is fully working. Will try a 32bits domU this week So, at this point, either because of 64bits, either because of "better" kernel, network with PCI passthrough is working. Still have to try 32bits kernel, as well as filtering OpenWRT patches. Regards, JB Le 15/01/2011 00:29, Jean Baptiste Favre a écrit : > Hello, > > Le 14/01/2011 15:53, Konrad Rzeszutek Wilk a écrit : >>> Tought I told it in my previou smails. Sorry to missed it: >> >> You probably did and I missed it too. Good that you have those options, now: >> >> .. snip .. >>>> What does tcpdump tell you when you try to send it at -s86? >>> >>> I can see echo requests coming in on my gateway, replies going back but >>> replies are never received on the domU. >> >> What is the NIC you are using? Is it a Intel one? What does /proc/interrupts look like? > > The NIC I use is "Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit > Ethernet Controller", not Intel, with sky2 driver. > > # cat /proc/interrupts > CPU0 > 55: 1393 xen-pirq-pcifront-msi sky2@pci:0000:00:00.0 > 246: 202 xen-dyn-event blkif > 247: 103 xen-dyn-event hvc_console > 248: 603 xen-dyn-event pcifront > 249: 259 xen-dyn-event xenbus > 250: 0 xen-percpu-ipi callfuncsingle0 > 251: 0 xen-percpu-virq debug0 > 252: 0 xen-percpu-ipi callfunc0 > 253: 0 xen-percpu-ipi resched0 > 254: 0 xen-percpu-ipi spinlock0 > 255: 246058 xen-percpu-virq timer0 > NMI: 0 Non-maskable interrupts > LOC: 0 Local timer interrupts > SPU: 0 Spurious interrupts > PMI: 0 Performance monitoring interrupts > IWI: 0 IRQ work interrupts > RES: 0 Rescheduling interrupts > CAL: 0 Function call interrupts > TLB: 0 TLB shootdowns > TRM: 0 Thermal event interrupts > THR: 0 Threshold APIC interrupts > MCE: 0 Machine check exceptions > MCP: 0 Machine check polls > ERR: 0 > MIS: 0 > > >> Can you send me the full domU output? What does the Xen hypervisor mapping look like > Output are attached to this mail. > >> (xm debug-keys q, xm debug-keys Q, xm debug-keys i)? What is the motherboard you have? > dmidecode > ... > Handle 0x0002, DMI type 2, 15 bytes > Base Board Information > Manufacturer: ASUSTeK Computer INC. > Product Name: M3N18L T-M3N8200 > Version: Rev x.xx > Serial Number: MS1C85B07000633 > Asset Tag: To Be Filled By O.E.M. > Features: > Board is a hosting board > Board is replaceable > Location In Chassis: To Be Filled By O.E.M. > Chassis Handle: 0x0003 > Type: Motherboard > Contained Object Handles: 0 > ... > >> Did you look up to see if there are any errate for the motherboard or the NIC you are using? > No, since everything was working fine (before I decided to test PCI > passthrough). > >> Do you have the latest firmware for the NIC and the latest BIOS for your motherboard? > Not sure. Will check > >> Is the OpenWRT kernel you are using one that you built yourself or do they package it >> for you? Can you try using a vaniall built one (2.6.37 vanilla is perfect). > It is packaged by openWRT Builroot. Just have to set the kernel version > I want to use. > Will try a vanilla kernel as soon as possible. > >> Is this a 64-bit kernel or 32-bit? Can you try using a 64-bit one (you should be >> able to run a 64-bit kernel alongside 32-bit userspace). > OK, will try it as well. > > Regards, > JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-17 8:59 ` Jean Baptiste Favre @ 2011-01-17 13:58 ` Jean Baptiste Favre 2011-01-22 10:22 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-17 13:58 UTC (permalink / raw) To: xen-devel Hello Konrad, Have tested with Debian 32bits domU. Once domU is started, log in via xm console and exec "ping 10.0.0.1" and "ping -s86 10.0.0.1" Here are the results: - 32bits kernel + 32bits Debian Squeeze => ping -s86 FAILS - 64bits kernel + 32bits Debian Squeeze => WORKS - 64bits kernel + 64bits Debian Squeeze => WORKS Used kernels are: 32bit: linux-image-2.6.37-trunk-686-bigmem 64bit: linux-image-2.6.37-trunk-amd64 Will now try to compile openwrt kernel as 64bits and update you. Regards, JB Le 17/01/2011 09:59, Jean Baptiste Favre a écrit : > Hello, > Has made little tests this week-end. > - Can not use full vanilla kernel with OpenWRT. Must apply some patch to > make OpenWRT boot and don't had enough time to figure out which one > exactly. Will work on it this week. > - Have tested a Debian PV 64bits domU with 2.6.37 kernel from Debian > experimental repository. Network is fully working. Will try a 32bits > domU this week > > So, at this point, either because of 64bits, either because of "better" > kernel, network with PCI passthrough is working. > > Still have to try 32bits kernel, as well as filtering OpenWRT patches. > > Regards, > JB > > > Le 15/01/2011 00:29, Jean Baptiste Favre a écrit : >> Hello, >> >> Le 14/01/2011 15:53, Konrad Rzeszutek Wilk a écrit : >>>> Tought I told it in my previou smails. Sorry to missed it: >>> >>> You probably did and I missed it too. Good that you have those options, now: >>> >>> .. snip .. >>>>> What does tcpdump tell you when you try to send it at -s86? >>>> >>>> I can see echo requests coming in on my gateway, replies going back but >>>> replies are never received on the domU. >>> >>> What is the NIC you are using? Is it a Intel one? What does /proc/interrupts look like? >> >> The NIC I use is "Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit >> Ethernet Controller", not Intel, with sky2 driver. >> >> # cat /proc/interrupts >> CPU0 >> 55: 1393 xen-pirq-pcifront-msi sky2@pci:0000:00:00.0 >> 246: 202 xen-dyn-event blkif >> 247: 103 xen-dyn-event hvc_console >> 248: 603 xen-dyn-event pcifront >> 249: 259 xen-dyn-event xenbus >> 250: 0 xen-percpu-ipi callfuncsingle0 >> 251: 0 xen-percpu-virq debug0 >> 252: 0 xen-percpu-ipi callfunc0 >> 253: 0 xen-percpu-ipi resched0 >> 254: 0 xen-percpu-ipi spinlock0 >> 255: 246058 xen-percpu-virq timer0 >> NMI: 0 Non-maskable interrupts >> LOC: 0 Local timer interrupts >> SPU: 0 Spurious interrupts >> PMI: 0 Performance monitoring interrupts >> IWI: 0 IRQ work interrupts >> RES: 0 Rescheduling interrupts >> CAL: 0 Function call interrupts >> TLB: 0 TLB shootdowns >> TRM: 0 Thermal event interrupts >> THR: 0 Threshold APIC interrupts >> MCE: 0 Machine check exceptions >> MCP: 0 Machine check polls >> ERR: 0 >> MIS: 0 >> >> >>> Can you send me the full domU output? What does the Xen hypervisor mapping look like >> Output are attached to this mail. >> >>> (xm debug-keys q, xm debug-keys Q, xm debug-keys i)? What is the motherboard you have? >> dmidecode >> ... >> Handle 0x0002, DMI type 2, 15 bytes >> Base Board Information >> Manufacturer: ASUSTeK Computer INC. >> Product Name: M3N18L T-M3N8200 >> Version: Rev x.xx >> Serial Number: MS1C85B07000633 >> Asset Tag: To Be Filled By O.E.M. >> Features: >> Board is a hosting board >> Board is replaceable >> Location In Chassis: To Be Filled By O.E.M. >> Chassis Handle: 0x0003 >> Type: Motherboard >> Contained Object Handles: 0 >> ... >> >>> Did you look up to see if there are any errate for the motherboard or the NIC you are using? >> No, since everything was working fine (before I decided to test PCI >> passthrough). >> >>> Do you have the latest firmware for the NIC and the latest BIOS for your motherboard? >> Not sure. Will check >> >>> Is the OpenWRT kernel you are using one that you built yourself or do they package it >>> for you? Can you try using a vaniall built one (2.6.37 vanilla is perfect). >> It is packaged by openWRT Builroot. Just have to set the kernel version >> I want to use. >> Will try a vanilla kernel as soon as possible. >> >>> Is this a 64-bit kernel or 32-bit? Can you try using a 64-bit one (you should be >>> able to run a 64-bit kernel alongside 32-bit userspace). >> OK, will try it as well. >> >> Regards, >> JB > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-17 13:58 ` Jean Baptiste Favre @ 2011-01-22 10:22 ` Jean Baptiste Favre 2011-01-27 20:27 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-22 10:22 UTC (permalink / raw) To: xen-devel Hello, Last investigations show that I've the latest BIOS version for my motherboard. Do you need more tests, if yes which ones ? This problem annoys me because I did not manage to get OpenWRT working with 64bits kernel. Regards, JB Le 17/01/2011 14:58, Jean Baptiste Favre a écrit : > Hello Konrad, > Have tested with Debian 32bits domU. Once domU is started, log in via xm > console and exec "ping 10.0.0.1" and "ping -s86 10.0.0.1" > Here are the results: > > - 32bits kernel + 32bits Debian Squeeze => ping -s86 FAILS > - 64bits kernel + 32bits Debian Squeeze => WORKS > - 64bits kernel + 64bits Debian Squeeze => WORKS > > Used kernels are: > 32bit: linux-image-2.6.37-trunk-686-bigmem > 64bit: linux-image-2.6.37-trunk-amd64 > > Will now try to compile openwrt kernel as 64bits and update you. > > Regards, > JB > > Le 17/01/2011 09:59, Jean Baptiste Favre a écrit : >> Hello, >> Has made little tests this week-end. >> - Can not use full vanilla kernel with OpenWRT. Must apply some patch to >> make OpenWRT boot and don't had enough time to figure out which one >> exactly. Will work on it this week. >> - Have tested a Debian PV 64bits domU with 2.6.37 kernel from Debian >> experimental repository. Network is fully working. Will try a 32bits >> domU this week >> >> So, at this point, either because of 64bits, either because of "better" >> kernel, network with PCI passthrough is working. >> >> Still have to try 32bits kernel, as well as filtering OpenWRT patches. >> >> Regards, >> JB >> >> >> Le 15/01/2011 00:29, Jean Baptiste Favre a écrit : >>> Hello, >>> >>> Le 14/01/2011 15:53, Konrad Rzeszutek Wilk a écrit : >>>>> Tought I told it in my previou smails. Sorry to missed it: >>>> >>>> You probably did and I missed it too. Good that you have those options, now: >>>> >>>> .. snip .. >>>>>> What does tcpdump tell you when you try to send it at -s86? >>>>> >>>>> I can see echo requests coming in on my gateway, replies going back but >>>>> replies are never received on the domU. >>>> >>>> What is the NIC you are using? Is it a Intel one? What does /proc/interrupts look like? >>> >>> The NIC I use is "Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit >>> Ethernet Controller", not Intel, with sky2 driver. >>> >>> # cat /proc/interrupts >>> CPU0 >>> 55: 1393 xen-pirq-pcifront-msi sky2@pci:0000:00:00.0 >>> 246: 202 xen-dyn-event blkif >>> 247: 103 xen-dyn-event hvc_console >>> 248: 603 xen-dyn-event pcifront >>> 249: 259 xen-dyn-event xenbus >>> 250: 0 xen-percpu-ipi callfuncsingle0 >>> 251: 0 xen-percpu-virq debug0 >>> 252: 0 xen-percpu-ipi callfunc0 >>> 253: 0 xen-percpu-ipi resched0 >>> 254: 0 xen-percpu-ipi spinlock0 >>> 255: 246058 xen-percpu-virq timer0 >>> NMI: 0 Non-maskable interrupts >>> LOC: 0 Local timer interrupts >>> SPU: 0 Spurious interrupts >>> PMI: 0 Performance monitoring interrupts >>> IWI: 0 IRQ work interrupts >>> RES: 0 Rescheduling interrupts >>> CAL: 0 Function call interrupts >>> TLB: 0 TLB shootdowns >>> TRM: 0 Thermal event interrupts >>> THR: 0 Threshold APIC interrupts >>> MCE: 0 Machine check exceptions >>> MCP: 0 Machine check polls >>> ERR: 0 >>> MIS: 0 >>> >>> >>>> Can you send me the full domU output? What does the Xen hypervisor mapping look like >>> Output are attached to this mail. >>> >>>> (xm debug-keys q, xm debug-keys Q, xm debug-keys i)? What is the motherboard you have? >>> dmidecode >>> ... >>> Handle 0x0002, DMI type 2, 15 bytes >>> Base Board Information >>> Manufacturer: ASUSTeK Computer INC. >>> Product Name: M3N18L T-M3N8200 >>> Version: Rev x.xx >>> Serial Number: MS1C85B07000633 >>> Asset Tag: To Be Filled By O.E.M. >>> Features: >>> Board is a hosting board >>> Board is replaceable >>> Location In Chassis: To Be Filled By O.E.M. >>> Chassis Handle: 0x0003 >>> Type: Motherboard >>> Contained Object Handles: 0 >>> ... >>> >>>> Did you look up to see if there are any errate for the motherboard or the NIC you are using? >>> No, since everything was working fine (before I decided to test PCI >>> passthrough). >>> >>>> Do you have the latest firmware for the NIC and the latest BIOS for your motherboard? >>> Not sure. Will check >>> >>>> Is the OpenWRT kernel you are using one that you built yourself or do they package it >>>> for you? Can you try using a vaniall built one (2.6.37 vanilla is perfect). >>> It is packaged by openWRT Builroot. Just have to set the kernel version >>> I want to use. >>> Will try a vanilla kernel as soon as possible. >>> >>>> Is this a 64-bit kernel or 32-bit? Can you try using a 64-bit one (you should be >>>> able to run a 64-bit kernel alongside 32-bit userspace). >>> OK, will try it as well. >>> >>> Regards, >>> JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-22 10:22 ` Jean Baptiste Favre @ 2011-01-27 20:27 ` Konrad Rzeszutek Wilk 2011-01-27 21:47 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-01-27 20:27 UTC (permalink / raw) To: xen-devel On Sat, Jan 22, 2011 at 11:22:59AM +0100, Jean Baptiste Favre wrote: > Hello, > Last investigations show that I've the latest BIOS version for my > motherboard. > Do you need more tests, if yes which ones ? I tried it on my 2.6.32.27 (32-bit and 64-bit) and I am not seeing the failures you have. These are the devices I passed in: 00:00.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02) 00:00.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02) 00:00.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02) 00:00.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02) 00:01.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02) 00:02.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) 00:02.1 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) 00:03.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) Granted the kernel I am using a jeremy/xen/stable-2.6.32.x so not sure how divergent from Debian or Debian Squeeze it is. How are you launching your guest? Do you something like this: kernel="/mnt/lab/2.6.32.27/vmlinuz" ramdisk="/mnt/lab/2.6.32.27/initramfs.cpio.gz" extra="console=hvc0 debug test=crashme iommu=soft" memory=1024 maxmem=2048 vcpus=4 on_crash="preserve" pci= ["00:1d.0","00:1d.1","00:1d.2","00:1d.7","0a:00.1","0000:06:01.1","0000:06:01.0", "09:00.0"] vif = [ 'mac=00:0f:4b:00:00:68, bridge=switch' ] ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-27 20:27 ` Konrad Rzeszutek Wilk @ 2011-01-27 21:47 ` Jean Baptiste Favre 2011-01-28 15:47 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-27 21:47 UTC (permalink / raw) To: xen-devel Hello Konrad, Le 27/01/2011 21:27, Konrad Rzeszutek Wilk a écrit : > On Sat, Jan 22, 2011 at 11:22:59AM +0100, Jean Baptiste Favre wrote: >> Hello, >> Last investigations show that I've the latest BIOS version for my >> motherboard. >> Do you need more tests, if yes which ones ? > > I tried it on my 2.6.32.27 (32-bit and 64-bit) and I am not seeing > the failures you have. These are the devices I passed in: > > 00:00.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02) > 00:00.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02) > 00:00.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02) > 00:00.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02) > 00:01.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02) > 00:02.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) > 00:02.1 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) > 00:03.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) > > Granted the kernel I am using a jeremy/xen/stable-2.6.32.x so not sure > how divergent from Debian or Debian Squeeze it is. > > How are you launching your guest? Do you something like this: > > kernel="/mnt/lab/2.6.32.27/vmlinuz" > ramdisk="/mnt/lab/2.6.32.27/initramfs.cpio.gz" > extra="console=hvc0 debug test=crashme iommu=soft" > memory=1024 > maxmem=2048 > vcpus=4 > on_crash="preserve" > pci= ["00:1d.0","00:1d.1","00:1d.2","00:1d.7","0a:00.1","0000:06:01.1","0000:06:01.0", "09:00.0"] > vif = [ 'mac=00:0f:4b:00:00:68, bridge=switch' ] Here is my domU config file: #################### kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' builder = 'linux' memory = '256' memory = '512' vcpus = '1' cpus = '2' localtime = 0 serial = 'pty' disk = [ 'drbd:xps-106,xvda,w' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty" pci = [ '04:00.0' ] name = 'xps-106' hostname = 'xps-106.clichy.jbfavre.org' #################### As I privately told you, I made the tests you suggested and the result is that problem occurs with 256mb of memory, but is solved with 512mb or more. Basically, what I find surprising is that with 256mb of memory, the max size for incoming packets to be blocked is 128 bytes. Makes me think about an unsigned integer or something like that, but I don't have enough kernel knowledge to be more precise. Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-27 21:47 ` Jean Baptiste Favre @ 2011-01-28 15:47 ` Jean Baptiste Favre 2011-02-01 11:34 ` Ian Campbell 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-28 15:47 UTC (permalink / raw) To: xen-devel Hello, I made some more tests today, still with 2.6.37 32bits kernel from Debian experimental, with various memory allocation value. For each test, I make ping on my gateway with various packet size: ping -s15 10.0.0.1 ping -s85 10.0.0.1 ping -s86 10.0.0.1 ping -s150 10.0.0.1 Results bellow: - less than 256mb: works - between 256 and 512mb: ping greater than 85 bytes does not work - more than 512mb: works I'm lost... Regards, JB Le 27/01/2011 22:47, Jean Baptiste Favre a écrit : > Hello Konrad, > > Le 27/01/2011 21:27, Konrad Rzeszutek Wilk a écrit : >> On Sat, Jan 22, 2011 at 11:22:59AM +0100, Jean Baptiste Favre wrote: >>> Hello, >>> Last investigations show that I've the latest BIOS version for my >>> motherboard. >>> Do you need more tests, if yes which ones ? >> >> I tried it on my 2.6.32.27 (32-bit and 64-bit) and I am not seeing >> the failures you have. These are the devices I passed in: >> >> 00:00.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02) >> 00:00.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02) >> 00:00.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02) >> 00:00.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02) >> 00:01.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02) >> 00:02.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) >> 00:02.1 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) >> 00:03.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) >> >> Granted the kernel I am using a jeremy/xen/stable-2.6.32.x so not sure >> how divergent from Debian or Debian Squeeze it is. >> >> How are you launching your guest? Do you something like this: >> >> kernel="/mnt/lab/2.6.32.27/vmlinuz" >> ramdisk="/mnt/lab/2.6.32.27/initramfs.cpio.gz" >> extra="console=hvc0 debug test=crashme iommu=soft" >> memory=1024 >> maxmem=2048 >> vcpus=4 >> on_crash="preserve" >> pci= ["00:1d.0","00:1d.1","00:1d.2","00:1d.7","0a:00.1","0000:06:01.1","0000:06:01.0", "09:00.0"] >> vif = [ 'mac=00:0f:4b:00:00:68, bridge=switch' ] > > Here is my domU config file: > #################### > kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' > ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' > builder = 'linux' > > memory = '256' > memory = '512' > vcpus = '1' > cpus = '2' > localtime = 0 > serial = 'pty' > > disk = [ 'drbd:xps-106,xvda,w' ] > > on_poweroff = 'destroy' > on_reboot = 'restart' > on_crash = 'restart' > > extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force > console=hvc0 xencons=tty" > > pci = [ '04:00.0' ] > > name = 'xps-106' > hostname = 'xps-106.clichy.jbfavre.org' > #################### > > As I privately told you, I made the tests you suggested and the result > is that problem occurs with 256mb of memory, but is solved with 512mb or > more. > > Basically, what I find surprising is that with 256mb of memory, the max > size for incoming packets to be blocked is 128 bytes. > Makes me think about an unsigned integer or something like that, but I > don't have enough kernel knowledge to be more precise. > > Regards, > JB > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-28 15:47 ` Jean Baptiste Favre @ 2011-02-01 11:34 ` Ian Campbell 2011-02-01 12:17 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-01 11:34 UTC (permalink / raw) To: xen-devel On Fri, 2011-01-28 at 15:47 +0000, Jean Baptiste Favre wrote: > Hello, > I made some more tests today, still with 2.6.37 32bits kernel from > Debian experimental, with various memory allocation value. > > For each test, I make ping on my gateway with various packet size: > ping -s15 10.0.0.1 > ping -s85 10.0.0.1 > ping -s86 10.0.0.1 > ping -s150 10.0.0.1 > > Results bellow: > > - less than 256mb: works > - between 256 and 512mb: ping greater than 85 bytes does not work > - more than 512mb: works > > I'm lost... Me too, this really is the most inexplicable set of symptoms... Does it work correctly with any other guest kernel, e.g. the xen/stable-2.6.32.x branch from xen.git or maybe one of the old-style Xen kernels? The network device in use is one of the Intel NICs below? Any luck just passing through that one device without all the others? Previously you mentioned using a Marvell NIC, so I guess the failure is independent of the NIC type? Your userspace is still OpenWRT in these most recent tests? Is that some sort of busybox based thing? Can you try with e.g. a regular Debian guest userspace to rule out any funnyness from that end? If you restrict dom0 to >256MB but <512MB (using dom0_mem= on hypervisor command line) does the NIC work correctly in non-passedthrough form? Similarly does the kernel running native with mem= cause the failure? Bit of a long shot but are you able to try a 4.0.2-rc hypervisor+tools and/or a 4.1.0-rc setup (not branched yet so still in xen-unstable.hg)? Is the 10.0.0.1 address you are testing against a VM on the same host or some sort of external entity? Ian. > Regards, > JB > > > Le 27/01/2011 22:47, Jean Baptiste Favre a écrit : > > Hello Konrad, > > > > Le 27/01/2011 21:27, Konrad Rzeszutek Wilk a écrit : > >> On Sat, Jan 22, 2011 at 11:22:59AM +0100, Jean Baptiste Favre wrote: > >>> Hello, > >>> Last investigations show that I've the latest BIOS version for my > >>> motherboard. > >>> Do you need more tests, if yes which ones ? > >> > >> I tried it on my 2.6.32.27 (32-bit and 64-bit) and I am not seeing > >> the failures you have. These are the devices I passed in: > >> > >> 00:00.0 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02) > >> 00:00.1 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02) > >> 00:00.2 USB Controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02) > >> 00:00.7 USB Controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02) > >> 00:01.1 Ethernet controller: Intel Corporation 82575EB Gigabit Network Connection (rev 02) > >> 00:02.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) > >> 00:02.1 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07) > >> 00:03.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) > >> > >> Granted the kernel I am using a jeremy/xen/stable-2.6.32.x so not sure > >> how divergent from Debian or Debian Squeeze it is. > >> > >> How are you launching your guest? Do you something like this: > >> > >> kernel="/mnt/lab/2.6.32.27/vmlinuz" > >> ramdisk="/mnt/lab/2.6.32.27/initramfs.cpio.gz" > >> extra="console=hvc0 debug test=crashme iommu=soft" > >> memory=1024 > >> maxmem=2048 > >> vcpus=4 > >> on_crash="preserve" > >> pci= ["00:1d.0","00:1d.1","00:1d.2","00:1d.7","0a:00.1","0000:06:01.1","0000:06:01.0", "09:00.0"] > >> vif = [ 'mac=00:0f:4b:00:00:68, bridge=switch' ] > > > > Here is my domU config file: > > #################### > > kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' > > ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' > > builder = 'linux' > > > > memory = '256' > > memory = '512' > > vcpus = '1' > > cpus = '2' > > localtime = 0 > > serial = 'pty' > > > > disk = [ 'drbd:xps-106,xvda,w' ] > > > > on_poweroff = 'destroy' > > on_reboot = 'restart' > > on_crash = 'restart' > > > > extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force > > console=hvc0 xencons=tty" > > > > pci = [ '04:00.0' ] > > > > name = 'xps-106' > > hostname = 'xps-106.clichy.jbfavre.org' > > #################### > > > > As I privately told you, I made the tests you suggested and the result > > is that problem occurs with 256mb of memory, but is solved with 512mb or > > more. > > > > Basically, what I find surprising is that with 256mb of memory, the max > > size for incoming packets to be blocked is 128 bytes. > > Makes me think about an unsigned integer or something like that, but I > > don't have enough kernel knowledge to be more precise. > > > > Regards, > > JB > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 11:34 ` Ian Campbell @ 2011-02-01 12:17 ` Jean Baptiste Favre 2011-02-01 13:20 ` Ian Campbell 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 12:17 UTC (permalink / raw) To: xen-devel Hello Ian, Le 01/02/2011 12:34, Ian Campbell a écrit : > On Fri, 2011-01-28 at 15:47 +0000, Jean Baptiste Favre wrote: > > Hello, > > I made some more tests today, still with 2.6.37 32bits kernel from > > Debian experimental, with various memory allocation value. > > > > For each test, I make ping on my gateway with various packet size: > > ping -s15 10.0.0.1 > > ping -s85 10.0.0.1 > > ping -s86 10.0.0.1 > > ping -s150 10.0.0.1 > > > > Results bellow: > > > > - less than 256mb: works > > - between 256 and 512mb: ping greater than 85 bytes does not work > > - more than 512mb: works > > > > I'm lost... > > Me too, this really is the most inexplicable set of symptoms... > > Does it work correctly with any other guest kernel, e.g. the > xen/stable-2.6.32.x branch from xen.git or maybe one of the old-style > Xen kernels? I'm compiling 2.6.32 kernel from Jeremy's GIT repos to check that. > The network device in use is one of the Intel NICs below? Any luck just > passing through that one device without all the others? > > Previously you mentioned using a Marvell NIC, so I guess the failure is > independent of the NIC type? I made all tests with Marvell NIC (driver sky2). I don't know if the same behaviour occurs with another NIC type. I'll try with an Intel one (I've a dual port Intel NIC, so I could passthrough only one port) > Your userspace is still OpenWRT in these most recent tests? Is that some > sort of busybox based thing? Can you try with e.g. a regular Debian > guest userspace to rule out any funnyness from that end? I made tests with both OpenWRT and Debian Squeeze. Had problems to compile OpenWRT kernel in 64bits :) I tested Debian Squeeze with 2.6.37 kernel from experimental (because of Xen PCI Frontend integration. Not sure it has been backported into 2.6.32). I will test with Jeremy's 2.6.32 kernel (see above). As a conclusion, last results are from Debian Squeeze with 2.6.37 and that's why I wrote on debian-kernel maillist :) > If you restrict dom0 to >256MB but <512MB (using dom0_mem= on hypervisor > command line) does the NIC work correctly in non-passedthrough form? My Xen hypervisor commandline is as follow: placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all guest_loglvl=all com1=115200,8n1 console=com1 Everything works great without passthrough, but my dom0 is 64bits which may explain that (I do have this strange behaviour only with 32bits kernels). I did not tried changing dom0_mem param. > Similarly does the kernel running native with mem= cause the failure? Not sure I understand what you mean here. BTW, I'm preparing a set of automatic tests with different memory values. That will be: * loop for each mem value - set memory in domU configfile - starting domU with 128Mb memory - rc.local will ping my gateway with different packet size, store result in file - halt domU * end of loop * Check results :-/ > Bit of a long shot but are you able to try a 4.0.2-rc hypervisor+tools > and/or a 4.1.0-rc setup (not branched yet so still in xen-unstable.hg)? I can eventually try it, but after my looong test list :) > Is the 10.0.0.1 address you are testing against a VM on the same host or > some sort of external entity? It's my gateway (for the history, WRT54GL with OpenWRT). Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 12:17 ` Jean Baptiste Favre @ 2011-02-01 13:20 ` Ian Campbell 2011-02-01 14:12 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-01 13:20 UTC (permalink / raw) To: xen-devel On Tue, 2011-02-01 at 12:17 +0000, Jean Baptiste Favre wrote: > > The network device in use is one of the Intel NICs below? Any luck just > > passing through that one device without all the others? > > > > Previously you mentioned using a Marvell NIC, so I guess the failure is > > independent of the NIC type? > I made all tests with Marvell NIC (driver sky2). > I don't know if the same behaviour occurs with another NIC type. I'll > try with an Intel one (I've a dual port Intel NIC, so I could > passthrough only one port) I was confused -- the quoted list of passthrough devices including Intel NICs I saw was from Konrad not you. It would still be interesting to confirm if the issue is specific to the particular NIC or not. > > Your userspace is still OpenWRT in these most recent tests? Is that some > > sort of busybox based thing? Can you try with e.g. a regular Debian > > guest userspace to rule out any funnyness from that end? > I made tests with both OpenWRT and Debian Squeeze. > Had problems to compile OpenWRT kernel in 64bits :) > I tested Debian Squeeze with 2.6.37 kernel from experimental (because of > Xen PCI Frontend integration. Not sure it has been backported into 2.6.32). The PCI frontend is in the xen.git xen/stable-2.6.32.x branch but not in mainline 2.6.32.y stable branches. > > If you restrict dom0 to >256MB but <512MB (using dom0_mem= on hypervisor > > command line) does the NIC work correctly in non-passedthrough form? > My Xen hypervisor commandline is as follow: > placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all > guest_loglvl=all com1=115200,8n1 console=com1 > > Everything works great without passthrough, but my dom0 is 64bits which > may explain that (I do have this strange behaviour only with 32bits > kernels). I don't suppose you could try installing a 32 bit OS in dom0? (e.g. as a skanky hack you might fit something into your existing swap partition ;-)) > I did not tried changing dom0_mem param. > > > Similarly does the kernel running native with mem= cause the failure? > Not sure I understand what you mean here. I meant to run the system as a native (non-Xen) system and use the kernels mem= command line parameter to restrict the system to the problematic amounts of memory. Really just trying to verify if something is up specifically due to Xen or not. Probably needs a 32 bit user/kernel to be a useful experiment. What do "ifconfig" or "ethtool -S" show for the device after some failures. Do any of the numbers increment inline with the dropped frames? Perhaps interestingly 85 bytes, plus the ICMP, IP and Ethernet header sizes is something around 128 bytes (depending on options etc). This is pure numerology but I notice that sky2 has a copybreak parameter ("Receive copy threshold") which defaults to 128. I think it would be worth trying 64 and 256. Are you able to see any traffic with tcpdump, either in guest and/or from another host (may require switch configuration to allow it to see the traffic). e.g. do you see the ICMP Echo Request but not the Echo Response or do you see nothing at all etc. Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 13:20 ` Ian Campbell @ 2011-02-01 14:12 ` Jean Baptiste Favre 2011-02-01 14:18 ` Ian Campbell 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 14:12 UTC (permalink / raw) To: xen-devel Hello, Le 01/02/2011 14:20, Ian Campbell a écrit : > On Tue, 2011-02-01 at 12:17 +0000, Jean Baptiste Favre wrote: > >> The network device in use is one of the Intel NICs below? Any luck just > >> passing through that one device without all the others? > >> > >> Previously you mentioned using a Marvell NIC, so I guess the failure is > >> independent of the NIC type? > > I made all tests with Marvell NIC (driver sky2). > > I don't know if the same behaviour occurs with another NIC type. I'll > > try with an Intel one (I've a dual port Intel NIC, so I could > > passthrough only one port) > > I was confused -- the quoted list of passthrough devices including Intel > NICs I saw was from Konrad not you. It would still be interesting to > confirm if the issue is specific to the particular NIC or not. > > >> Your userspace is still OpenWRT in these most recent tests? Is that > some > >> sort of busybox based thing? Can you try with e.g. a regular Debian > >> guest userspace to rule out any funnyness from that end? > > I made tests with both OpenWRT and Debian Squeeze. > > Had problems to compile OpenWRT kernel in 64bits :) > > I tested Debian Squeeze with 2.6.37 kernel from experimental (because of > > Xen PCI Frontend integration. Not sure it has been backported into > 2.6.32). > > The PCI frontend is in the xen.git xen/stable-2.6.32.x branch but not in > mainline 2.6.32.y stable branches. Yes, that's why I tried with 2.6.37 from experimental, after Pasi's answer to one of my question on the maillist one month ago or so. > >> If you restrict dom0 to >256MB but <512MB (using dom0_mem= on > hypervisor > >> command line) does the NIC work correctly in non-passedthrough form? > > My Xen hypervisor commandline is as follow: > > placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all > > guest_loglvl=all com1=115200,8n1 console=com1 > > > > Everything works great without passthrough, but my dom0 is 64bits which > > may explain that (I do have this strange behaviour only with 32bits > > kernels). > > I don't suppose you could try installing a 32 bit OS in dom0? (e.g. as a > skanky hack you might fit something into your existing swap > partition ;-)) I could try, but that would means breaking my test platform. Let's say I prefer complete other tests before :) > > I did not tried changing dom0_mem param. > > > >> Similarly does the kernel running native with mem= cause the failure? > > Not sure I understand what you mean here. > > I meant to run the system as a native (non-Xen) system and use the > kernels mem= command line parameter to restrict the system to the > problematic amounts of memory. Really just trying to verify if something > is up specifically due to Xen or not. Probably needs a 32 bit > user/kernel to be a useful experiment. Ah ok. But then, running 32bits kernel with 64 bits OS will work ? (I didn't know 64bits kernel worked with 32bits OS before Konrad told me) > What do "ifconfig" or "ethtool -S" show for the device after some > failures. Do any of the numbers increment inline with the dropped > frames? > > Perhaps interestingly 85 bytes, plus the ICMP, IP and Ethernet header > sizes is something around 128 bytes (depending on options etc). This is > pure numerology but I notice that sky2 has a copybreak parameter > ("Receive copy threshold") which defaults to 128. I think it would be > worth trying 64 and 256. That's exactly the problem I faced with 256mb memory for my domU. Outgoing packets reached my gateway (tcpdump saw them on it, as well as replies) but incoming packets greater than 128bits were blocked somewhere between Xen hypervisor and domU user space (where I was listening traffic with tcpdump). I didn't notice the copybreak from sky2. Where can I change it ? It could be exactly what we're looking for from the beginning, couldn't it ? How can I change it ? > Are you able to see any traffic with tcpdump, either in guest and/or > from another host (may require switch configuration to allow it to see > the traffic). e.g. do you see the ICMP Echo Request but not the Echo > Response or do you see nothing at all etc. tcpdump tests have been done from my gateway and from domU. On the GW: can see all incoming packets (whatever could be the size) and send all replies. On the DomU: can see all outgoing packets but only incoming ones shorter than 128bytes (global size, means "ping -s85" and less) Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 14:12 ` Jean Baptiste Favre @ 2011-02-01 14:18 ` Ian Campbell 2011-02-01 15:14 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-01 14:18 UTC (permalink / raw) To: xen-devel On Tue, 2011-02-01 at 14:12 +0000, Jean Baptiste Favre wrote: > Le 01/02/2011 14:20, Ian Campbell a écrit : > > >> If you restrict dom0 to >256MB but <512MB (using dom0_mem= on > > hypervisor > > >> command line) does the NIC work correctly in non-passedthrough form? > > > My Xen hypervisor commandline is as follow: > > > placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all > > > guest_loglvl=all com1=115200,8n1 console=com1 > > > > > > Everything works great without passthrough, but my dom0 is 64bits which > > > may explain that (I do have this strange behaviour only with 32bits > > > kernels). > > > > I don't suppose you could try installing a 32 bit OS in dom0? (e.g. as a > > skanky hack you might fit something into your existing swap > > partition ;-)) > I could try, but that would means breaking my test platform. Let's say I > prefer complete other tests before :) Sure, no problem. > > > I did not tried changing dom0_mem param. > > > > > >> Similarly does the kernel running native with mem= cause the failure? > > > Not sure I understand what you mean here. > > > > I meant to run the system as a native (non-Xen) system and use the > > kernels mem= command line parameter to restrict the system to the > > problematic amounts of memory. Really just trying to verify if something > > is up specifically due to Xen or not. Probably needs a 32 bit > > user/kernel to be a useful experiment. > Ah ok. But then, running 32bits kernel with 64 bits OS will work ? (I > didn't know 64bits kernel worked with 32bits OS before Konrad told me) 32 user on 64 kernel works, 64 user on 32 kernel does not so this will tie in with the 32 bit test above too. > > What do "ifconfig" or "ethtool -S" show for the device after some > > failures. Do any of the numbers increment inline with the dropped > > frames? > > > > Perhaps interestingly 85 bytes, plus the ICMP, IP and Ethernet header > > sizes is something around 128 bytes (depending on options etc). This is > > pure numerology but I notice that sky2 has a copybreak parameter > > ("Receive copy threshold") which defaults to 128. I think it would be > > worth trying 64 and 256. > That's exactly the problem I faced with 256mb memory for my domU. > Outgoing packets reached my gateway (tcpdump saw them on it, as well as > replies) but incoming packets greater than 128bits were blocked > somewhere between Xen hypervisor and domU user space (where I was > listening traffic with tcpdump). > > I didn't notice the copybreak from sky2. Where can I change it ? It > could be exactly what we're looking for from the beginning, couldn't it ? > How can I change it ? Assuming the driver is modular: "modprobe sky2 copybreak=<N>" Depending on your distro there will be somewhere in /etc you can add this. e.g. on Debian you can create a file in /etc/modprobe.d/ containing "option sky copybreak=<N>" other distros use /etc/modprobe.conf etc. > > Are you able to see any traffic with tcpdump, either in guest and/or > > from another host (may require switch configuration to allow it to see > > the traffic). e.g. do you see the ICMP Echo Request but not the Echo > > Response or do you see nothing at all etc. > tcpdump tests have been done from my gateway and from domU. > On the GW: can see all incoming packets (whatever could be the size) and > send all replies. > On the DomU: can see all outgoing packets but only incoming ones shorter > than 128bytes (global size, means "ping -s85" and less) OK, so it is the sky2 receive path which is at fault, that's very useful info. It corresponds with the copybreak theory too. Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 14:18 ` Ian Campbell @ 2011-02-01 15:14 ` Jean Baptiste Favre 2011-02-01 15:38 ` Jean Baptiste Favre 2011-02-01 15:38 ` Ian Campbell 0 siblings, 2 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 15:14 UTC (permalink / raw) To: xen-devel Le 01/02/2011 15:18, Ian Campbell a écrit : > On Tue, 2011-02-01 at 14:12 +0000, Jean Baptiste Favre wrote: > > > Le 01/02/2011 14:20, Ian Campbell a écrit : > > >>>> If you restrict dom0 to >256MB but <512MB (using dom0_mem= on > >> hypervisor > >>>> command line) does the NIC work correctly in non-passedthrough form? > >>> My Xen hypervisor commandline is as follow: > >>> placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all > >>> guest_loglvl=all com1=115200,8n1 console=com1 > >>> > >>> Everything works great without passthrough, but my dom0 is 64bits > which > >>> may explain that (I do have this strange behaviour only with 32bits > >>> kernels). > >> > >> I don't suppose you could try installing a 32 bit OS in dom0? (e.g. > as a > >> skanky hack you might fit something into your existing swap > >> partition ;-)) > > I could try, but that would means breaking my test platform. Let's say I > > prefer complete other tests before :) > > Sure, no problem. > > >>> I did not tried changing dom0_mem param. > >>> > >>>> Similarly does the kernel running native with mem= cause the failure? > >>> Not sure I understand what you mean here. > >> > >> I meant to run the system as a native (non-Xen) system and use the > >> kernels mem= command line parameter to restrict the system to the > >> problematic amounts of memory. Really just trying to verify if > something > >> is up specifically due to Xen or not. Probably needs a 32 bit > >> user/kernel to be a useful experiment. > > Ah ok. But then, running 32bits kernel with 64 bits OS will work ? (I > > didn't know 64bits kernel worked with 32bits OS before Konrad told me) > > 32 user on 64 kernel works, 64 user on 32 kernel does not so this will > tie in with the 32 bit test above too. > > >> What do "ifconfig" or "ethtool -S" show for the device after some > >> failures. Do any of the numbers increment inline with the dropped > >> frames? > >> > >> Perhaps interestingly 85 bytes, plus the ICMP, IP and Ethernet header > >> sizes is something around 128 bytes (depending on options etc). This is > >> pure numerology but I notice that sky2 has a copybreak parameter > >> ("Receive copy threshold") which defaults to 128. I think it would be > >> worth trying 64 and 256. > > That's exactly the problem I faced with 256mb memory for my domU. > > Outgoing packets reached my gateway (tcpdump saw them on it, as well as > > replies) but incoming packets greater than 128bits were blocked > > somewhere between Xen hypervisor and domU user space (where I was > > listening traffic with tcpdump). > > > > I didn't notice the copybreak from sky2. Where can I change it ? It > > could be exactly what we're looking for from the beginning, couldn't > it ? > > How can I change it ? > > Assuming the driver is modular: > "modprobe sky2 copybreak=<N>" > > Depending on your distro there will be somewhere in /etc you can add > this. e.g. on Debian you can create a file in /etc/modprobe.d/ > containing "option sky copybreak=<N>" other distros > use /etc/modprobe.conf etc. OK I see but it doesn't seems to have any effect. I tried "option sky copybreak=0" to get all packet copied with no change. But I have to say that I'm a bit confused: as I run a PV domU, kernel and initrd are provided by dom0. So basically, I had no module related binaries installed. After installation, I tried to remove module and reload it with different configuration without changes. Is there any way to provide this sort of option in kernel commandline so that it 'll be taken into account even in initrd ? > >> Are you able to see any traffic with tcpdump, either in guest and/or > >> from another host (may require switch configuration to allow it to see > >> the traffic). e.g. do you see the ICMP Echo Request but not the Echo > >> Response or do you see nothing at all etc. > > tcpdump tests have been done from my gateway and from domU. > > On the GW: can see all incoming packets (whatever could be the size) and > > send all replies. > > On the DomU: can see all outgoing packets but only incoming ones shorter > > than 128bytes (global size, means "ping -s85" and less) > > OK, so it is the sky2 receive path which is at fault, that's very useful > info. It corresponds with the copybreak theory too. And that makes me happy, because I began to desperate :) JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 15:14 ` Jean Baptiste Favre @ 2011-02-01 15:38 ` Jean Baptiste Favre 2011-02-01 16:23 ` Ian Campbell 2011-02-01 15:38 ` Ian Campbell 1 sibling, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 15:38 UTC (permalink / raw) To: xen-devel Hello, I think I'll have to read another time "man modprobe" :) See bellow, I've good news Le 01/02/2011 16:14, Jean Baptiste Favre a écrit : > Le 01/02/2011 15:18, Ian Campbell a écrit : > > On Tue, 2011-02-01 at 14:12 +0000, Jean Baptiste Favre wrote: > > > >> Le 01/02/2011 14:20, Ian Campbell a écrit : > > > >>>>> If you restrict dom0 to >256MB but <512MB (using dom0_mem= on > >>> hypervisor > >>>>> command line) does the NIC work correctly in non-passedthrough form? > >>>> My Xen hypervisor commandline is as follow: > >>>> placeholder dom0_mem=256M dom0_max_vcpus=1 dom0_vcpus_pin loglvl=all > >>>> guest_loglvl=all com1=115200,8n1 console=com1 > >>>> > >>>> Everything works great without passthrough, but my dom0 is 64bits > > which > >>>> may explain that (I do have this strange behaviour only with 32bits > >>>> kernels). > >>> > >>> I don't suppose you could try installing a 32 bit OS in dom0? (e.g. > > as a > >>> skanky hack you might fit something into your existing swap > >>> partition ;-)) > >> I could try, but that would means breaking my test platform. Let's > say I > >> prefer complete other tests before :) > > > > Sure, no problem. > > > >>>> I did not tried changing dom0_mem param. > >>>> > >>>>> Similarly does the kernel running native with mem= cause the > failure? > >>>> Not sure I understand what you mean here. > >>> > >>> I meant to run the system as a native (non-Xen) system and use the > >>> kernels mem= command line parameter to restrict the system to the > >>> problematic amounts of memory. Really just trying to verify if > > something > >>> is up specifically due to Xen or not. Probably needs a 32 bit > >>> user/kernel to be a useful experiment. > >> Ah ok. But then, running 32bits kernel with 64 bits OS will work ? (I > >> didn't know 64bits kernel worked with 32bits OS before Konrad told me) > > > > 32 user on 64 kernel works, 64 user on 32 kernel does not so this will > > tie in with the 32 bit test above too. > > > >>> What do "ifconfig" or "ethtool -S" show for the device after some > >>> failures. Do any of the numbers increment inline with the dropped > >>> frames? > >>> > >>> Perhaps interestingly 85 bytes, plus the ICMP, IP and Ethernet header > >>> sizes is something around 128 bytes (depending on options etc). > This is > >>> pure numerology but I notice that sky2 has a copybreak parameter > >>> ("Receive copy threshold") which defaults to 128. I think it would be > >>> worth trying 64 and 256. > >> That's exactly the problem I faced with 256mb memory for my domU. > >> Outgoing packets reached my gateway (tcpdump saw them on it, as well as > >> replies) but incoming packets greater than 128bits were blocked > >> somewhere between Xen hypervisor and domU user space (where I was > >> listening traffic with tcpdump). > >> > >> I didn't notice the copybreak from sky2. Where can I change it ? It > >> could be exactly what we're looking for from the beginning, couldn't > > it ? > >> How can I change it ? > > > > Assuming the driver is modular: > > "modprobe sky2 copybreak=<N>" > > > > Depending on your distro there will be somewhere in /etc you can add > > this. e.g. on Debian you can create a file in /etc/modprobe.d/ > > containing "option sky copybreak=<N>" other distros > > use /etc/modprobe.conf etc. > OK I see but it doesn't seems to have any effect. > I tried "option sky copybreak=0" to get all packet copied with no change. > > But I have to say that I'm a bit confused: as I run a PV domU, kernel > and initrd are provided by dom0. > So basically, I had no module related binaries installed. After > installation, I tried to remove module and reload it with different > configuration without changes. > Is there any way to provide this sort of option in kernel commandline so > that it 'll be taken into account even in initrd ? > > >>> Are you able to see any traffic with tcpdump, either in guest and/or > >>> from another host (may require switch configuration to allow it to see > >>> the traffic). e.g. do you see the ICMP Echo Request but not the Echo > >>> Response or do you see nothing at all etc. > >> tcpdump tests have been done from my gateway and from domU. > >> On the GW: can see all incoming packets (whatever could be the > size) and > >> send all replies. > >> On the DomU: can see all outgoing packets but only incoming ones > shorter > >> than 128bytes (global size, means "ping -s85" and less) > > > > OK, so it is the sky2 receive path which is at fault, that's very useful > > info. It corresponds with the copybreak theory too. OK, just found it: after domU boot: - log in - ping -s 86 10.0.0.1 (fails) - rmmod sky2 - modprobe sky2 copybreak=0 (no packet copied) - ping -s 86 10.0.0.1 (works) So it's clearly related to that option. Now the question: what am I supposed to do ? It seems that adding this options in /etc/modprobe.d/sky2.conf does not work, neither using /etc/modules Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 15:38 ` Jean Baptiste Favre @ 2011-02-01 16:23 ` Ian Campbell 2011-02-01 19:37 ` Konrad Rzeszutek Wilk 2011-02-01 22:04 ` Jean Baptiste Favre 0 siblings, 2 replies; 73+ messages in thread From: Ian Campbell @ 2011-02-01 16:23 UTC (permalink / raw) To: xen-devel; +Cc: Konrad Rzeszutek Wilk On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote: > OK, just found it: > after domU boot: > - log in > - ping -s 86 10.0.0.1 (fails) > - rmmod sky2 > - modprobe sky2 copybreak=0 (no packet copied) > - ping -s 86 10.0.0.1 (works) > > So it's clearly related to that option. Awesome! > Now the question: what am I supposed to do ? I think the next step is to try and reproduce on native 32 bit, with RAM artificially limited via the mem= kernel command in option, this will let us determine if this is a generic issue or is somehow Xen specific. The main difference caused by the copybreak option is that for larger frames (i.e. always when copybreak==0) it hits a code path which uses pci_map_single and pci_map_page to access the received data. When len < copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it seems like the later path is broken somehow. If the issue does turn out to be related to Xen then I think that points to the swiotlb code. I assume you are not seeing "rx mapping error" in your domU dmesg? Did you post a full guest console log at some point? Comparing the logs for the 256MB, 398MB and 512MB guest RAM case might be useful. Konrad, is there some way to force swiotlb use even for native to make the native test cases more relevant? Or is there some other direction we should try first? > It seems that adding this options in /etc/modprobe.d/sky2.conf does not > work, neither using /etc/modules That will be a userspace issue in your distro, perhaps with openwrt they are not expected to work that way. Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 16:23 ` Ian Campbell @ 2011-02-01 19:37 ` Konrad Rzeszutek Wilk 2011-02-01 22:06 ` Jean Baptiste Favre 2011-02-01 23:01 ` Jean Baptiste Favre 2011-02-01 22:04 ` Jean Baptiste Favre 1 sibling, 2 replies; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-02-01 19:37 UTC (permalink / raw) To: Ian Campbell; +Cc: xen-devel On Tue, Feb 01, 2011 at 04:23:09PM +0000, Ian Campbell wrote: > On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote: > > OK, just found it: > > after domU boot: > > - log in > > - ping -s 86 10.0.0.1 (fails) > > - rmmod sky2 > > - modprobe sky2 copybreak=0 (no packet copied) > > - ping -s 86 10.0.0.1 (works) > > > > So it's clearly related to that option. > > Awesome! > > > Now the question: what am I supposed to do ? > > I think the next step is to try and reproduce on native 32 bit, with RAM > artificially limited via the mem= kernel command in option, this will > let us determine if this is a generic issue or is somehow Xen specific. > > The main difference caused by the copybreak option is that for larger > frames (i.e. always when copybreak==0) it hits a code path which uses > pci_map_single and pci_map_page to access the received data. When len < > copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it > seems like the later path is broken somehow. If the issue does turn out It could also have gotten the direction reverted (the 3c5XX code had it wrong at some point so..). Might make sense to compile the kernel with CONFIG_DMA_API_DEBUG which is good at detecting these issues. > to be related to Xen then I think that points to the swiotlb code. > > I assume you are not seeing "rx mapping error" in your domU dmesg? Did > you post a full guest console log at some point? Comparing the logs for > the 256MB, 398MB and 512MB guest RAM case might be useful. > > Konrad, is there some way to force swiotlb use even for native to make > the native test cases more relevant? Or is there some other direction we swiotlb=force will do it. > should try first? <shakes his head> I think we have a good lead. > > > It seems that adding this options in /etc/modprobe.d/sky2.conf does not > > work, neither using /etc/modules You could just pass as Linux kernel parameter 'sky2.copybreak=0'. But I am not sure if that stays if the 'sky2' driver is compiled as module. > > That will be a userspace issue in your distro, perhaps with openwrt they > are not expected to work that way. > > Ian. > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 19:37 ` Konrad Rzeszutek Wilk @ 2011-02-01 22:06 ` Jean Baptiste Favre 2011-02-02 9:12 ` Ian Campbell 2011-02-01 23:01 ` Jean Baptiste Favre 1 sibling, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 22:06 UTC (permalink / raw) To: xen-devel Le 01/02/2011 20:37, Konrad Rzeszutek Wilk a écrit : > On Tue, Feb 01, 2011 at 04:23:09PM +0000, Ian Campbell wrote: >> On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote: >>> OK, just found it: >>> after domU boot: >>> - log in >>> - ping -s 86 10.0.0.1 (fails) >>> - rmmod sky2 >>> - modprobe sky2 copybreak=0 (no packet copied) >>> - ping -s 86 10.0.0.1 (works) >>> >>> So it's clearly related to that option. >> >> Awesome! >> >>> Now the question: what am I supposed to do ? >> >> I think the next step is to try and reproduce on native 32 bit, with RAM >> artificially limited via the mem= kernel command in option, this will >> let us determine if this is a generic issue or is somehow Xen specific. >> >> The main difference caused by the copybreak option is that for larger >> frames (i.e. always when copybreak==0) it hits a code path which uses >> pci_map_single and pci_map_page to access the received data. When len < >> copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it >> seems like the later path is broken somehow. If the issue does turn out > > It could also have gotten the direction reverted (the 3c5XX code had it > wrong at some point so..). Might make sense to compile the kernel with > CONFIG_DMA_API_DEBUG which is good at detecting these issues. > >> to be related to Xen then I think that points to the swiotlb code. > >> >> I assume you are not seeing "rx mapping error" in your domU dmesg? Did >> you post a full guest console log at some point? Comparing the logs for >> the 256MB, 398MB and 512MB guest RAM case might be useful. >> >> Konrad, is there some way to force swiotlb use even for native to make >> the native test cases more relevant? Or is there some other direction we > > swiotlb=force will do it. I already use this option as specified on wiki: http://wiki.xen.org/xenwiki/XenPCIpassthrough >> should try first? > > <shakes his head> I think we have a good lead. So let's go for testing that way :) >>> It seems that adding this options in /etc/modprobe.d/sky2.conf does not >>> work, neither using /etc/modules > > You could just pass as Linux kernel parameter 'sky2.copybreak=0'. But I am > not sure if that stays if the 'sky2' driver is compiled as module. According to Ian, this will work only with statically compiled driver. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 22:06 ` Jean Baptiste Favre @ 2011-02-02 9:12 ` Ian Campbell 0 siblings, 0 replies; 73+ messages in thread From: Ian Campbell @ 2011-02-02 9:12 UTC (permalink / raw) To: xen-devel On Tue, 2011-02-01 at 22:06 +0000, Jean Baptiste Favre wrote: > Le 01/02/2011 20:37, Konrad Rzeszutek Wilk a écrit : > > On Tue, Feb 01, 2011 at 04:23:09PM +0000, Ian Campbell wrote: > >> Konrad, is there some way to force swiotlb use even for native to make > >> the native test cases more relevant? Or is there some other direction we > > > > swiotlb=force will do it. > I already use this option as specified on wiki: > http://wiki.xen.org/xenwiki/XenPCIpassthrough Yes, we were just confirming that you should also use this option for your native tests. > > >> should try first? > > > > <shakes his head> I think we have a good lead. > So let's go for testing that way :) > > >>> It seems that adding this options in /etc/modprobe.d/sky2.conf does not > >>> work, neither using /etc/modules > > > > You could just pass as Linux kernel parameter 'sky2.copybreak=0'. But I am > > not sure if that stays if the 'sky2' driver is compiled as module. > According to Ian, this will work only with statically compiled driver. Well, I'm only sure it works for a statically compiled driver. I don't know about modules -- depends on whether modprobe groks /proc/cmdline or something, I suspect it does not though. Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 19:37 ` Konrad Rzeszutek Wilk 2011-02-01 22:06 ` Jean Baptiste Favre @ 2011-02-01 23:01 ` Jean Baptiste Favre 2011-02-02 9:44 ` Ian Campbell 1 sibling, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 23:01 UTC (permalink / raw) To: xen-devel Le 01/02/2011 20:37, Konrad Rzeszutek Wilk a écrit : > On Tue, Feb 01, 2011 at 04:23:09PM +0000, Ian Campbell wrote: >> On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote: >>> OK, just found it: >>> after domU boot: >>> - log in >>> - ping -s 86 10.0.0.1 (fails) >>> - rmmod sky2 >>> - modprobe sky2 copybreak=0 (no packet copied) >>> - ping -s 86 10.0.0.1 (works) >>> >>> So it's clearly related to that option. >> >> Awesome! >> >>> Now the question: what am I supposed to do ? >> >> I think the next step is to try and reproduce on native 32 bit, with RAM >> artificially limited via the mem= kernel command in option, this will >> let us determine if this is a generic issue or is somehow Xen specific. >> >> The main difference caused by the copybreak option is that for larger >> frames (i.e. always when copybreak==0) it hits a code path which uses >> pci_map_single and pci_map_page to access the received data. When len < >> copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it >> seems like the later path is broken somehow. If the issue does turn out > > It could also have gotten the direction reverted (the 3c5XX code had it > wrong at some point so..). Might make sense to compile the kernel with > CONFIG_DMA_API_DEBUG which is good at detecting these issues. > >> to be related to Xen then I think that points to the swiotlb code. > >> >> I assume you are not seeing "rx mapping error" in your domU dmesg? Did >> you post a full guest console log at some point? Comparing the logs for >> the 256MB, 398MB and 512MB guest RAM case might be useful. >> >> Konrad, is there some way to force swiotlb use even for native to make >> the native test cases more relevant? Or is there some other direction we > > swiotlb=force will do it. OK, just performed native test. Installed 32bits Debian Squeeze, add 2.6.37 32bits kernel from experimental, setup grub with following options: "mem=256M swiotlb=force" Ping tests work whatever can be packet size. If I understood well what you explain to me, it's now clear that the problem is somewhat Xen related, isn't it ? I'll be able to continue any tests you want tomorrow. Do you still need me to compile domU kernel with "CONFIG_DMA_API_DEBUG" enabled ? If so, what tools will I need to get debug informations ? Now, it's time to sleep here in France :) (or I'll get killed by my girl friend :D ) Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 23:01 ` Jean Baptiste Favre @ 2011-02-02 9:44 ` Ian Campbell 2011-02-02 15:38 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-02 9:44 UTC (permalink / raw) To: xen-devel On Tue, 2011-02-01 at 23:01 +0000, Jean Baptiste Favre wrote: > Le 01/02/2011 20:37, Konrad Rzeszutek Wilk a écrit : > > On Tue, Feb 01, 2011 at 04:23:09PM +0000, Ian Campbell wrote: > >> On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote: > >>> OK, just found it: > >>> after domU boot: > >>> - log in > >>> - ping -s 86 10.0.0.1 (fails) > >>> - rmmod sky2 > >>> - modprobe sky2 copybreak=0 (no packet copied) > >>> - ping -s 86 10.0.0.1 (works) > >>> > >>> So it's clearly related to that option. > >> > >> Awesome! > >> > >>> Now the question: what am I supposed to do ? > >> > >> I think the next step is to try and reproduce on native 32 bit, with RAM > >> artificially limited via the mem= kernel command in option, this will > >> let us determine if this is a generic issue or is somehow Xen specific. > >> > >> The main difference caused by the copybreak option is that for larger > >> frames (i.e. always when copybreak==0) it hits a code path which uses > >> pci_map_single and pci_map_page to access the received data. When len < > >> copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it > >> seems like the later path is broken somehow. If the issue does turn out > > > > It could also have gotten the direction reverted (the 3c5XX code had it > > wrong at some point so..). Might make sense to compile the kernel with > > CONFIG_DMA_API_DEBUG which is good at detecting these issues. > > > >> to be related to Xen then I think that points to the swiotlb code. > > > >> > >> I assume you are not seeing "rx mapping error" in your domU dmesg? Did > >> you post a full guest console log at some point? Comparing the logs for > >> the 256MB, 398MB and 512MB guest RAM case might be useful. > >> > >> Konrad, is there some way to force swiotlb use even for native to make > >> the native test cases more relevant? Or is there some other direction we > > > > swiotlb=force will do it. > OK, just performed native test. > Installed 32bits Debian Squeeze, add 2.6.37 32bits kernel from > experimental, setup grub with following options: > "mem=256M swiotlb=force" > > Ping tests work whatever can be packet size. > > If I understood well what you explain to me, it's now clear that the > problem is somewhat Xen related, isn't it ? It certainly seems that way. I'm not 100% sure that swiotlb=force will have actually made the driver use the swiotlb, it may just have forced the swiotlb to be allocated. Konrad? > I'll be able to continue any tests you want tomorrow. > Do you still need me to compile domU kernel with "CONFIG_DMA_API_DEBUG" > enabled ? Yes please. > If so, what tools will I need to get debug informations ? Uh, Konrad? Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 9:44 ` Ian Campbell @ 2011-02-02 15:38 ` Konrad Rzeszutek Wilk 2011-02-02 15:55 ` Ian Campbell 2011-02-02 15:56 ` Jean Baptiste Favre 0 siblings, 2 replies; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-02-02 15:38 UTC (permalink / raw) To: Jean Baptiste Favre, Ian Campbell; +Cc: xen-devel > > Ping tests work whatever can be packet size. > > > > If I understood well what you explain to me, it's now clear that the > > problem is somewhat Xen related, isn't it ? > > It certainly seems that way. I'm not 100% sure that swiotlb=force will > have actually made the driver use the swiotlb, it may just have forced It should have. > the swiotlb to be allocated. Konrad? Both. It would allocate it and force all DMA operations to go through the bounce buffer. Let me take a look at the driver itself. Ah, so I had a similar card: 3c59x which hit the same type of failure way back in 2.6.31 time. It had the pci_dma_sync_single_for_cpu(.. PCI_DMA_FROMDEVICE) turned around so it would never copy properly. But in my tree (2.6.38) the sky driver looks to be doing just right.. It has PCI_DMA_FROMDEVICE.. Well this looks odd: 2330 skb = netdev_alloc_skb_ip_align(sky2->netdev, length); 2331 if (likely(skb)) { 2332 pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, 2333 length, PCI_DMA_FROMDEVICE); =======> skb_copy_from_linear_data(re->skb, skb->data, length); <==== 2335 skb->ip_summed = re->skb->ip_summed; 2336 skb->csum = re->skb->csum; 2337 pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, 2338 length, PCI_DMA_FROMDEVICE); 2339 re->skb->ip_summed = CHECKSUM_NONE; 2340 skb_put(skb, length); 2341 } It copies from skb->data (just allocated) to re->skb (seems that the re->skb->data has been earlier DMA-mapped). But there is no data in skb->data as we just allocated it? Shouldn't this be the other way around? Like this: diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..37c0631 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -2331,7 +2331,7 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, if (likely(skb)) { pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, length, PCI_DMA_FROMDEVICE); - skb_copy_from_linear_data(re->skb, skb->data, length); + skb_copy_from_linear_data(skb, re->skb->data, length); skb->ip_summed = re->skb->ip_summed; skb->csum = re->skb->csum; pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, > > > I'll be able to continue any tests you want tomorrow. > > Do you still need me to compile domU kernel with "CONFIG_DMA_API_DEBUG" > > enabled ? > > Yes please. > > > If so, what tools will I need to get debug informations ? > > Uh, Konrad? Here, try this patch. Run it under baremetal and Xen and we can compare the outputs .. but I think the issue here is not off mis-using the DMA API but rather a driver bug. /* * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License v2.0 as published by * the Free Software Foundation * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. */ #include <linux/module.h> #include <linux/string.h> #include <linux/types.h> #include <linux/init.h> #include <linux/stat.h> #include <linux/err.h> #include <linux/ctype.h> #include <linux/slab.h> #include <linux/limits.h> #include <linux/device.h> #include <linux/pci.h> #include <linux/blkdev.h> #include <linux/device.h> #include <linux/init.h> #include <linux/mm.h> #include <linux/fcntl.h> #include <linux/slab.h> #include <linux/kmod.h> #include <linux/major.h> #include <linux/smp_lock.h> #include <linux/highmem.h> #include <linux/blkdev.h> #include <linux/module.h> #include <linux/blkpg.h> #include <linux/buffer_head.h> #include <linux/mpage.h> #include <linux/mount.h> #include <linux/uio.h> #include <linux/namei.h> #include <asm/uaccess.h> #include <linux/pagemap.h> #include <linux/pagevec.h> #include <linux/dma-debug.h> #define DUMP_DMA_FUN "0.1" MODULE_AUTHOR("Konrad Rzeszutek Wilk <konrad@virtualiron>"); MODULE_DESCRIPTION("dump dma"); MODULE_LICENSE("GPL"); MODULE_VERSION(DUMP_DMA_FUN); static int __init dump_dma_init(void) { debug_dma_dump_mappings(NULL); return 0; } static void __exit dump_dma_exit(void) { } module_init(dump_dma_init); module_exit(dump_dma_exit); > > Ian. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 15:38 ` Konrad Rzeszutek Wilk @ 2011-02-02 15:55 ` Ian Campbell 2011-02-02 15:56 ` Jean Baptiste Favre 1 sibling, 0 replies; 73+ messages in thread From: Ian Campbell @ 2011-02-02 15:55 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: Jean Baptiste Favre, xen-devel On Wed, 2011-02-02 at 15:38 +0000, Konrad Rzeszutek Wilk wrote: > > > Ping tests work whatever can be packet size. > > > > > > If I understood well what you explain to me, it's now clear that the > > > problem is somewhat Xen related, isn't it ? > > > > It certainly seems that way. I'm not 100% sure that swiotlb=force will > > have actually made the driver use the swiotlb, it may just have forced > > It should have. > > > the swiotlb to be allocated. Konrad? > > Both. It would allocate it and force all DMA operations to go through > the bounce buffer. > > Let me take a look at the driver itself. Ah, so I had a similar card: 3c59x > which hit the same type of failure way back in 2.6.31 time. It had the > pci_dma_sync_single_for_cpu(.. PCI_DMA_FROMDEVICE) turned around so it would never > copy properly. But in my tree (2.6.38) the sky driver looks to be doing just > right.. It has PCI_DMA_FROMDEVICE.. > > Well this looks odd: > > 2330 skb = netdev_alloc_skb_ip_align(sky2->netdev, length); > 2331 if (likely(skb)) { > 2332 pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, > 2333 length, PCI_DMA_FROMDEVICE); > =======> skb_copy_from_linear_data(re->skb, skb->data, length); <==== > 2335 skb->ip_summed = re->skb->ip_summed; > 2336 skb->csum = re->skb->csum; > 2337 pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, > 2338 length, PCI_DMA_FROMDEVICE); > 2339 re->skb->ip_summed = CHECKSUM_NONE; > 2340 skb_put(skb, length); > 2341 } > > It copies from skb->data (just allocated) to re->skb (seems that the re->skb->data > has been earlier DMA-mapped). > > But there is no data in skb->data as we just allocated it? Shouldn't this > be the other way around? Like this: skb_copy_from_linear_data's arguments are the other way round to what you may be expecting: static inline void skb_copy_from_linear_data(const struct sk_buff *skb, void *to, const unsigned int len) So the above line is correctly copying from re->skb->data to skb->data not the other way round. I stared at that exact line for ages this morning ;-) Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 15:38 ` Konrad Rzeszutek Wilk 2011-02-02 15:55 ` Ian Campbell @ 2011-02-02 15:56 ` Jean Baptiste Favre 1 sibling, 0 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-02 15:56 UTC (permalink / raw) To: xen-devel > >> Ping tests work whatever can be packet size. > >> > >> If I understood well what you explain to me, it's now clear that the > >> problem is somewhat Xen related, isn't it ? > > > > It certainly seems that way. I'm not 100% sure that swiotlb=force will > > have actually made the driver use the swiotlb, it may just have forced > > It should have. > > > the swiotlb to be allocated. Konrad? > > Both. It would allocate it and force all DMA operations to go through > the bounce buffer. > > Let me take a look at the driver itself. Ah, so I had a similar card: > 3c59x > which hit the same type of failure way back in 2.6.31 time. It had the > pci_dma_sync_single_for_cpu(.. PCI_DMA_FROMDEVICE) turned around so it > would never > copy properly. But in my tree (2.6.38) the sky driver looks to be > doing just > right.. It has PCI_DMA_FROMDEVICE.. > > Well this looks odd: > > 2330 skb = netdev_alloc_skb_ip_align(sky2->netdev, length); > 2331 if (likely(skb)) { > 2332 pci_dma_sync_single_for_cpu(sky2->hw->pdev, > re->data_addr, > 2333 length, > PCI_DMA_FROMDEVICE); > =======> skb_copy_from_linear_data(re->skb, skb->data, > length); <==== > 2335 skb->ip_summed = re->skb->ip_summed; > 2336 skb->csum = re->skb->csum; > 2337 pci_dma_sync_single_for_device(sky2->hw->pdev, > re->data_addr, > 2338 length, > PCI_DMA_FROMDEVICE); > 2339 re->skb->ip_summed = CHECKSUM_NONE; > 2340 skb_put(skb, length); > 2341 } > > It copies from skb->data (just allocated) to re->skb (seems that the > re->skb->data > has been earlier DMA-mapped). > > But there is no data in skb->data as we just allocated it? Shouldn't this > be the other way around? Like this: > > > diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c > index 7d85a38..37c0631 100644 > --- a/drivers/net/sky2.c > +++ b/drivers/net/sky2.c > @@ -2331,7 +2331,7 @@ static struct sk_buff *receive_copy(struct > sky2_port *sky2, > if (likely(skb)) { > pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, > length, PCI_DMA_FROMDEVICE); > - skb_copy_from_linear_data(re->skb, skb->data, length); > + skb_copy_from_linear_data(skb, re->skb->data, length); > skb->ip_summed = re->skb->ip_summed; > skb->csum = re->skb->csum; > pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, > But, how could a "simple" driver bug automagically appear with Xen and not as Baremetal ? And why only with 32bits kernels ? BTW, I'll try your patch as soon as I can. > > > >> I'll be able to continue any tests you want tomorrow. > >> Do you still need me to compile domU kernel with "CONFIG_DMA_API_DEBUG" > >> enabled ? > > > > Yes please. > > > >> If so, what tools will I need to get debug informations ? > > > > Uh, Konrad? > > Here, try this patch. Run it under baremetal and Xen and we can > compare the > outputs .. but I think the issue here is not off mis-using the DMA API but > rather a driver bug. OK, so if you first patch does not solve problem, I'll compile this one as kernel module and load it, both on 32bits Baremetal and in 32bits DomU Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 16:23 ` Ian Campbell 2011-02-01 19:37 ` Konrad Rzeszutek Wilk @ 2011-02-01 22:04 ` Jean Baptiste Favre 2011-02-02 9:27 ` Ian Campbell 1 sibling, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 22:04 UTC (permalink / raw) To: xen-devel Le 01/02/2011 17:23, Ian Campbell a écrit : > On Tue, 2011-02-01 at 15:38 +0000, Jean Baptiste Favre wrote: >> OK, just found it: >> after domU boot: >> - log in >> - ping -s 86 10.0.0.1 (fails) >> - rmmod sky2 >> - modprobe sky2 copybreak=0 (no packet copied) >> - ping -s 86 10.0.0.1 (works) >> >> So it's clearly related to that option. > > Awesome! > >> Now the question: what am I supposed to do ? > > I think the next step is to try and reproduce on native 32 bit, with RAM > artificially limited via the mem= kernel command in option, this will > let us determine if this is a generic issue or is somehow Xen specific. OK, so I'll install a 32bits Debian Squeeze with 2.6.37 32bits kernel to check it. > The main difference caused by the copybreak option is that for larger > frames (i.e. always when copybreak==0) it hits a code path which uses > pci_map_single and pci_map_page to access the received data. When len < > copybreak it takes a path which uses pci_dma_sync_single_for_cpu, so it > seems like the later path is broken somehow. If the issue does turn out > to be related to Xen then I think that points to the swiotlb code. > > I assume you are not seeing "rx mapping error" in your domU dmesg? Did > you post a full guest console log at some point? Comparing the logs for > the 256MB, 398MB and 512MB guest RAM case might be useful. No sure I've ever posted that logs. But I can redo my tests :) > Konrad, is there some way to force swiotlb use even for native to make > the native test cases more relevant? Or is there some other direction we > should try first? > >> It seems that adding this options in /etc/modprobe.d/sky2.conf does not >> work, neither using /etc/modules > > That will be a userspace issue in your distro, perhaps with openwrt they > are not expected to work that way. All tests were done with Debian. As the module is included in initramfs and not on domU, since I've no kernel installed in the domU, module is loaded before system FS, so /etc/modprobe.d/sky2.conf is not taken into account. I think that the best solution here is to compile kernel with static sky2 driver inclusion and use kernel commandline to pass sky2 copybreak option. Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 22:04 ` Jean Baptiste Favre @ 2011-02-02 9:27 ` Ian Campbell 2011-02-02 10:24 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-02 9:27 UTC (permalink / raw) To: xen-devel On Tue, 2011-02-01 at 22:04 +0000, Jean Baptiste Favre wrote: > Le 01/02/2011 17:23, Ian Campbell a écrit : > > I assume you are not seeing "rx mapping error" in your domU dmesg? Did > > you post a full guest console log at some point? Comparing the logs for > > the 256MB, 398MB and 512MB guest RAM case might be useful. > No sure I've ever posted that logs. But I can redo my tests :) yes, please do that. Please can you also collect and post the information from ifconfig and ethtool -S which I asked for earlier. Can you confirm that on the domU tcpdump shows no incoming frames at all, including no corrupted or strange frames. It has been suggested that the frames could be being received ok but contain garbage (e.g. due to swiotlb not syncing the memory properly) and hence are not ICMP Responses but appear as some random frame type. Please can you use "tcpdump -w" on both the gateway/peer side and the affected domU to capture a short session while the failing ping is running and post (or link to if large) the resulting pcap files. On the peer side you can use "ether host <domU-MAC>" (plus optionally "or ether broadcast") on the tcpdump command line to limit the capture to just the one peer and reduce the size of the capture. Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 9:27 ` Ian Campbell @ 2011-02-02 10:24 ` Jean Baptiste Favre 2011-02-02 10:59 ` Ian Campbell 2011-02-02 17:42 ` Konrad Rzeszutek Wilk 0 siblings, 2 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-02 10:24 UTC (permalink / raw) To: xen-devel [-- Attachment #1: Type: text/plain, Size: 3064 bytes --] Hello Ian, My domU config file: # cat /cluster/xen/xps-106.cfg kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' builder = 'linux' memory = '398' vcpus = '1' cpus = '2' localtime = 0 serial = 'pty' boot = 'cdn' disk = [ 'drbd:xps-106,xvda,w' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty" pci = [ '04:00.0' ] name = 'xps-106' hostname = 'xps-106.clichy.jbfavre.org' Le 02/02/2011 10:27, Ian Campbell a écrit : > On Tue, 2011-02-01 at 22:04 +0000, Jean Baptiste Favre wrote: > > Le 01/02/2011 17:23, Ian Campbell a écrit : > > >> I assume you are not seeing "rx mapping error" in your domU dmesg? Did > >> you post a full guest console log at some point? Comparing the logs for > >> the 256MB, 398MB and 512MB guest RAM case might be useful. > > No sure I've ever posted that logs. But I can redo my tests :) > > yes, please do that. Please find attached both console startup logs with 256M and 512M: 256M_domU_console_logs.txt 512M_domU_console_logs.txt For 512M, I saw some kernel CallTrace I can not explain. There are not present with 256M. For 398M memory, I can't even start domU : # xm create /cluster/xen/xps-106.cfg -c Using config file "/cluster/xen/xps-106.cfg". [215739.007871] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. Started domain xps-106 (id=23) (XEN) mm.c:798:d23 Non-privileged (23) attempt to map I/O space 00000000 (XEN) mm.c:4644:d23 ptwr_emulate: could not get_page_from_l1e() As I told you, I'm still using Debian 2.6.37 kernel because I've some problem to compile 2.6.32.27 from Jeremy's git repository. I hope I can get it compiled today so I'll be able to test with that kernel as well. > > Please can you also collect and post the information from ifconfig and > ethtool -S which I asked for earlier. Attached as well: 256_domU_ifconfig_ping_ethtool.txt 512_domU_ifconfig_ping_ethtool.txt > Can you confirm that on the domU tcpdump shows no incoming frames at > all, including no corrupted or strange frames. It has been suggested > that the frames could be being received ok but contain garbage (e.g. due > to swiotlb not syncing the memory properly) and hence are not ICMP > Responses but appear as some random frame type. > > Please can you use "tcpdump -w" on both the gateway/peer side and the > affected domU to capture a short session while the failing ping is > running and post (or link to if large) the resulting pcap files. On the > peer side you can use "ether host <domU-MAC>" (plus optionally "or ether > broadcast") on the tcpdump command line to limit the capture to just the > one peer and reduce the size of the capture. I'll update you as soon as I have tcpdump captures ready. Regards, JB [-- Attachment #2: 512M_domU_ifconfig_ping_ethtool.txt --] [-- Type: text/plain, Size: 2936 bytes --] # ifconfig eth0 Link encap:Ethernet HWaddr 00:1f:c6:eb:71:43 inet addr:10.0.0.106 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::21f:c6ff:feeb:7143/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:120 (120.0 B) TX bytes:468 (468.0 B) Interrupt:18 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) # ping -c5 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. 64 bytes from 10.0.0.1: icmp_req=1 ttl=64 time=0.823 ms 64 bytes from 10.0.0.1: icmp_req=2 ttl=64 time=0.856 ms 64 bytes from 10.0.0.1: icmp_req=3 ttl=64 time=0.831 ms 64 bytes from 10.0.0.1: icmp_req=4 ttl=64 time=0.824 ms 64 bytes from 10.0.0.1: icmp_req=5 ttl=64 time=0.830 ms --- 10.0.0.1 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 3998ms rtt min/avg/max/mdev = 0.823/0.832/0.856/0.038 ms # ping -c5 -s86 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 86(114) bytes of data. 94 bytes from 10.0.0.1: icmp_req=1 ttl=64 time=1.25 ms 94 bytes from 10.0.0.1: icmp_req=2 ttl=64 time=0.861 ms 94 bytes from 10.0.0.1: icmp_req=3 ttl=64 time=0.895 ms 94 bytes from 10.0.0.1: icmp_req=4 ttl=64 time=0.868 ms 94 bytes from 10.0.0.1: icmp_req=5 ttl=64 time=0.864 ms --- 10.0.0.1 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 3998ms rtt min/avg/max/mdev = 0.861/0.949/1.259/0.157 ms # ethtool -S eth0 NIC statistics: tx_bytes: 5651 rx_bytes: 78580 tx_broadcast: 1 rx_broadcast: 5 tx_multicast: 6 rx_multicast: 0 tx_unicast: 60 rx_unicast: 75 tx_mac_pause: 0 rx_mac_pause: 0 collisions: 0 late_collision: 0 aborted: 0 single_collisions: 0 multi_collisions: 0 rx_short: 0 rx_runt: 0 rx_64_byte_packets: 9 rx_65_to_127_byte_packets: 12 rx_128_to_255_byte_packets: 9 rx_256_to_511_byte_packets: 0 rx_512_to_1023_byte_packets: 0 rx_1024_to_1518_byte_packets: 50 rx_1518_to_max_byte_packets: 0 rx_too_long: 0 rx_fifo_overflow: 0 rx_jabber: 0 rx_fcs_error: 0 tx_64_byte_packets: 4 tx_65_to_127_byte_packets: 55 tx_128_to_255_byte_packets: 8 tx_256_to_511_byte_packets: 0 tx_512_to_1023_byte_packets: 0 tx_1024_to_1518_byte_packets: 0 tx_1519_to_max_byte_packets: 0 tx_fifo_underrun: 0 [-- Attachment #3: 256M_domU_console_logs.txt --] [-- Type: text/plain, Size: 12426 bytes --] # xm create /cluster/xen/xps-106.cfg -c Using config file "/cluster/xen/xps-106.cfg". [213362.865921] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. Started domain xps-106 (id=16) [ 0.000000] Reserving virtual address space above 0xf5800000 [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.37-trunk-686-bigmem (Debian 2.6.37-1~experimental.1) (ben@decadent.org.uk) (gcc version 4.4.5 (Debian 4.4.5-10) ) #1 SMP Thu Jan 6 15:21:44 UTC 2011 [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] released 0 pages of unused memory [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] (XEN) traps.c:2308:d16 Domain attempted WRMSR 00000000c0010004 from 0000c0b2:cbfa8461 to 00000000:0000abcd. Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 0000000010800000 (usable) [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] last_pfn = 0x10800 max_arch_pfn = 0x1000000 [ 0.000000] init_memory_mapping: 0000000000000000-0000000010800000 [ 0.000000] RAMDISK: 01793000 - 02ecc000 [ 0.000000] 0MB HIGHMEM available. [ 0.000000] 264MB LOWMEM available. [ 0.000000] mapped low ram: 0 - 10800000 [ 0.000000] low ram: 0 - 10800000 [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] Normal 0x00001000 -> 0x00010800 [ 0.000000] HighMem empty [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x000000a0 [ 0.000000] 0: 0x00000100 -> 0x00010800 [ 0.000000] Using APIC driver default [ 0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org [ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs [ 0.000000] Local APIC disabled by BIOS -- you can enable it with "lapic" [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [ 0.000000] Allocating PCI resources starting at 10800000 (gap: 10800000:ef800000) [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.0.1 (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:1 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 13 pages/cpu @cfddf000 s30336 r0 d22912 u53248 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 66944 [ 0.000000] Kernel command line: root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty [ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes) [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) [ 0.000000] Initializing CPU#0 [ 0.000000] Placing 64MB software IO TLB between cbd7ee40 - cfd7ee40 [ 0.000000] software IO TLB at phys 0xbd7ee40 - 0xfd7ee40 [ 0.000000] Initializing HighMem for node 0 (00000000:00000000) [ 0.000000] Memory: 163268k/270336k available (2703k kernel code, 106620k reserved, 1410k data, 412k init, 0k highmem) [ 0.000000] virtual kernel memory layout: [ 0.000000] fixmap : 0xf5536000 - 0xf57ff000 (2852 kB) [ 0.000000] pkmap : 0xf5200000 - 0xf5400000 (2048 kB) [ 0.000000] vmalloc : 0xd1000000 - 0xf51fe000 ( 577 MB) [ 0.000000] lowmem : 0xc0000000 - 0xd0800000 ( 264 MB) [ 0.000000] .init : 0xc1405000 - 0xc146c000 ( 412 kB) [ 0.000000] .data : 0xc12a3d40 - 0xc14046e0 (1410 kB) [ 0.000000] .text : 0xc1000000 - 0xc12a3d40 (2703 kB) [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU dyntick-idle grace-period acceleration is enabled. [ 0.000000] RCU-based detection of stalled CPUs is disabled. [ 0.000000] NR_IRQS:1280 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [hvc0] enabled [ 0.000000] installing Xen timer for CPU 0 [ 0.000000] Detected 2506.222 MHz processor. [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 5012.44 BogoMIPS (lpj=10024888) [ 0.004000] pid_max: default: 32768 minimum: 301 [ 0.004000] Security Framework initialized [ 0.004000] SELinux: Disabled at boot. [ 0.004000] Mount-cache hash table entries: 512 [ 0.004000] Initializing cgroup subsys ns [ 0.004000] ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup. [ 0.004000] Initializing cgroup subsys cpuacct [ 0.004000] Initializing cgroup subsys devices [ 0.004000] Initializing cgroup subsys freezer [ 0.004000] Initializing cgroup subsys net_cls [ 0.004000] Initializing cgroup subsys blkio [ 0.004000] CPU: Physical Processor ID: 0 [ 0.004000] CPU: Processor Core ID: 0 [ 0.004000] Performance Events: [ 0.004000] no APIC, boot with the "lapic" boot parameter to force-enable it. [ 0.004000] no hardware sampling interrupt available. [ 0.004000] Broken PMU hardware detected, software events only. [ 0.004000] SMP alternatives: switching to UP code [ 0.008000] Freeing SMP alternatives: 16k freed [ 0.008246] NMI watchdog disabled for cpu0: unable to create perf event: -2 [ 0.008354] Brought up 1 CPUs [ 0.008444] devtmpfs: initialized [ 0.008790] Grant table initialized [ 0.008797] regulator: core version 0.5 [ 0.008851] regulator: dummy: [ 0.008900] NET: Registered protocol family 16 [ 0.009099] PCI: setting up Xen PCI frontend stub [ 0.009099] bio: create slab <bio-0> at 0 [ 0.009099] ACPI: Interpreter disabled. [ 0.009099] xen_balloon: Initialising balloon driver. [ 0.009099] last_pfn = 0x10800 max_arch_pfn = 0x1000000 [ 0.009099] vgaarb: loaded [ 0.009099] PCI: System does not support PCI [ 0.009099] PCI: System does not support PCI [ 0.009099] Switching to clocksource xen [ 0.009443] pnp: PnP ACPI: disabled [ 0.009452] PnPBIOS: Disabled [ 0.011710] NET: Registered protocol family 2 [ 0.011757] IP route cache hash table entries: 4096 (order: 2, 16384 bytes) [ 0.011940] TCP established hash table entries: 16384 (order: 5, 131072 bytes) [ 0.012014] TCP bind hash table entries: 16384 (order: 5, 131072 bytes) [ 0.012075] TCP: Hash tables configured (established 16384 bind 16384) [ 0.012082] TCP reno registered [ 0.012088] UDP hash table entries: 256 (order: 1, 8192 bytes) [ 0.012098] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes) [ 0.012164] NET: Registered protocol family 1 [ 0.012221] Unpacking initramfs... [ 0.068739] Freeing initrd memory: 23780k freed [ 0.076752] platform rtc_cmos: registered platform RTC device (no PNP device found) [ 0.076997] audit: initializing netlink socket (disabled) [ 0.077014] type=2000 audit(1296639642.987:1): initialized [ 0.085690] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.087728] VFS: Disk quotas dquot_6.5.2 [ 0.087800] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) [ 0.087899] msgmni has been set to 365 [ 0.088100] alg: No test for stdrng (krng) [ 0.088169] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) [ 0.088178] io scheduler noop registered [ 0.088183] io scheduler deadline registered [ 0.088199] io scheduler cfq registered (default) [ 0.088422] isapnp: Scanning for PnP cards... [ 0.804965] isapnp: No Plug & Play device found [ 0.810882] Linux agpgart interface v0.103 [ 0.810978] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.811280] PNP: No PS/2 controller found. Probing ports directly. [ 0.812099] i8042.c: No controller found. [ 0.812196] mice: PS/2 mouse device common for all mice [ 0.812315] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [ 0.812339] pcifront pci-0: Installing PCI frontend [ 0.812380] rtc_cmos: probe of rtc_cmos failed with error -38 [ 0.812407] cpuidle: using governor ladder [ 0.812413] cpuidle: using governor menu [ 0.812690] pcifront pci-0: Creating PCI Frontend Bus 0000:00 [ 0.816145] pcifront pci-0: claiming resource 0000:00:00.0/0 [ 0.816145] pcifront pci-0: claiming resource 0000:00:00.0/2 [ 0.818358] TCP cubic registered [ 0.818508] NET: Registered protocol family 10 [ 0.818840] lo: Disabled Privacy Extensions [ 0.819043] Mobile IPv6 [ 0.819050] NET: Registered protocol family 17 [ 0.819058] Registering the dns_resolver key type [ 0.819084] Using IPI No-Shortcut mode [ 0.819211] registered taskstats version 1 [ 0.916059] XENBUS: Device with no driver: device/vbd/51712 [ 0.916074] XENBUS: Device with no driver: device/console/0 [ 0.916101] /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0) [ 0.916134] Initalizing network drop monitor service [ 0.916238] Freeing unused kernel memory: 412k freed [ 0.917011] Write protecting the kernel text: 2704k [ 0.917255] Write protecting the kernel read-only data: 1016k Loading, please wait... [ 0.976345] udev[46]: starting version 164 [ 1.105266] sky2: driver version 1.28 [ 1.121166] sky2 0000:00:00.0: BAR 0: set to [mem 0xfeb00000-0xfeb03fff 64bit] (PCI address [0xfeb00000-0xfeb03fff]) [ 1.121258] sky2 0000:00:00.0: BAR 2: set to [io 0xe800-0xe8ff] (PCI address [0xe800-0xe8ff]) [ 1.121330] sky2 0000:00:00.0: enabling device (0000 -> 0003) [ 1.121454] sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 [ 1.121649] sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 [ 1.122871] sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 [ 1.133172] blkfront: xvda: barriers enabled [ 1.134906] xvda: xvda1 xvda2 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... [ 1.340290] device-mapper: uevent: version 1.0.3 [ 1.340867] device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com done. Begin: Running /scripts/local-premount ... done. [ 1.579694] EXT3-fs: barriers not enabled [ 1.585025] kjournald starting. Commit interval 5 seconds [ 1.585063] EXT3-fs (dm-0): mounted filesystem with ordered data mode Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd[ 2.575753] udev[167]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated...[ 2.904843] input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. [ 6.253299] EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended [ 6.284848] EXT3-fs (dm-0): using internal journal Loading kernel modules...done. Cleaning up ifupdown.... Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. Configuring network interfaces...[ 6.831911] sky2 0000:00:00.0: eth0: enabling interface [ 6.831997] ADDRCONF(NETDEV_UP): eth0: link is not ready done. Cleaning up temporary files.... INIT: Entering runlevel: 2 Using makefile-style concurrent boot in runlevel 2. Starting enhanced syslogd: rsyslogd. Starting periodic command scheduler: cron. Debian GNU/Linux 6.0 xps-106 hvc0 xps-106 login: [ 9.298080] sky2 0000:00:00.0: eth0: Link is up at 1000 Mbps, full duplex, flow control both [ 9.298150] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [-- Attachment #4: 256M_domU_ifconfig_ping_ethtool.txt --] [-- Type: text/plain, Size: 3787 bytes --] # ifconfig eth0 Link encap:Ethernet HWaddr 00:1f:c6:eb:71:43 inet addr:10.0.0.106 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::21f:c6ff:feeb:7143/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:6 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:468 (468.0 B) Interrupt:18 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) # ping -c5 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. From 10.0.0.106 icmp_seq=1 Destination Host Unreachable From 10.0.0.106 icmp_seq=2 Destination Host Unreachable From 10.0.0.106 icmp_seq=3 Destination Host Unreachable From 10.0.0.106 icmp_seq=4 Destination Host Unreachable From 10.0.0.106 icmp_seq=5 Destination Host Unreachable --- 10.0.0.1 ping statistics --- 5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4023ms pipe 3 # ping -c5 -s86 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 86(114) bytes of data. From 10.0.0.106 icmp_seq=1 Destination Host Unreachable From 10.0.0.106 icmp_seq=2 Destination Host Unreachable From 10.0.0.106 icmp_seq=3 Destination Host Unreachable From 10.0.0.106 icmp_seq=4 Destination Host Unreachable From 10.0.0.106 icmp_seq=5 Destination Host Unreachable --- 10.0.0.1 ping statistics --- 5 packets transmitted, 0 received, +5 errors, 100% packet loss, time 4024ms pipe 3 # ifconfig eth0 Link encap:Ethernet HWaddr 00:1f:c6:eb:71:43 inet addr:10.0.0.106 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::21f:c6ff:feeb:7143/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:14 errors:0 dropped:14 overruns:0 frame:0 TX packets:18 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:840 (840.0 B) TX bytes:972 (972.0 B) Interrupt:18 lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:10 errors:0 dropped:0 overruns:0 frame:0 TX packets:10 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1270 (1.2 KiB) TX bytes:1270 (1.2 KiB) # ethtool -S eth0 NIC statistics: tx_bytes: 1260 rx_bytes: 896 tx_broadcast: 12 rx_broadcast: 2 tx_multicast: 6 rx_multicast: 0 tx_unicast: 0 rx_unicast: 12 tx_mac_pause: 0 rx_mac_pause: 0 collisions: 0 late_collision: 0 aborted: 0 single_collisions: 0 multi_collisions: 0 rx_short: 0 rx_runt: 0 rx_64_byte_packets: 14 rx_65_to_127_byte_packets: 0 rx_128_to_255_byte_packets: 0 rx_256_to_511_byte_packets: 0 rx_512_to_1023_byte_packets: 0 rx_1024_to_1518_byte_packets: 0 rx_1518_to_max_byte_packets: 0 rx_too_long: 0 rx_fifo_overflow: 0 rx_jabber: 0 rx_fcs_error: 0 tx_64_byte_packets: 12 tx_65_to_127_byte_packets: 6 tx_128_to_255_byte_packets: 0 tx_256_to_511_byte_packets: 0 tx_512_to_1023_byte_packets: 0 tx_1024_to_1518_byte_packets: 0 tx_1519_to_max_byte_packets: 0 tx_fifo_underrun: 0 [-- Attachment #5: 512M_domU_console_logs.txt --] [-- Type: text/plain, Size: 16412 bytes --] # xm create /cluster/xen/xps-106.cfg -c Using config file "/cluster/xen/xps-106.cfg". [214817.006300] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. Started domain xps-106 (id=21) (XEN) traps.c:2308:d21 Domain attempted WRMSR 00000000c0010004 from 0000c0b2:cbfa8461 to 00000000:0000abcd. [ 0.000000] Reserving virtual address space above 0xf5800000 [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.37-trunk-686-bigmem (Debian 2.6.37-1~experimental.1) (ben@decadent.org.uk) (gcc version 4.4.5 (Debian 4.4.5-10) ) #1 SMP Thu Jan 6 15:21:44 UTC 2011 [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] released 0 pages of unused memory [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 0000000020800000 (usable) [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] last_pfn = 0x20800 max_arch_pfn = 0x1000000 [ 0.000000] init_memory_mapping: 0000000000000000-0000000020800000 [ 0.000000] RAMDISK: 01793000 - 02ecc000 [ 0.000000] 0MB HIGHMEM available. [ 0.000000] 520MB LOWMEM available. [ 0.000000] mapped low ram: 0 - 20800000 [ 0.000000] low ram: 0 - 20800000 [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] Normal 0x00001000 -> 0x00020800 [ 0.000000] HighMem empty [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x000000a0 [ 0.000000] 0: 0x00000100 -> 0x00020800 [ 0.000000] Using APIC driver default [ 0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org [ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs [ 0.000000] Local APIC disabled by BIOS -- you can enable it with "lapic" [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [ 0.000000] Allocating PCI resources starting at 20800000 (gap: 20800000:df800000) [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.0.1 (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:1 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 13 pages/cpu @dfbde000 s30336 r0 d22912 u53248 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 131968 [ 0.000000] Kernel command line: root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes) [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Initializing CPU#0 [ 0.000000] Placing 64MB software IO TLB between dbb1a000 - dfb1a000 [ 0.000000] software IO TLB at phys 0x1bb1a000 - 0x1fb1a000 [ 0.000000] Initializing HighMem for node 0 (00000000:00000000) [ 0.000000] Memory: 422200k/532480k available (2703k kernel code, 109832k reserved, 1410k data, 412k init, 0k highmem) [ 0.000000] virtual kernel memory layout: [ 0.000000] fixmap : 0xf5536000 - 0xf57ff000 (2852 kB) [ 0.000000] pkmap : 0xf5200000 - 0xf5400000 (2048 kB) [ 0.000000] vmalloc : 0xe1000000 - 0xf51fe000 ( 321 MB) [ 0.000000] lowmem : 0xc0000000 - 0xe0800000 ( 520 MB) [ 0.000000] .init : 0xc1405000 - 0xc146c000 ( 412 kB) [ 0.000000] .data : 0xc12a3d40 - 0xc14046e0 (1410 kB) [ 0.000000] .text : 0xc1000000 - 0xc12a3d40 (2703 kB) [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU dyntick-idle grace-period acceleration is enabled. [ 0.000000] RCU-based detection of stalled CPUs is disabled. [ 0.000000] NR_IRQS:1280 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [hvc0] enabled [ 0.000000] installing Xen timer for CPU 0 [ 0.000000] Detected 2506.222 MHz processor. [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 5012.44 BogoMIPS (lpj=10024888) [ 0.004000] pid_max: default: 32768 minimum: 301 [ 0.004000] Security Framework initialized [ 0.004000] SELinux: Disabled at boot. [ 0.004000] Mount-cache hash table entries: 512 [ 0.004000] Initializing cgroup subsys ns [ 0.004000] ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup. [ 0.004000] Initializing cgroup subsys cpuacct [ 0.004000] Initializing cgroup subsys devices [ 0.004000] Initializing cgroup subsys freezer [ 0.004000] Initializing cgroup subsys net_cls [ 0.004000] Initializing cgroup subsys blkio [ 0.004000] CPU: Physical Processor ID: 0 [ 0.004000] CPU: Processor Core ID: 0 [ 0.004000] Performance Events: [ 0.004000] no APIC, boot with the "lapic" boot parameter to force-enable it. [ 0.004000] no hardware sampling interrupt available. [ 0.004000] Broken PMU hardware detected, software events only. [ 0.004000] SMP alternatives: switching to UP code [ 0.010317] Freeing SMP alternatives: 16k freed [ 0.010560] NMI watchdog disabled for cpu0: unable to create perf event: -2 [ 0.010645] Brought up 1 CPUs [ 0.010735] devtmpfs: initialized [ 0.011056] Grant table initialized [ 0.011063] regulator: core version 0.5 [ 0.011122] regulator: dummy: [ 0.011169] NET: Registered protocol family 16 [ 0.011366] PCI: setting up Xen PCI frontend stub [ 0.012488] bio: create slab <bio-0> at 0 [ 0.012557] ACPI: Interpreter disabled. [ 0.012582] xen_balloon: Initialising balloon driver. [ 0.012582] last_pfn = 0x20800 max_arch_pfn = 0x1000000 [ 0.012582] vgaarb: loaded [ 0.012582] PCI: System does not support PCI [ 0.012582] PCI: System does not support PCI [ 0.012582] Switching to clocksource xen [ 0.013423] pnp: PnP ACPI: disabled [ 0.013430] PnPBIOS: Disabled [ 0.015641] NET: Registered protocol family 2 [ 0.015690] IP route cache hash table entries: 8192 (order: 3, 32768 bytes) [ 0.015896] TCP established hash table entries: 32768 (order: 6, 262144 bytes) [ 0.016029] TCP bind hash table entries: 32768 (order: 6, 262144 bytes) [ 0.016148] TCP: Hash tables configured (established 32768 bind 32768) [ 0.016155] TCP reno registered [ 0.016161] UDP hash table entries: 512 (order: 2, 16384 bytes) [ 0.016176] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes) [ 0.016308] NET: Registered protocol family 1 [ 0.016366] Unpacking initramfs... [ 0.072960] Freeing initrd memory: 23780k freed [ 0.080894] platform rtc_cmos: registered platform RTC device (no PNP device found) [ 0.081131] audit: initializing netlink socket (disabled) [ 0.081148] type=2000 audit(1296641097.317:1): initialized [ 0.089493] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.091528] VFS: Disk quotas dquot_6.5.2 [ 0.091595] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) [ 0.091696] msgmni has been set to 871 [ 0.091893] alg: No test for stdrng (krng) [ 0.091961] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) [ 0.091970] io scheduler noop registered [ 0.091976] io scheduler deadline registered [ 0.091992] io scheduler cfq registered (default) [ 0.152319] isapnp: Scanning for PnP cards... [ 0.856077] isapnp: No Plug & Play device found [ 0.861733] Linux agpgart interface v0.103 [ 0.861829] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.862138] PNP: No PS/2 controller found. Probing ports directly. [ 0.862960] i8042.c: No controller found. [ 0.863034] mice: PS/2 mouse device common for all mice [ 0.863161] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [ 0.863210] rtc_cmos: probe of rtc_cmos failed with error -38 [ 0.863235] cpuidle: using governor ladder [ 0.863242] cpuidle: using governor menu [ 0.863432] pcifront pci-0: Installing PCI frontend [ 0.863498] TCP cubic registered [ 0.863646] pcifront pci-0: Creating PCI Frontend Bus 0000:00 [ 0.881473] NET: Registered protocol family 10 [ 0.881816] lo: Disabled Privacy Extensions [ 0.882024] Mobile IPv6 [ 0.882031] NET: Registered protocol family 17 [ 0.882039] Registering the dns_resolver key type [ 0.882058] Using IPI No-Shortcut mode [ 0.882190] registered taskstats version 1 [ 0.884859] pcifront pci-0: claiming resource 0000:00:00.0/0 [ 0.884868] pcifront pci-0: claiming resource 0000:00:00.0/2 [ 0.980061] XENBUS: Device with no driver: device/vbd/51712 [ 0.980075] XENBUS: Device with no driver: device/console/0 [ 0.980100] /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0) [ 0.980132] Initalizing network drop monitor service [ 0.980232] Freeing unused kernel memory: 412k freed [ 0.981036] Write protecting the kernel text: 2704k [ 0.981303] Write protecting the kernel read-only data: 1016k Loading, please wait... [ 1.060454] udev[46]: starting version 164 [ 1.183225] sky2: driver version 1.28 [ 1.183333] sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 [ 1.183528] sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 [ 1.184784] sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 [ 1.193834] blkfront: xvda: barriers enabled [ 1.195628] xvda: xvda1 xvda2 [ 1.349422] ------------[ cut here ]------------ [ 1.349443] WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:337 blk_start_queue+0x1d/0x31() [ 1.349457] Modules linked in: sky2 xen_blkfront [ 1.349471] Pid: 0, comm: swapper Not tainted 2.6.37-trunk-686-bigmem #1 [ 1.349478] Call Trace: [ 1.349487] [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [ 1.349494] [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [ 1.349502] [<c1036023>] ? warn_slowpath_null+0xd/0x10 [ 1.349509] [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [ 1.349519] [<e102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [ 1.349529] [<e102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [ 1.349542] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [ 1.349550] [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [ 1.349559] [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [ 1.349567] [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [ 1.349577] [<c1009117>] ? xen_do_upcall+0x7/0xc [ 1.349584] [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [ 1.349591] [<c1006246>] ? xen_safe_halt+0xf/0x1b [ 1.349598] [<c100e276>] ? default_idle+0x3f/0x5d [ 1.349605] [<c1007dc4>] ? cpu_idle+0x98/0xdc [ 1.349613] [<c14058a0>] ? start_kernel+0x320/0x325 [ 1.349620] [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [ 1.349627] [<c1409046>] ? trap_init+0x1fe/0x367 [ 1.349633] ---[ end trace c57c5a433d71a17e ]--- [ 1.349638] ------------[ cut here ]------------ [ 1.349645] WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:245 blk_remove_plug+0x1d/0x78() [ 1.349657] Modules linked in: sky2 xen_blkfront [ 1.349667] Pid: 0, comm: swapper Tainted: G W 2.6.37-trunk-686-bigmem #1 [ 1.349674] Call Trace: [ 1.349682] [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [ 1.349689] [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [ 1.349696] [<c1036023>] ? warn_slowpath_null+0xd/0x10 [ 1.349702] [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [ 1.349709] [<c1139ed1>] ? __blk_run_queue+0x8/0x5b [ 1.349716] [<e102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [ 1.349725] [<e102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [ 1.349733] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [ 1.349743] [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [ 1.349750] [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [ 1.349758] [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [ 1.349764] [<c1009117>] ? xen_do_upcall+0x7/0xc [ 1.349773] [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [ 1.349780] [<c1006246>] ? xen_safe_halt+0xf/0x1b [ 1.349788] [<c100e276>] ? default_idle+0x3f/0x5d [ 1.349795] [<c1007dc4>] ? cpu_idle+0x98/0xdc [ 1.349801] [<c14058a0>] ? start_kernel+0x320/0x325 [ 1.349809] [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [ 1.349816] [<c1409046>] ? trap_init+0x1fe/0x367 [ 1.349824] ---[ end trace c57c5a433d71a17f ]--- Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... [ 1.507113] device-mapper: uevent: version 1.0.3 [ 1.507692] device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com done. Begin: Running /scripts/local-premount ... done. [ 1.644939] EXT3-fs: barriers not enabled [ 1.657461] kjournald starting. Commit interval 5 seconds [ 1.657498] EXT3-fs (dm-0): mounted filesystem with ordered data mode Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd[ 2.746299] udev[156]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated...[ 3.080564] input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. [ 6.062837] EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended [ 6.098388] EXT3-fs (dm-0): using internal journal Loading kernel modules...done. Cleaning up ifupdown.... Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. [ 6.652343] sky2 0000:00:00.0: eth0: enabling interface [ 6.652423] ADDRCONF(NETDEV_UP): eth0: link is not ready Configuring network interfaces...done. Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd[ 2.746299] udev[156]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated...[ 3.080564] input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. [ 6.062837] EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended [ 6.098388] EXT3-fs (dm-0): using internal journal Loading kernel modules...done. Cleaning up ifupdown.... Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. [ 6.652343] sky2 0000:00:00.0: eth0: enabling interface [ 6.652423] ADDRCONF(NETDEV_UP): eth0: link is not ready Configuring network interfaces...done. Cleaning up temporary files.... INIT: Entering runlevel: 2 Using makefile-style concurrent boot in runlevel 2. Starting enhanced syslogd: rsyslogd. Starting periodic command scheduler: cron. Debian GNU/Linux 6.0 xps-106 hvc0 xps-106 login: [ 9.148520] sky2 0000:00:00.0: eth0: Link is up at 1000 Mbps, full duplex, flow control both [ 9.148605] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [-- Attachment #6: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 10:24 ` Jean Baptiste Favre @ 2011-02-02 10:59 ` Ian Campbell 2011-02-02 11:33 ` Jean Baptiste Favre 2011-02-02 17:42 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-02 10:59 UTC (permalink / raw) To: xen-devel [-- Attachment #1: Type: text/plain, Size: 5354 bytes --] On Wed, 2011-02-02 at 10:24 +0000, Jean Baptiste Favre wrote: > Hello Ian, > > My domU config file: > > # cat /cluster/xen/xps-106.cfg > kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' > ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' > builder = 'linux' > memory = '398' > vcpus = '1' > cpus = '2' > localtime = 0 > serial = 'pty' > boot = 'cdn' > disk = [ 'drbd:xps-106,xvda,w' ] > on_poweroff = 'destroy' > on_reboot = 'restart' > on_crash = 'restart' > extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force > console=hvc0 xencons=tty" > pci = [ '04:00.0' ] > name = 'xps-106' > hostname = 'xps-106.clichy.jbfavre.org' > > > Le 02/02/2011 10:27, Ian Campbell a écrit : > > On Tue, 2011-02-01 at 22:04 +0000, Jean Baptiste Favre wrote: > > > Le 01/02/2011 17:23, Ian Campbell a écrit : > > > > >> I assume you are not seeing "rx mapping error" in your domU dmesg? Did > > >> you post a full guest console log at some point? Comparing the logs for > > >> the 256MB, 398MB and 512MB guest RAM case might be useful. > > > No sure I've ever posted that logs. But I can redo my tests :) > > > > yes, please do that. > Please find attached both console startup logs with 256M and 512M: > 256M_domU_console_logs.txt > 512M_domU_console_logs.txt Thanks, I stripped the timestamps and cleaned up some differences due to missing carriage returns, those versions are attached. Interesting bits of diff (- == 256, + == 512) are: -Allocating PCI resources starting at 10800000 (gap: 10800000:ef800000) +Allocating PCI resources starting at 20800000 (gap: 20800000:df800000) So the PCI resources are higher and smaller in the 512MB case. -Placing 64MB software IO TLB between cbd7ee40 - cfd7ee40 -software IO TLB at phys 0xbd7ee40 - 0xfd7ee40 +Placing 64MB software IO TLB between dbb1a000 - dfb1a000 +software IO TLB at phys 0x1bb1a000 - 0x1fb1a000 Fair enough? sky2: driver version 1.28 -sky2 0000:00:00.0: BAR 0: set to [mem 0xfeb00000-0xfeb03fff 64bit] (PCI address [0xfeb00000-0xfeb03fff]) -sky2 0000:00:00.0: BAR 2: set to [io 0xe800-0xe8ff] (PCI address [0xe800-0xe8ff]) -sky2 0000:00:00.0: enabling device (0000 -> 0003) sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 So there is some sort of remapping going on in the 256MB case? (or the 512MB logs are missing a bit, which can happen). Can you post the content of domU's /proc/{interrupts,iomem,ioports} for both cases? > > > > Please can you also collect and post the information from ifconfig and > > ethtool -S which I asked for earlier. > Attached as well: > 256_domU_ifconfig_ping_ethtool.txt > 512_domU_ifconfig_ping_ethtool.txt Thanks. The interesting bit here is that the 256MB case is registering RX dropped frames. 512_domU_ifconfig_ping_ethtool.txt did not include an ifconfig after the experiment but I assume the dropped frames are not present there. The rx_dropped statistic is only cranked in a small number of places, once in the driver and a handful in the networking core, only one of the core cases looks likely to be relevant. The following patch should help us figure out where the frames are dropped... Ian. diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..645d9e9 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -2339,6 +2339,7 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, re->skb->ip_summed = CHECKSUM_NONE; skb_put(skb, length); } + WARN(skb == NULL, "sky2: receive_copy failed netdev_alloc_skb_ip_align length %u\n", length); return skb; } @@ -2386,10 +2387,15 @@ static struct sk_buff *receive_new(struct sky2_port *sky2, nre.skb = sky2_rx_alloc(sky2); if (unlikely(!nre.skb)) + { + WARN(1, "sky2: receive_new failed sky2_rx_alloc\n"); goto nobuf; - + } if (sky2_rx_map_skb(sky2->hw->pdev, &nre, hdr_space)) + { + WARN(1, "sky2: receive_new failed sky2_rx_map_skb\n"); goto nomap; + } skb = re->skb; sky2_rx_unmap_skb(sky2->hw->pdev, re); diff --git a/net/core/dev.c b/net/core/dev.c index 54277df..9c99612 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1516,6 +1516,7 @@ int dev_forward_skb(struct net_device *dev, struct sk_buff *skb) if (unlikely(!(dev->flags & IFF_UP) || (skb->len > (dev->mtu + dev->hard_header_len + VLAN_HLEN)))) { + printk(KERN_CRIT "dev_forward_skb dropping skb\n"); atomic_long_inc(&dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -2700,6 +2701,7 @@ enqueue: local_irq_restore(flags); + printk(KERN_CRIT "enqueue_to_backlog dropping skb\n"); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -3125,6 +3127,7 @@ ncls: if (pt_prev) { ret = pt_prev->func(skb, skb->dev, pt_prev, orig_dev); } else { + printk(KERN_CRIT "__netif_receive_skb dropping skb proto %#x\n", type); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); /* Jamal, now you will not able to escape explaining [-- Attachment #2: 512M_domU_console_logs.txtX --] [-- Type: text/plain, Size: 12631 bytes --] # xm create /cluster/xen/xps-106.cfg -c Using config file "/cluster/xen/xps-106.cfg". [214817.006300] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. Started domain xps-106 (id=21) (XEN) traps.c:2308:d21 Domain attempted WRMSR 00000000c0010004 from 0000c0b2:cbfa8461 to 00000000:0000abcd. Reserving virtual address space above 0xf5800000 Initializing cgroup subsys cpuset Initializing cgroup subsys cpu Linux version 2.6.37-trunk-686-bigmem (Debian 2.6.37-1~experimental.1) (ben@decadent.org.uk) (gcc version 4.4.5 (Debian 4.4.5-10) ) #1 SMP Thu Jan 6 15:21:44 UTC 2011 ACPI in unprivileged domain disabled released 0 pages of unused memory BIOS-provided physical RAM map: Xen: 0000000000000000 - 00000000000a0000 (usable) Xen: 00000000000a0000 - 0000000000100000 (reserved) Xen: 0000000000100000 - 0000000020800000 (usable) NX (Execute Disable) protection: active DMI not present or invalid. last_pfn = 0x20800 max_arch_pfn = 0x1000000 init_memory_mapping: 0000000000000000-0000000020800000 RAMDISK: 01793000 - 02ecc000 0MB HIGHMEM available. 520MB LOWMEM available. mapped low ram: 0 - 20800000 low ram: 0 - 20800000 Zone PFN ranges: DMA 0x00000010 -> 0x00001000 Normal 0x00001000 -> 0x00020800 HighMem empty Movable zone start PFN for each node early_node_map[2] active PFN ranges 0: 0x00000010 -> 0x000000a0 0: 0x00000100 -> 0x00020800 Using APIC driver default SFI: Simple Firmware Interface v0.81 http://simplefirmware.org SMP: Allowing 1 CPUs, 0 hotplug CPUs Local APIC disabled by BIOS -- you can enable it with "lapic" APIC: disable apic facility APIC: switched to apic NOOP PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 Allocating PCI resources starting at 20800000 (gap: 20800000:df800000) Booting paravirtualized kernel on Xen Xen version: 4.0.1 (preserve-AD) setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:1 nr_node_ids:1 PERCPU: Embedded 13 pages/cpu @dfbde000 s30336 r0 d22912 u53248 Built 1 zonelists in Zone order, mobility grouping on. Total pages: 131968 Kernel command line: root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty PID hash table entries: 4096 (order: 2, 16384 bytes) Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) Initializing CPU#0 Placing 64MB software IO TLB between dbb1a000 - dfb1a000 software IO TLB at phys 0x1bb1a000 - 0x1fb1a000 Initializing HighMem for node 0 (00000000:00000000) Memory: 422200k/532480k available (2703k kernel code, 109832k reserved, 1410k data, 412k init, 0k highmem) virtual kernel memory layout: fixmap : 0xf5536000 - 0xf57ff000 (2852 kB) pkmap : 0xf5200000 - 0xf5400000 (2048 kB) vmalloc : 0xe1000000 - 0xf51fe000 ( 321 MB) lowmem : 0xc0000000 - 0xe0800000 ( 520 MB) .init : 0xc1405000 - 0xc146c000 ( 412 kB) .data : 0xc12a3d40 - 0xc14046e0 (1410 kB) .text : 0xc1000000 - 0xc12a3d40 (2703 kB) SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Hierarchical RCU implementation. RCU dyntick-idle grace-period acceleration is enabled. RCU-based detection of stalled CPUs is disabled. NR_IRQS:1280 Console: colour dummy device 80x25 console [tty0] enabled console [hvc0] enabled installing Xen timer for CPU 0 Detected 2506.222 MHz processor. Calibrating delay loop (skipped), value calculated using timer frequency.. 5012.44 BogoMIPS (lpj=10024888) pid_max: default: 32768 minimum: 301 Security Framework initialized SELinux: Disabled at boot. Mount-cache hash table entries: 512 Initializing cgroup subsys ns ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup. Initializing cgroup subsys cpuacct Initializing cgroup subsys devices Initializing cgroup subsys freezer Initializing cgroup subsys net_cls Initializing cgroup subsys blkio CPU: Physical Processor ID: 0 CPU: Processor Core ID: 0 Performance Events: no APIC, boot with the "lapic" boot parameter to force-enable it. no hardware sampling interrupt available. Broken PMU hardware detected, software events only. SMP alternatives: switching to UP code Freeing SMP alternatives: 16k freed NMI watchdog disabled for cpu0: unable to create perf event: -2 Brought up 1 CPUs devtmpfs: initialized Grant table initialized regulator: core version 0.5 regulator: dummy: NET: Registered protocol family 16 PCI: setting up Xen PCI frontend stub bio: create slab <bio-0> at 0 ACPI: Interpreter disabled. xen_balloon: Initialising balloon driver. last_pfn = 0x20800 max_arch_pfn = 0x1000000 vgaarb: loaded PCI: System does not support PCI PCI: System does not support PCI Switching to clocksource xen pnp: PnP ACPI: disabled PnPBIOS: Disabled NET: Registered protocol family 2 IP route cache hash table entries: 8192 (order: 3, 32768 bytes) TCP established hash table entries: 32768 (order: 6, 262144 bytes) TCP bind hash table entries: 32768 (order: 6, 262144 bytes) TCP: Hash tables configured (established 32768 bind 32768) TCP reno registered UDP hash table entries: 512 (order: 2, 16384 bytes) UDP-Lite hash table entries: 512 (order: 2, 16384 bytes) NET: Registered protocol family 1 Unpacking initramfs... Freeing initrd memory: 23780k freed platform rtc_cmos: registered platform RTC device (no PNP device found) audit: initializing netlink socket (disabled) type=2000 audit(1296641097.317:1): initialized HugeTLB registered 2 MB page size, pre-allocated 0 pages VFS: Disk quotas dquot_6.5.2 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) msgmni has been set to 871 alg: No test for stdrng (krng) Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) io scheduler noop registered io scheduler deadline registered io scheduler cfq registered (default) isapnp: Scanning for PnP cards... isapnp: No Plug & Play device found Linux agpgart interface v0.103 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled PNP: No PS/2 controller found. Probing ports directly. i8042.c: No controller found. mice: PS/2 mouse device common for all mice rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 rtc_cmos: probe of rtc_cmos failed with error -38 cpuidle: using governor ladder cpuidle: using governor menu pcifront pci-0: Installing PCI frontend TCP cubic registered pcifront pci-0: Creating PCI Frontend Bus 0000:00 NET: Registered protocol family 10 lo: Disabled Privacy Extensions Mobile IPv6 NET: Registered protocol family 17 Registering the dns_resolver key type Using IPI No-Shortcut mode registered taskstats version 1 pcifront pci-0: claiming resource 0000:00:00.0/0 pcifront pci-0: claiming resource 0000:00:00.0/2 XENBUS: Device with no driver: device/vbd/51712 XENBUS: Device with no driver: device/console/0 /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0) Initalizing network drop monitor service Freeing unused kernel memory: 412k freed Write protecting the kernel text: 2704k Write protecting the kernel read-only data: 1016k Loading, please wait... udev[46]: starting version 164 sky2: driver version 1.28 sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 blkfront: xvda: barriers enabled xvda: xvda1 xvda2 ------------[ cut here ]------------ WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:337 blk_start_queue+0x1d/0x31() Modules linked in: sky2 xen_blkfront Pid: 0, comm: swapper Not tainted 2.6.37-trunk-686-bigmem #1 Call Trace: [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [<c1036023>] ? warn_slowpath_null+0xd/0x10 [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [<e102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [<e102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [<c1009117>] ? xen_do_upcall+0x7/0xc [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [<c1006246>] ? xen_safe_halt+0xf/0x1b [<c100e276>] ? default_idle+0x3f/0x5d [<c1007dc4>] ? cpu_idle+0x98/0xdc [<c14058a0>] ? start_kernel+0x320/0x325 [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [<c1409046>] ? trap_init+0x1fe/0x367 ---[ end trace c57c5a433d71a17e ]--- ------------[ cut here ]------------ WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:245 blk_remove_plug+0x1d/0x78() Modules linked in: sky2 xen_blkfront Pid: 0, comm: swapper Tainted: G W 2.6.37-trunk-686-bigmem #1 Call Trace: [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [<c1036023>] ? warn_slowpath_null+0xd/0x10 [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [<c1139ed1>] ? __blk_run_queue+0x8/0x5b [<e102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [<e102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [<c1009117>] ? xen_do_upcall+0x7/0xc [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [<c1006246>] ? xen_safe_halt+0xf/0x1b [<c100e276>] ? default_idle+0x3f/0x5d [<c1007dc4>] ? cpu_idle+0x98/0xdc [<c14058a0>] ? start_kernel+0x320/0x325 [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [<c1409046>] ? trap_init+0x1fe/0x367 ---[ end trace c57c5a433d71a17f ]--- Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com done. Begin: Running /scripts/local-premount ... done. EXT3-fs: barriers not enabled kjournald starting. Commit interval 5 seconds EXT3-fs (dm-0): mounted filesystem with ordered data mode Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd udev[156]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated... input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended EXT3-fs (dm-0): using internal journal Loading kernel modules...done. Cleaning up ifupdown.... Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. sky2 0000:00:00.0: eth0: enabling interface ADDRCONF(NETDEV_UP): eth0: link is not ready Configuring network interfaces...done. Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd udev[156]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated... input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended EXT3-fs (dm-0): using internal journal Loading kernel modules...done. Cleaning up ifupdown.... Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. sky2 0000:00:00.0: eth0: enabling interface ADDRCONF(NETDEV_UP): eth0: link is not ready Configuring network interfaces...done. Cleaning up temporary files.... INIT: Entering runlevel: 2 Using makefile-style concurrent boot in runlevel 2. Starting enhanced syslogd: rsyslogd. Starting periodic command scheduler: cron. Debian GNU/Linux 6.0 xps-106 hvc0 xps-106 login: sky2 0000:00:00.0: eth0: Link is up at 1000 Mbps, full duplex, flow control both ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [-- Attachment #3: 256M_domU_console_logs.txtX --] [-- Type: text/plain, Size: 9495 bytes --] # xm create /cluster/xen/xps-106.cfg -c Using config file "/cluster/xen/xps-106.cfg". [213362.865921] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. Started domain xps-106 (id=16) Reserving virtual address space above 0xf5800000 Initializing cgroup subsys cpuset Initializing cgroup subsys cpu Linux version 2.6.37-trunk-686-bigmem (Debian 2.6.37-1~experimental.1) (ben@decadent.org.uk) (gcc version 4.4.5 (Debian 4.4.5-10) ) #1 SMP Thu Jan 6 15:21:44 UTC 2011 ACPI in unprivileged domain disabled released 0 pages of unused memory BIOS-provided physical RAM map: (XEN) traps.c:2308:d16 Domain attempted WRMSR 00000000c0010004 from 0000c0b2:cbfa8461 to 00000000:0000abcd. Xen: 0000000000000000 - 00000000000a0000 (usable) Xen: 00000000000a0000 - 0000000000100000 (reserved) Xen: 0000000000100000 - 0000000010800000 (usable) NX (Execute Disable) protection: active DMI not present or invalid. last_pfn = 0x10800 max_arch_pfn = 0x1000000 init_memory_mapping: 0000000000000000-0000000010800000 RAMDISK: 01793000 - 02ecc000 0MB HIGHMEM available. 264MB LOWMEM available. mapped low ram: 0 - 10800000 low ram: 0 - 10800000 Zone PFN ranges: DMA 0x00000010 -> 0x00001000 Normal 0x00001000 -> 0x00010800 HighMem empty Movable zone start PFN for each node early_node_map[2] active PFN ranges 0: 0x00000010 -> 0x000000a0 0: 0x00000100 -> 0x00010800 Using APIC driver default SFI: Simple Firmware Interface v0.81 http://simplefirmware.org SMP: Allowing 1 CPUs, 0 hotplug CPUs Local APIC disabled by BIOS -- you can enable it with "lapic" APIC: disable apic facility APIC: switched to apic NOOP PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 Allocating PCI resources starting at 10800000 (gap: 10800000:ef800000) Booting paravirtualized kernel on Xen Xen version: 4.0.1 (preserve-AD) setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:1 nr_node_ids:1 PERCPU: Embedded 13 pages/cpu @cfddf000 s30336 r0 d22912 u53248 Built 1 zonelists in Zone order, mobility grouping on. Total pages: 66944 Kernel command line: root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty PID hash table entries: 2048 (order: 1, 8192 bytes) Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) Initializing CPU#0 Placing 64MB software IO TLB between cbd7ee40 - cfd7ee40 software IO TLB at phys 0xbd7ee40 - 0xfd7ee40 Initializing HighMem for node 0 (00000000:00000000) Memory: 163268k/270336k available (2703k kernel code, 106620k reserved, 1410k data, 412k init, 0k highmem) virtual kernel memory layout: fixmap : 0xf5536000 - 0xf57ff000 (2852 kB) pkmap : 0xf5200000 - 0xf5400000 (2048 kB) vmalloc : 0xd1000000 - 0xf51fe000 ( 577 MB) lowmem : 0xc0000000 - 0xd0800000 ( 264 MB) .init : 0xc1405000 - 0xc146c000 ( 412 kB) .data : 0xc12a3d40 - 0xc14046e0 (1410 kB) .text : 0xc1000000 - 0xc12a3d40 (2703 kB) SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 Hierarchical RCU implementation. RCU dyntick-idle grace-period acceleration is enabled. RCU-based detection of stalled CPUs is disabled. NR_IRQS:1280 Console: colour dummy device 80x25 console [tty0] enabled console [hvc0] enabled installing Xen timer for CPU 0 Detected 2506.222 MHz processor. Calibrating delay loop (skipped), value calculated using timer frequency.. 5012.44 BogoMIPS (lpj=10024888) pid_max: default: 32768 minimum: 301 Security Framework initialized SELinux: Disabled at boot. Mount-cache hash table entries: 512 Initializing cgroup subsys ns ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup. Initializing cgroup subsys cpuacct Initializing cgroup subsys devices Initializing cgroup subsys freezer Initializing cgroup subsys net_cls Initializing cgroup subsys blkio CPU: Physical Processor ID: 0 CPU: Processor Core ID: 0 Performance Events: no APIC, boot with the "lapic" boot parameter to force-enable it. no hardware sampling interrupt available. Broken PMU hardware detected, software events only. SMP alternatives: switching to UP code Freeing SMP alternatives: 16k freed NMI watchdog disabled for cpu0: unable to create perf event: -2 Brought up 1 CPUs devtmpfs: initialized Grant table initialized regulator: core version 0.5 regulator: dummy: NET: Registered protocol family 16 PCI: setting up Xen PCI frontend stub bio: create slab <bio-0> at 0 ACPI: Interpreter disabled. xen_balloon: Initialising balloon driver. last_pfn = 0x10800 max_arch_pfn = 0x1000000 vgaarb: loaded PCI: System does not support PCI PCI: System does not support PCI Switching to clocksource xen pnp: PnP ACPI: disabled PnPBIOS: Disabled NET: Registered protocol family 2 IP route cache hash table entries: 4096 (order: 2, 16384 bytes) TCP established hash table entries: 16384 (order: 5, 131072 bytes) TCP bind hash table entries: 16384 (order: 5, 131072 bytes) TCP: Hash tables configured (established 16384 bind 16384) TCP reno registered UDP hash table entries: 256 (order: 1, 8192 bytes) UDP-Lite hash table entries: 256 (order: 1, 8192 bytes) NET: Registered protocol family 1 Unpacking initramfs... Freeing initrd memory: 23780k freed platform rtc_cmos: registered platform RTC device (no PNP device found) audit: initializing netlink socket (disabled) type=2000 audit(1296639642.987:1): initialized HugeTLB registered 2 MB page size, pre-allocated 0 pages VFS: Disk quotas dquot_6.5.2 Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) msgmni has been set to 365 alg: No test for stdrng (krng) Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) io scheduler noop registered io scheduler deadline registered io scheduler cfq registered (default) isapnp: Scanning for PnP cards... isapnp: No Plug & Play device found Linux agpgart interface v0.103 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled PNP: No PS/2 controller found. Probing ports directly. i8042.c: No controller found. mice: PS/2 mouse device common for all mice rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 pcifront pci-0: Installing PCI frontend rtc_cmos: probe of rtc_cmos failed with error -38 cpuidle: using governor ladder cpuidle: using governor menu pcifront pci-0: Creating PCI Frontend Bus 0000:00 pcifront pci-0: claiming resource 0000:00:00.0/0 pcifront pci-0: claiming resource 0000:00:00.0/2 TCP cubic registered NET: Registered protocol family 10 lo: Disabled Privacy Extensions Mobile IPv6 NET: Registered protocol family 17 Registering the dns_resolver key type Using IPI No-Shortcut mode registered taskstats version 1 XENBUS: Device with no driver: device/vbd/51712 XENBUS: Device with no driver: device/console/0 /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0) Initalizing network drop monitor service Freeing unused kernel memory: 412k freed Write protecting the kernel text: 2704k Write protecting the kernel read-only data: 1016k Loading, please wait... udev[46]: starting version 164 sky2: driver version 1.28 sky2 0000:00:00.0: BAR 0: set to [mem 0xfeb00000-0xfeb03fff 64bit] (PCI address [0xfeb00000-0xfeb03fff]) sky2 0000:00:00.0: BAR 2: set to [io 0xe800-0xe8ff] (PCI address [0xe800-0xe8ff]) sky2 0000:00:00.0: enabling device (0000 -> 0003) sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 blkfront: xvda: barriers enabled xvda: xvda1 xvda2 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com done. Begin: Running /scripts/local-premount ... done. EXT3-fs: barriers not enabled kjournald starting. Commit interval 5 seconds EXT3-fs (dm-0): mounted filesystem with ordered data mode Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd udev[167]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated... input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended EXT3-fs (dm-0): using internal journal Loading kernel modules...done. Cleaning up ifupdown.... Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. Configuring network interfaces... sky2 0000:00:00.0: eth0: enabling interface ADDRCONF(NETDEV_UP): eth0: link is not ready done. Cleaning up temporary files.... INIT: Entering runlevel: 2 Using makefile-style concurrent boot in runlevel 2. Starting enhanced syslogd: rsyslogd. Starting periodic command scheduler: cron. Debian GNU/Linux 6.0 xps-106 hvc0 xps-106 login: sky2 0000:00:00.0: eth0: Link is up at 1000 Mbps, full duplex, flow control both ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [-- Attachment #4: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 10:59 ` Ian Campbell @ 2011-02-02 11:33 ` Jean Baptiste Favre 0 siblings, 0 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-02 11:33 UTC (permalink / raw) To: xen-devel [-- Attachment #1: Type: text/plain, Size: 2831 bytes --] Hello, Le 02/02/2011 11:59, Ian Campbell a écrit : > [...] > >>> No sure I've ever posted that logs. But I can redo my tests :) > >> > >> yes, please do that. > > Please find attached both console startup logs with 256M and 512M: > > 256M_domU_console_logs.txt > > 512M_domU_console_logs.txt > > Thanks, I stripped the timestamps and cleaned up some differences due to > missing carriage returns, those versions are attached. > > Interesting bits of diff (- == 256, + == 512) are: > -Allocating PCI resources starting at 10800000 (gap: > 10800000:ef800000) > +Allocating PCI resources starting at 20800000 (gap: > 20800000:df800000) > > So the PCI resources are higher and smaller in the 512MB case. > > -Placing 64MB software IO TLB between cbd7ee40 - cfd7ee40 > -software IO TLB at phys 0xbd7ee40 - 0xfd7ee40 > +Placing 64MB software IO TLB between dbb1a000 - dfb1a000 > +software IO TLB at phys 0x1bb1a000 - 0x1fb1a000 > > Fair enough? > > sky2: driver version 1.28 > -sky2 0000:00:00.0: BAR 0: set to [mem 0xfeb00000-0xfeb03fff > 64bit] (PCI address [0xfeb00000-0xfeb03fff]) > -sky2 0000:00:00.0: BAR 2: set to [io 0xe800-0xe8ff] (PCI > address [0xe800-0xe8ff]) > -sky2 0000:00:00.0: enabling device (0000 -> 0003) > sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 > sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 > sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 > > So there is some sort of remapping going on in the 256MB case? (or the > 512MB logs are missing a bit, which can happen). Can you post the > content of domU's /proc/{interrupts,iomem,ioports} for both cases? Please find them attached. I also join last full console logs for both 256M and 512M: I saw "Call Trace" for 256M I did not had last time. > >> Please can you also collect and post the information from ifconfig and > >> ethtool -S which I asked for earlier. > > Attached as well: > > 256_domU_ifconfig_ping_ethtool.txt > > 512_domU_ifconfig_ping_ethtool.txt > > Thanks. The interesting bit here is that the 256MB case is registering > RX dropped frames. 512_domU_ifconfig_ping_ethtool.txt did not include an > ifconfig after the experiment but I assume the dropped frames are not > present there. I did not include second ifconfig for 512M as ping always works with 512M memory. As far as I can remember, I did not noticed anything special. > The rx_dropped statistic is only cranked in a small number of places, > once in the driver and a handful in the networking core, only one of the > core cases looks likely to be relevant. The following patch should help > us figure out where the frames are dropped... Let's patch Debian 2.6.37 32bits kernel so :) Regards, JB [-- Attachment #2: 256M_domU_proc_outputs.txt --] [-- Type: text/plain, Size: 1733 bytes --] # cat /proc/interrupts CPU0 55: 8 xen-pirq-pcifront-msi sky2@pci:0000:00:00.0 1271: 946 xen-dyn-event blkif 1272: 133 xen-dyn-event hvc_console 1273: 0 xen-dyn-event pcifront 1274: 259 xen-dyn-event xenbus 1275: 0 xen-percpu-ipi callfuncsingle0 1276: 0 xen-percpu-virq debug0 1277: 0 xen-percpu-ipi callfunc0 1278: 0 xen-percpu-ipi resched0 1279: 761 xen-percpu-virq timer0 NMI: 0 Non-maskable interrupts LOC: 0 Local timer interrupts SPU: 0 Spurious interrupts PMI: 0 Performance monitoring interrupts IWI: 0 IRQ work interrupts RES: 0 Rescheduling interrupts CAL: 0 Function call interrupts TLB: 0 TLB shootdowns TRM: 0 Thermal event interrupts THR: 0 Threshold APIC interrupts MCE: 0 Machine check exceptions MCP: 0 Machine check polls ERR: 0 MIS: 0 # cat /proc/iomem 00000000-0000ffff : reserved 00010000-0009ffff : System RAM 000a0000-000fffff : reserved 000f0000-000fffff : System ROM 00100000-107fffff : System RAM 01000000-012a3d3f : Kernel code 012a3d40-014046df : Kernel data 01470000-015215ab : Kernel bss feb00000-feb03fff : 0000:00:00.0 feb00000-feb03fff : sky2 # cat /proc/ioports 0000-001f : dma1 0020-0021 : pic1 0040-0043 : timer0 0050-0053 : timer1 0060-0060 : keyboard 0064-0064 : keyboard 0070-0071 : rtc_cmos 0080-008f : dma page reg 00a0-00a1 : pic2 00c0-00df : dma2 00f0-00ff : fpu e800-e8ff : 0000:00:00.0 e800-e8ff : sky2 [-- Attachment #3: 512M_domU_console_logs_2.txt --] [-- Type: text/plain, Size: 15584 bytes --] # xm create /cluster/xen/xps-106.cfg -c Using config file "/cluster/xen/xps-106.cfg". [219194.082018] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. Started domain xps-106 (id=25) (XEN) traps.c:2308:d25 Domain attempted WRMSR 00000000c0010004 from 0000c0b2:cbfa8461 to 00000000:0000abcd. [ 0.000000] Reserving virtual address space above 0xf5800000 [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.37-trunk-686-bigmem (Debian 2.6.37-1~experimental.1) (ben@decadent.org.uk) (gcc version 4.4.5 (Debian 4.4.5-10) ) #1 SMP Thu Jan 6 15:21:44 UTC 2011 [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] released 0 pages of unused memory [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 0000000020800000 (usable) [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] last_pfn = 0x20800 max_arch_pfn = 0x1000000 [ 0.000000] init_memory_mapping: 0000000000000000-0000000020800000 [ 0.000000] RAMDISK: 01793000 - 02ecc000 [ 0.000000] 0MB HIGHMEM available. [ 0.000000] 520MB LOWMEM available. [ 0.000000] mapped low ram: 0 - 20800000 [ 0.000000] low ram: 0 - 20800000 [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] Normal 0x00001000 -> 0x00020800 [ 0.000000] HighMem empty [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x000000a0 [ 0.000000] 0: 0x00000100 -> 0x00020800 [ 0.000000] Using APIC driver default [ 0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org [ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs [ 0.000000] Local APIC disabled by BIOS -- you can enable it with "lapic" [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [ 0.000000] Allocating PCI resources starting at 20800000 (gap: 20800000:df800000) [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.0.1 (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:1 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 13 pages/cpu @dfbde000 s30336 r0 d22912 u53248 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 131968 [ 0.000000] Kernel command line: root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes) [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Initializing CPU#0 [ 0.000000] Placing 64MB software IO TLB between dbb1a000 - dfb1a000 [ 0.000000] software IO TLB at phys 0x1bb1a000 - 0x1fb1a000 [ 0.000000] Initializing HighMem for node 0 (00000000:00000000) [ 0.000000] Memory: 422200k/532480k available (2703k kernel code, 109832k reserved, 1410k data, 412k init, 0k highmem) [ 0.000000] virtual kernel memory layout: [ 0.000000] fixmap : 0xf5536000 - 0xf57ff000 (2852 kB) [ 0.000000] pkmap : 0xf5200000 - 0xf5400000 (2048 kB) [ 0.000000] vmalloc : 0xe1000000 - 0xf51fe000 ( 321 MB) [ 0.000000] lowmem : 0xc0000000 - 0xe0800000 ( 520 MB) [ 0.000000] .init : 0xc1405000 - 0xc146c000 ( 412 kB) [ 0.000000] .data : 0xc12a3d40 - 0xc14046e0 (1410 kB) [ 0.000000] .text : 0xc1000000 - 0xc12a3d40 (2703 kB) [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU dyntick-idle grace-period acceleration is enabled. [ 0.000000] RCU-based detection of stalled CPUs is disabled. [ 0.000000] NR_IRQS:1280 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [hvc0] enabled [ 0.000000] installing Xen timer for CPU 0 [ 0.000000] Detected 2506.222 MHz processor. [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 5012.44 BogoMIPS (lpj=10024888) [ 0.004000] pid_max: default: 32768 minimum: 301 [ 0.004000] Security Framework initialized [ 0.004000] SELinux: Disabled at boot. [ 0.004000] Mount-cache hash table entries: 512 [ 0.004000] Initializing cgroup subsys ns [ 0.004000] ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup. [ 0.004000] Initializing cgroup subsys cpuacct [ 0.004000] Initializing cgroup subsys devices [ 0.004000] Initializing cgroup subsys freezer [ 0.004000] Initializing cgroup subsys net_cls [ 0.004000] Initializing cgroup subsys blkio [ 0.004000] CPU: Physical Processor ID: 0 [ 0.004000] CPU: Processor Core ID: 0 [ 0.004000] Performance Events: [ 0.004000] no APIC, boot with the "lapic" boot parameter to force-enable it. [ 0.004000] no hardware sampling interrupt available. [ 0.004000] Broken PMU hardware detected, software events only. [ 0.004000] SMP alternatives: switching to UP code [ 0.014434] Freeing SMP alternatives: 16k freed [ 0.014650] NMI watchdog disabled for cpu0: unable to create perf event: -2 [ 0.014726] Brought up 1 CPUs [ 0.014806] devtmpfs: initialized [ 0.015100] Grant table initialized [ 0.015107] regulator: core version 0.5 [ 0.015161] regulator: dummy: [ 0.015207] NET: Registered protocol family 16 [ 0.015410] PCI: setting up Xen PCI frontend stub [ 0.016503] bio: create slab <bio-0> at 0 [ 0.016570] ACPI: Interpreter disabled. [ 0.016596] xen_balloon: Initialising balloon driver. [ 0.016596] last_pfn = 0x20800 max_arch_pfn = 0x1000000 [ 0.016596] vgaarb: loaded [ 0.016596] PCI: System does not support PCI [ 0.016596] PCI: System does not support PCI [ 0.016596] Switching to clocksource xen [ 0.017420] pnp: PnP ACPI: disabled [ 0.017427] PnPBIOS: Disabled [ 0.019619] NET: Registered protocol family 2 [ 0.019667] IP route cache hash table entries: 8192 (order: 3, 32768 bytes) [ 0.019847] TCP established hash table entries: 32768 (order: 6, 262144 bytes) [ 0.019964] TCP bind hash table entries: 32768 (order: 6, 262144 bytes) [ 0.020091] TCP: Hash tables configured (established 32768 bind 32768) [ 0.020098] TCP reno registered [ 0.020104] UDP hash table entries: 512 (order: 2, 16384 bytes) [ 0.020119] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes) [ 0.020217] NET: Registered protocol family 1 [ 0.020275] Unpacking initramfs... [ 0.046676] Freeing initrd memory: 23780k freed [ 0.054687] platform rtc_cmos: registered platform RTC device (no PNP device found) [ 0.054923] audit: initializing netlink socket (disabled) [ 0.054940] type=2000 audit(1296645473.960:1): initialized [ 0.063287] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.065699] VFS: Disk quotas dquot_6.5.2 [ 0.065764] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) [ 0.065859] msgmni has been set to 871 [ 0.066056] alg: No test for stdrng (krng) [ 0.066121] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) [ 0.066130] io scheduler noop registered [ 0.066136] io scheduler deadline registered [ 0.066153] io scheduler cfq registered (default) [ 0.163208] isapnp: Scanning for PnP cards... [ 0.515462] isapnp: No Plug & Play device found [ 0.520929] Linux agpgart interface v0.103 [ 0.521028] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.521307] PNP: No PS/2 controller found. Probing ports directly. [ 0.522128] i8042.c: No controller found. [ 0.522198] mice: PS/2 mouse device common for all mice [ 0.522291] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [ 0.522342] rtc_cmos: probe of rtc_cmos failed with error -38 [ 0.522368] cpuidle: using governor ladder [ 0.522373] cpuidle: using governor menu [ 0.522627] pcifront pci-0: Installing PCI frontend [ 0.522639] TCP cubic registered [ 0.522804] NET: Registered protocol family 10 [ 0.522843] pcifront pci-0: Creating PCI Frontend Bus 0000:00 [ 0.527895] pcifront pci-0: claiming resource 0000:00:00.0/0 [ 0.527904] pcifront pci-0: claiming resource 0000:00:00.0/2 [ 0.528255] lo: Disabled Privacy Extensions [ 0.528454] Mobile IPv6 [ 0.528460] NET: Registered protocol family 17 [ 0.528468] Registering the dns_resolver key type [ 0.528502] Using IPI No-Shortcut mode [ 0.528608] registered taskstats version 1 [ 0.628031] XENBUS: Device with no driver: device/vbd/51712 [ 0.628040] XENBUS: Device with no driver: device/console/0 [ 0.628061] /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0) [ 0.628086] Initalizing network drop monitor service [ 0.628172] Freeing unused kernel memory: 412k freed [ 0.628969] Write protecting the kernel text: 2704k [ 0.629233] Write protecting the kernel read-only data: 1016k Loading, please wait... [ 0.660285] udev[46]: starting version 164 [ 0.725951] sky2: driver version 1.28 [ 0.741101] sky2 0000:00:00.0: BAR 0: set to [mem 0xfeb00000-0xfeb03fff 64bit] (PCI address [0xfeb00000-0xfeb03fff]) [ 0.741187] sky2 0000:00:00.0: BAR 2: set to [io 0xe800-0xe8ff] (PCI address [0xe800-0xe8ff]) [ 0.741258] sky2 0000:00:00.0: enabling device (0000 -> 0003) [ 0.741382] sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 [ 0.741554] sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 [ 0.742751] sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 [ 0.751244] blkfront: xvda: barriers enabled [ 0.753096] xvda: xvda1 xvda2 [ 0.871721] ------------[ cut here ]------------ [ 0.871737] WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:337 blk_start_queue+0x1d/0x31() [ 0.871750] Modules linked in: sky2 xen_blkfront [ 0.871763] Pid: 0, comm: swapper Not tainted 2.6.37-trunk-686-bigmem #1 [ 0.871769] Call Trace: [ 0.871778] [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [ 0.871785] [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [ 0.871792] [<c1036023>] ? warn_slowpath_null+0xd/0x10 [ 0.871800] [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [ 0.871808] [<e102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [ 0.871817] [<e102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [ 0.871827] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [ 0.871835] [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [ 0.871843] [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [ 0.871851] [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [ 0.871858] [<c1009117>] ? xen_do_upcall+0x7/0xc [ 0.871865] [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [ 0.871873] [<c1006246>] ? xen_safe_halt+0xf/0x1b [ 0.871879] [<c100e276>] ? default_idle+0x3f/0x5d [ 0.871886] [<c1007dc4>] ? cpu_idle+0x98/0xdc [ 0.871893] [<c14058a0>] ? start_kernel+0x320/0x325 [ 0.871900] [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [ 0.871907] [<c1409046>] ? trap_init+0x1fe/0x367 [ 0.871913] ---[ end trace 4924792c31d7209b ]--- [ 0.871919] ------------[ cut here ]------------ [ 0.871925] WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:245 blk_remove_plug+0x1d/0x78() [ 0.871937] Modules linked in: sky2 xen_blkfront [ 0.871948] Pid: 0, comm: swapper Tainted: G W 2.6.37-trunk-686-bigmem #1 [ 0.871955] Call Trace: [ 0.871960] [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [ 0.871967] [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [ 0.871974] [<c1036023>] ? warn_slowpath_null+0xd/0x10 [ 0.871981] [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [ 0.871990] [<c1139ed1>] ? __blk_run_queue+0x8/0x5b [ 0.871997] [<e102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [ 0.872007] [<e102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [ 0.872015] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [ 0.872022] [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [ 0.872029] [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [ 0.872037] [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [ 0.872044] [<c1009117>] ? xen_do_upcall+0x7/0xc [ 0.872050] [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [ 0.872057] [<c1006246>] ? xen_safe_halt+0xf/0x1b [ 0.872063] [<c100e276>] ? default_idle+0x3f/0x5d [ 0.872071] [<c1007dc4>] ? cpu_idle+0x98/0xdc [ 0.872078] [<c14058a0>] ? start_kernel+0x320/0x325 [ 0.872087] [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [ 0.872098] [<c1409046>] ? trap_init+0x1fe/0x367 [ 0.872104] ---[ end trace 4924792c31d7209c ]--- Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... [ 0.986899] device-mapper: uevent: version 1.0.3 [ 0.987478] device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com done. Begin: Running /scripts/local-premount ... done. [ 1.121342] EXT3-fs: barriers not enabled [ 1.121674] kjournald starting. Commit interval 5 seconds [ 1.121691] EXT3-fs (dm-0): mounted filesystem with ordered data mode Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd[ 1.897070] udev[161]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated...[ 2.183288] input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. [ 5.581677] EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended [ 5.624960] EXT3-fs (dm-0): using internal journal Loading kernel modules...done. Cleaning up ifupdown.... Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. Configuring network interfaces...[ 6.098215] sky2 0000:00:00.0: eth0: enabling interface [ 6.098298] ADDRCONF(NETDEV_UP): eth0: link is not ready done. Cleaning up temporary files.... INIT: Entering runlevel: 2 Using makefile-style concurrent boot in runlevel 2. Starting enhanced syslogd: rsyslogd. Starting periodic command scheduler: cron. Debian GNU/Linux 6.0 xps-106 hvc0 xps-106 login: [ 8.916909] sky2 0000:00:00.0: eth0: Link is up at 1000 Mbps, full duplex, flow control both [ 8.916977] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [-- Attachment #4: 512M_domU_proc_outputs.txt --] [-- Type: text/plain, Size: 1733 bytes --] # cat /proc/interrupts CPU0 55: 12 xen-pirq-pcifront-msi sky2@pci:0000:00:00.0 1271: 972 xen-dyn-event blkif 1272: 139 xen-dyn-event hvc_console 1273: 1 xen-dyn-event pcifront 1274: 260 xen-dyn-event xenbus 1275: 0 xen-percpu-ipi callfuncsingle0 1276: 0 xen-percpu-virq debug0 1277: 0 xen-percpu-ipi callfunc0 1278: 0 xen-percpu-ipi resched0 1279: 1895 xen-percpu-virq timer0 NMI: 0 Non-maskable interrupts LOC: 0 Local timer interrupts SPU: 0 Spurious interrupts PMI: 0 Performance monitoring interrupts IWI: 0 IRQ work interrupts RES: 0 Rescheduling interrupts CAL: 0 Function call interrupts TLB: 0 TLB shootdowns TRM: 0 Thermal event interrupts THR: 0 Threshold APIC interrupts MCE: 0 Machine check exceptions MCP: 0 Machine check polls ERR: 0 MIS: 0 # cat /proc/iomem 00000000-0000ffff : reserved 00010000-0009ffff : System RAM 000a0000-000fffff : reserved 000f0000-000fffff : System ROM 00100000-207fffff : System RAM 01000000-012a3d3f : Kernel code 012a3d40-014046df : Kernel data 01470000-015215ab : Kernel bss feb00000-feb03fff : 0000:00:00.0 feb00000-feb03fff : sky2 # cat /proc/ioports 0000-001f : dma1 0020-0021 : pic1 0040-0043 : timer0 0050-0053 : timer1 0060-0060 : keyboard 0064-0064 : keyboard 0070-0071 : rtc_cmos 0080-008f : dma page reg 00a0-00a1 : pic2 00c0-00df : dma2 00f0-00ff : fpu e800-e8ff : 0000:00:00.0 e800-e8ff : sky2 [-- Attachment #5: 256M_domU_console_logs_2.txt --] [-- Type: text/plain, Size: 15291 bytes --] # xm create /cluster/xen/xps-106.cfg -c Using config file "/cluster/xen/xps-106.cfg". [218927.183880] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. Started domain xps-106 (id=24) (XEN) traps.c:2308:d24 Domain attempted WRMSR 00000000c0010004 from 0000c0b2:cbfa8461 to 00000000:0000abcd. [ 0.000000] Reserving virtual address space above 0xf5800000 [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Linux version 2.6.37-trunk-686-bigmem (Debian 2.6.37-1~experimental.1) (ben@decadent.org.uk) (gcc version 4.4.5 (Debian 4.4.5-10) ) #1 SMP Thu Jan 6 15:21:44 UTC 2011 [ 0.000000] ACPI in unprivileged domain disabled [ 0.000000] released 0 pages of unused memory [ 0.000000] BIOS-provided physical RAM map: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable) [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved) [ 0.000000] Xen: 0000000000100000 - 0000000010800000 (usable) [ 0.000000] NX (Execute Disable) protection: active [ 0.000000] DMI not present or invalid. [ 0.000000] last_pfn = 0x10800 max_arch_pfn = 0x1000000 [ 0.000000] init_memory_mapping: 0000000000000000-0000000010800000 [ 0.000000] RAMDISK: 01793000 - 02ecc000 [ 0.000000] 0MB HIGHMEM available. [ 0.000000] 264MB LOWMEM available. [ 0.000000] mapped low ram: 0 - 10800000 [ 0.000000] low ram: 0 - 10800000 [ 0.000000] Zone PFN ranges: [ 0.000000] DMA 0x00000010 -> 0x00001000 [ 0.000000] Normal 0x00001000 -> 0x00010800 [ 0.000000] HighMem empty [ 0.000000] Movable zone start PFN for each node [ 0.000000] early_node_map[2] active PFN ranges [ 0.000000] 0: 0x00000010 -> 0x000000a0 [ 0.000000] 0: 0x00000100 -> 0x00010800 [ 0.000000] Using APIC driver default [ 0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org [ 0.000000] SMP: Allowing 1 CPUs, 0 hotplug CPUs [ 0.000000] Local APIC disabled by BIOS -- you can enable it with "lapic" [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000 [ 0.000000] Allocating PCI resources starting at 10800000 (gap: 10800000:ef800000) [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.0.1 (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:1 nr_node_ids:1 [ 0.000000] PERCPU: Embedded 13 pages/cpu @cfddf000 s30336 r0 d22912 u53248 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 66944 [ 0.000000] Kernel command line: root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty [ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes) [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes) [ 0.000000] Initializing CPU#0 [ 0.000000] Placing 64MB software IO TLB between cbd7ee40 - cfd7ee40 [ 0.000000] software IO TLB at phys 0xbd7ee40 - 0xfd7ee40 [ 0.000000] Initializing HighMem for node 0 (00000000:00000000) [ 0.000000] Memory: 163268k/270336k available (2703k kernel code, 106620k reserved, 1410k data, 412k init, 0k highmem) [ 0.000000] virtual kernel memory layout: [ 0.000000] fixmap : 0xf5536000 - 0xf57ff000 (2852 kB) [ 0.000000] pkmap : 0xf5200000 - 0xf5400000 (2048 kB) [ 0.000000] vmalloc : 0xd1000000 - 0xf51fe000 ( 577 MB) [ 0.000000] lowmem : 0xc0000000 - 0xd0800000 ( 264 MB) [ 0.000000] .init : 0xc1405000 - 0xc146c000 ( 412 kB) [ 0.000000] .data : 0xc12a3d40 - 0xc14046e0 (1410 kB) [ 0.000000] .text : 0xc1000000 - 0xc12a3d40 (2703 kB) [ 0.000000] SLUB: Genslabs=15, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] Hierarchical RCU implementation. [ 0.000000] RCU dyntick-idle grace-period acceleration is enabled. [ 0.000000] RCU-based detection of stalled CPUs is disabled. [ 0.000000] NR_IRQS:1280 [ 0.000000] Console: colour dummy device 80x25 [ 0.000000] console [tty0] enabled [ 0.000000] console [hvc0] enabled [ 0.000000] installing Xen timer for CPU 0 [ 0.000000] Detected 2506.222 MHz processor. [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 5012.44 BogoMIPS (lpj=10024888) [ 0.004000] pid_max: default: 32768 minimum: 301 [ 0.004000] Security Framework initialized [ 0.004000] SELinux: Disabled at boot. [ 0.004000] Mount-cache hash table entries: 512 [ 0.004000] Initializing cgroup subsys ns [ 0.004000] ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup. [ 0.004000] Initializing cgroup subsys cpuacct [ 0.004000] Initializing cgroup subsys devices [ 0.004000] Initializing cgroup subsys freezer [ 0.004000] Initializing cgroup subsys net_cls [ 0.004000] Initializing cgroup subsys blkio [ 0.004000] CPU: Physical Processor ID: 0 [ 0.004000] CPU: Processor Core ID: 0 [ 0.004000] Performance Events: [ 0.004000] no APIC, boot with the "lapic" boot parameter to force-enable it. [ 0.004000] no hardware sampling interrupt available. [ 0.004000] Broken PMU hardware detected, software events only. [ 0.004000] SMP alternatives: switching to UP code [ 0.016000] Freeing SMP alternatives: 16k freed [ 0.016233] NMI watchdog disabled for cpu0: unable to create perf event: -2 [ 0.016345] Brought up 1 CPUs [ 0.016428] devtmpfs: initialized [ 0.016745] Grant table initialized [ 0.016753] regulator: core version 0.5 [ 0.016807] regulator: dummy: [ 0.016854] NET: Registered protocol family 16 [ 0.017058] PCI: setting up Xen PCI frontend stub [ 0.017058] bio: create slab <bio-0> at 0 [ 0.017058] ACPI: Interpreter disabled. [ 0.017058] xen_balloon: Initialising balloon driver. [ 0.017058] last_pfn = 0x10800 max_arch_pfn = 0x1000000 [ 0.017058] vgaarb: loaded [ 0.017058] PCI: System does not support PCI [ 0.017058] PCI: System does not support PCI [ 0.017058] Switching to clocksource xen [ 0.017417] pnp: PnP ACPI: disabled [ 0.017425] PnPBIOS: Disabled [ 0.019715] NET: Registered protocol family 2 [ 0.019763] IP route cache hash table entries: 4096 (order: 2, 16384 bytes) [ 0.019948] TCP established hash table entries: 16384 (order: 5, 131072 bytes) [ 0.020022] TCP bind hash table entries: 16384 (order: 5, 131072 bytes) [ 0.020083] TCP: Hash tables configured (established 16384 bind 16384) [ 0.020091] TCP reno registered [ 0.020097] UDP hash table entries: 256 (order: 1, 8192 bytes) [ 0.020107] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes) [ 0.020173] NET: Registered protocol family 1 [ 0.020230] Unpacking initramfs... [ 0.046821] Freeing initrd memory: 23780k freed [ 0.054855] platform rtc_cmos: registered platform RTC device (no PNP device found) [ 0.055094] audit: initializing netlink socket (disabled) [ 0.055112] type=2000 audit(1296645206.868:1): initialized [ 0.063463] HugeTLB registered 2 MB page size, pre-allocated 0 pages [ 0.065902] VFS: Disk quotas dquot_6.5.2 [ 0.065967] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) [ 0.066064] msgmni has been set to 365 [ 0.066254] alg: No test for stdrng (krng) [ 0.066324] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253) [ 0.066333] io scheduler noop registered [ 0.066338] io scheduler deadline registered [ 0.066354] io scheduler cfq registered (default) [ 0.066589] isapnp: Scanning for PnP cards... [ 0.515649] isapnp: No Plug & Play device found [ 0.521110] Linux agpgart interface v0.103 [ 0.521204] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled [ 0.521479] PNP: No PS/2 controller found. Probing ports directly. [ 0.522297] i8042.c: No controller found. [ 0.522371] mice: PS/2 mouse device common for all mice [ 0.522460] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0 [ 0.522509] rtc_cmos: probe of rtc_cmos failed with error -38 [ 0.522537] cpuidle: using governor ladder [ 0.522543] cpuidle: using governor menu [ 0.522819] pcifront pci-0: Installing PCI frontend [ 0.522830] TCP cubic registered [ 0.522998] NET: Registered protocol family 10 [ 0.523038] pcifront pci-0: Creating PCI Frontend Bus 0000:00 [ 0.539171] lo: Disabled Privacy Extensions [ 0.539379] Mobile IPv6 [ 0.539385] NET: Registered protocol family 17 [ 0.539394] Registering the dns_resolver key type [ 0.539411] Using IPI No-Shortcut mode [ 0.539534] registered taskstats version 1 [ 0.541437] pcifront pci-0: claiming resource 0000:00:00.0/0 [ 0.541445] pcifront pci-0: claiming resource 0000:00:00.0/2 [ 0.636034] XENBUS: Device with no driver: device/vbd/51712 [ 0.636043] XENBUS: Device with no driver: device/console/0 [ 0.636064] /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0) [ 0.636089] Initalizing network drop monitor service [ 0.636174] Freeing unused kernel memory: 412k freed [ 0.636935] Write protecting the kernel text: 2704k [ 0.637179] Write protecting the kernel read-only data: 1016k Loading, please wait... [ 0.668289] udev[46]: starting version 164 [ 0.719079] sky2: driver version 1.28 [ 0.719174] sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 [ 0.719346] sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 [ 0.733989] sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 [ 0.743630] blkfront: xvda: barriers enabled [ 0.745505] xvda: xvda1 xvda2 [ 0.865767] ------------[ cut here ]------------ [ 0.865782] WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:337 blk_start_queue+0x1d/0x31() [ 0.865796] Modules linked in: sky2 xen_blkfront [ 0.865808] Pid: 0, comm: swapper Not tainted 2.6.37-trunk-686-bigmem #1 [ 0.865815] Call Trace: [ 0.865824] [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [ 0.865832] [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [ 0.865839] [<c1036023>] ? warn_slowpath_null+0xd/0x10 [ 0.865847] [<c1139fd9>] ? blk_start_queue+0x1d/0x31 [ 0.865855] [<d102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [ 0.865865] [<d102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [ 0.865875] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [ 0.865883] [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [ 0.865892] [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [ 0.865900] [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [ 0.865910] [<c1009117>] ? xen_do_upcall+0x7/0xc [ 0.865918] [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [ 0.865926] [<c1006246>] ? xen_safe_halt+0xf/0x1b [ 0.865933] [<c100e276>] ? default_idle+0x3f/0x5d [ 0.865940] [<c1007dc4>] ? cpu_idle+0x98/0xdc [ 0.865948] [<c14058a0>] ? start_kernel+0x320/0x325 [ 0.865955] [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [ 0.865962] [<c1409046>] ? trap_init+0x1fe/0x367 [ 0.865969] ---[ end trace b6935cbfedb011b5 ]--- [ 0.865974] ------------[ cut here ]------------ [ 0.865981] WARNING: at /build/buildd-linux-2.6_2.6.37-1~experimental.1-i386-rePuD0/linux-2.6-2.6.37/debian/build/source_i386_none/block/blk-core.c:245 blk_remove_plug+0x1d/0x78() [ 0.865993] Modules linked in: sky2 xen_blkfront [ 0.866005] Pid: 0, comm: swapper Tainted: G W 2.6.37-trunk-686-bigmem #1 [ 0.866012] Call Trace: [ 0.866018] [<c1036005>] ? warn_slowpath_common+0x6a/0x7b [ 0.866025] [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [ 0.866032] [<c1036023>] ? warn_slowpath_null+0xd/0x10 [ 0.866040] [<c1139e6e>] ? blk_remove_plug+0x1d/0x78 [ 0.866047] [<c1139ed1>] ? __blk_run_queue+0x8/0x5b [ 0.866054] [<d102a723>] ? kick_pending_request_queues+0x19/0x27 [xen_blkfront] [ 0.866064] [<d102a925>] ? blkif_interrupt+0x1c8/0x1e3 [xen_blkfront] [ 0.866076] [<c10753dc>] ? handle_IRQ_event+0x45/0xef [ 0.866083] [<c1076c3d>] ? handle_fasteoi_irq+0x80/0xb1 [ 0.866091] [<c11a85d1>] ? __xen_evtchn_do_upcall+0xd0/0x155 [ 0.866098] [<c11a8cce>] ? xen_evtchn_do_upcall+0x18/0x26 [ 0.866108] [<c1009117>] ? xen_do_upcall+0x7/0xc [ 0.866115] [<c10023a7>] ? hypercall_page+0x3a7/0x1001 [ 0.866122] [<c1006246>] ? xen_safe_halt+0xf/0x1b [ 0.866129] [<c100e276>] ? default_idle+0x3f/0x5d [ 0.866135] [<c1007dc4>] ? cpu_idle+0x98/0xdc [ 0.866142] [<c14058a0>] ? start_kernel+0x320/0x325 [ 0.866150] [<c1407479>] ? xen_start_kernel+0x5a1/0x5a8 [ 0.866157] [<c1409046>] ? trap_init+0x1fe/0x367 [ 0.866162] ---[ end trace b6935cbfedb011b6 ]--- Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... [ 0.977508] device-mapper: uevent: version 1.0.3 [ 0.978078] device-mapper: ioctl: 4.18.0-ioctl (2010-06-29) initialised: dm-devel@redhat.com done. Begin: Running /scripts/local-premount ... done. [ 1.125678] EXT3-fs: barriers not enabled [ 1.131378] kjournald starting. Commit interval 5 seconds [ 1.131395] EXT3-fs (dm-0): mounted filesystem with ordered data mode Begin: Running /scripts/local-bottom ... done. done. Begin: Running /scripts/init-bottom ... done. INIT: version 2.88 booting Using makefile-style concurrent boot in runlevel S. Starting the hotplug events dispatcher: udevd[ 1.957027] udev[160]: starting version 164 . Synthesizing the initial hotplug events...done. Waiting for /dev to be fully populated...[ 2.239895] input: PC Speaker as /devices/platform/pcspkr/input/input0 done. Activating swap...done. [ 5.367412] EXT3-fs (dm-0): warning: maximal mount count reached, running e2fsck is recommended [ 5.408349] EXT3-fs (dm-0): using internal journal Cleaning up ifupdown.... Loading kernel modules...done. Setting up networking.... Activating lvm and md swap...done. Checking file systems...fsck from util-linux-ng 2.17.2 done. Mounting local filesystems...done. Activating swapfile swap...done. Cleaning up temporary files.... Setting kernel variables ...done. Configuring network interfaces...[ 5.895422] sky2 0000:00:00.0: eth0: enabling interface [ 5.895488] ADDRCONF(NETDEV_UP): eth0: link is not ready done. Cleaning up temporary files.... INIT: Entering runlevel: 2 Using makefile-style concurrent boot in runlevel 2. Starting enhanced syslogd: rsyslogd. Starting periodic command scheduler: cron. Debian GNU/Linux 6.0 xps-106 hvc0 xps-106 login: [ 8.332107] sky2 0000:00:00.0: eth0: Link is up at 1000 Mbps, full duplex, flow control both [ 8.332161] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [-- Attachment #6: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 10:24 ` Jean Baptiste Favre 2011-02-02 10:59 ` Ian Campbell @ 2011-02-02 17:42 ` Konrad Rzeszutek Wilk 2011-02-04 8:43 ` Jean Baptiste Favre 1 sibling, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-02-02 17:42 UTC (permalink / raw) To: xen-devel, Jean Baptiste Favre, Ian Campbell On Wed, Feb 02, 2011 at 11:24:51AM +0100, Jean Baptiste Favre wrote: > Hello Ian, > > My domU config file: > > # cat /cluster/xen/xps-106.cfg > kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' > ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' > builder = 'linux' > memory = '398' > vcpus = '1' > cpus = '2' > localtime = 0 > serial = 'pty' > boot = 'cdn' > disk = [ 'drbd:xps-106,xvda,w' ] > on_poweroff = 'destroy' > on_reboot = 'restart' > on_crash = 'restart' > extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force > console=hvc0 xencons=tty" > pci = [ '04:00.0' ] > name = 'xps-106' > hostname = 'xps-106.clichy.jbfavre.org' > > > Le 02/02/2011 10:27, Ian Campbell a écrit : > > On Tue, 2011-02-01 at 22:04 +0000, Jean Baptiste Favre wrote: > > > Le 01/02/2011 17:23, Ian Campbell a écrit : > > > > >> I assume you are not seeing "rx mapping error" in your domU dmesg? Did > > >> you post a full guest console log at some point? Comparing the logs for > > >> the 256MB, 398MB and 512MB guest RAM case might be useful. > > > No sure I've ever posted that logs. But I can redo my tests :) > > > > yes, please do that. > Please find attached both console startup logs with 256M and 512M: > 256M_domU_console_logs.txt > 512M_domU_console_logs.txt > > For 512M, I saw some kernel CallTrace I can not explain. There are not > present with 256M. > > For 398M memory, I can't even start domU : > # xm create /cluster/xen/xps-106.cfg -c > Using config file "/cluster/xen/xps-106.cfg". > [215739.007871] pciback 0000:04:00.0: device has been assigned to > another domain! Over-writting the ownership, but beware. > Started domain xps-106 (id=23) > (XEN) mm.c:798:d23 Non-privileged (23) attempt to map I/O space 00000000 > (XEN) mm.c:4644:d23 ptwr_emulate: could not get_page_from_l1e() > > As I told you, I'm still using Debian 2.6.37 kernel because I've some > problem to compile 2.6.32.27 from Jeremy's git repository. > I hope I can get it compiled today so I'll be able to test with that > kernel as well. So I've tried this on my Abit IP-35 box which has a 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 13) Subsystem: ABIT Computer Corp. Device 1085 Flags: bus master, fast devsel, latency 0, IRQ 29 Memory at fdefc000 (64-bit, non-prefetchable) [size=16K] I/O ports at be00 [size=256] Expansion ROM at <ignored> [disabled] Capabilities: [48] Power Management version 3 Capabilities: [50] Vital Product Data Capabilities: [5c] MSI: Enable+ Count=1/1 Maskable- 64bit+ Capabilities: [e0] Express Legacy Endpoint, MSI 00 Capabilities: [100] Advanced Error Reporting Kernel driver in use: sky2 Kernel modules: sky2 And when I launch this guest with a 32-bit DomU: kernel="/mnt/lab/latest/vmlinuz" ramdisk="/mnt/lab/latest/initramfs.cpio.gz" extra="console=hvc0 debug iommu=soft" memory=320 vcpus=1 cpu='2' on_crash="preserve" #vif = [ 'bridge=switch' ] pci = ["04:00.0"] vfb = [ 'vnc=1, vnclisten=0.0.0.0,vncunused=1'] And played around with the 'extra' to add 'swiotlb=force'. The moment I had 'swiotlb=force' I could not get any DHCP address from the NIC. If I did not have 'swiotlb=force' it would work just fine (can ping any size, etc, this is with 320MB) For fun, I upped the memory (320->720) and kept 'swiotlb=force' in effect. Same effect: can't do DHCP. I look to have a different issue than you, which is that whenever I use swiotlb=force, things go haywire. Fyi, this is what DomU tells me: 12:22:41 # 9 :~/ > dmesg |grep Memor [ 0.000000] Memory: 145640k/335872k available (3731k kernel code, 189784k reserved, 1565k data, 436k init, 0k highmem) 12:22:56 # 10 :~/ > uname -a Linux (none) 2.6.38-rc2-00028-gf2a2d8b #2 SMP Wed Feb 2 12:10:25 EST 2011 i686 i686 i386 GNU/Linux rnet driver. [ 0.921406] udevd (1126): /proc/1126/oom_adj is deprecated, please use /proc/1126/oom_score_adj instead. [ 0.984886] sky2: driver version 1.28 [ 0.995595] sky2 0000:04:00.0: BAR 0: set to [mem 0xfdefc000-0xfdefffff 64bit] (PCI address [0xfdefc000-0xfdefffff]) [ 0.995662] sky2 0000:04:00.0: BAR 2: set to [io 0xbe00-0xbeff] (PCI address [0xbe00-0xbeff]) [ 0.995697] sky2 0000:04:00.0: enabling device (0000 -> 0003) [ 0.996440] sky2 0000:04:00.0: Xen PCI mapped GSI18 to IRQ27 (This is the #master branch from git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git) ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-02 17:42 ` Konrad Rzeszutek Wilk @ 2011-02-04 8:43 ` Jean Baptiste Favre 2011-02-04 8:53 ` Ian Campbell 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-04 8:43 UTC (permalink / raw) To: xen-devel Hello, Sorry for not answering earlier. I made a test with OpenWRT, applying patches bellow. DomU boots, can connect, but nothing changed. I still have incoming packets dropped without any console log. Looking deeper, I can see ARP request coming in on my GW, replies coming out. On the domU, arp command output is: # arp IP address HW type Flags HW address Mask Device 10.0.0.1 0x1 0x0 00:00:00:00:00:00 * eth0 So, now I've an Layer 2 issue as well :( I'm still trying to make my Debian kernel working so that I can test on Debian too. Regards, JB >From Konrad: diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..37c0631 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -2331,7 +2331,7 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, if (likely(skb)) { pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, length, PCI_DMA_FROMDEVICE); - skb_copy_from_linear_data(re->skb, skb->data, length); + skb_copy_from_linear_data(skb, re->skb->data, length); skb->ip_summed = re->skb->ip_summed; skb->csum = re->skb->csum; pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, And from Ian: diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..645d9e9 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -2339,6 +2339,7 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, re->skb->ip_summed = CHECKSUM_NONE; skb_put(skb, length); } + WARN(skb == NULL, "sky2: receive_copy failed netdev_alloc_skb_ip_align length %u\n", length); return skb; } @@ -2386,10 +2387,15 @@ static struct sk_buff *receive_new(struct sky2_port *sky2, nre.skb = sky2_rx_alloc(sky2); if (unlikely(!nre.skb)) + { + WARN(1, "sky2: receive_new failed sky2_rx_alloc\n"); goto nobuf; - + } if (sky2_rx_map_skb(sky2->hw->pdev, &nre, hdr_space)) + { + WARN(1, "sky2: receive_new failed sky2_rx_map_skb\n"); goto nomap; + } skb = re->skb; sky2_rx_unmap_skb(sky2->hw->pdev, re); diff --git a/net/core/dev.c b/net/core/dev.c index 54277df..9c99612 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1516,6 +1516,7 @@ int dev_forward_skb(struct net_device *dev, struct sk_buff *skb) if (unlikely(!(dev->flags & IFF_UP) || (skb->len > (dev->mtu + dev->hard_header_len + VLAN_HLEN)))) { + printk(KERN_CRIT "dev_forward_skb dropping skb\n"); atomic_long_inc(&dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -2700,6 +2701,7 @@ enqueue: local_irq_restore(flags); + printk(KERN_CRIT "enqueue_to_backlog dropping skb\n"); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -3125,6 +3127,7 @@ ncls: if (pt_prev) { ret = pt_prev->func(skb, skb->dev, pt_prev, orig_dev); } else { + printk(KERN_CRIT "__netif_receive_skb dropping skb proto %#x\n", type); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); /* Jamal, now you will not able to escape explaining Le 02/02/2011 18:42, Konrad Rzeszutek Wilk a écrit : > On Wed, Feb 02, 2011 at 11:24:51AM +0100, Jean Baptiste Favre wrote: >> Hello Ian, >> >> My domU config file: >> >> # cat /cluster/xen/xps-106.cfg >> kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' >> ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' >> builder = 'linux' >> memory = '398' >> vcpus = '1' >> cpus = '2' >> localtime = 0 >> serial = 'pty' >> boot = 'cdn' >> disk = [ 'drbd:xps-106,xvda,w' ] >> on_poweroff = 'destroy' >> on_reboot = 'restart' >> on_crash = 'restart' >> extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force >> console=hvc0 xencons=tty" >> pci = [ '04:00.0' ] >> name = 'xps-106' >> hostname = 'xps-106.clichy.jbfavre.org' >> >> >> Le 02/02/2011 10:27, Ian Campbell a écrit : >>> On Tue, 2011-02-01 at 22:04 +0000, Jean Baptiste Favre wrote: >>>> Le 01/02/2011 17:23, Ian Campbell a écrit : >>> >>>>> I assume you are not seeing "rx mapping error" in your domU dmesg? Did >>>>> you post a full guest console log at some point? Comparing the logs for >>>>> the 256MB, 398MB and 512MB guest RAM case might be useful. >>>> No sure I've ever posted that logs. But I can redo my tests :) >>> >>> yes, please do that. >> Please find attached both console startup logs with 256M and 512M: >> 256M_domU_console_logs.txt >> 512M_domU_console_logs.txt >> >> For 512M, I saw some kernel CallTrace I can not explain. There are not >> present with 256M. >> >> For 398M memory, I can't even start domU : >> # xm create /cluster/xen/xps-106.cfg -c >> Using config file "/cluster/xen/xps-106.cfg". >> [215739.007871] pciback 0000:04:00.0: device has been assigned to >> another domain! Over-writting the ownership, but beware. >> Started domain xps-106 (id=23) >> (XEN) mm.c:798:d23 Non-privileged (23) attempt to map I/O space 00000000 >> (XEN) mm.c:4644:d23 ptwr_emulate: could not get_page_from_l1e() >> >> As I told you, I'm still using Debian 2.6.37 kernel because I've some >> problem to compile 2.6.32.27 from Jeremy's git repository. >> I hope I can get it compiled today so I'll be able to test with that >> kernel as well. > > So I've tried this on my Abit IP-35 box which has a > > 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 13) > Subsystem: ABIT Computer Corp. Device 1085 > Flags: bus master, fast devsel, latency 0, IRQ 29 > Memory at fdefc000 (64-bit, non-prefetchable) [size=16K] > I/O ports at be00 [size=256] > Expansion ROM at <ignored> [disabled] > Capabilities: [48] Power Management version 3 > Capabilities: [50] Vital Product Data > Capabilities: [5c] MSI: Enable+ Count=1/1 Maskable- 64bit+ > Capabilities: [e0] Express Legacy Endpoint, MSI 00 > Capabilities: [100] Advanced Error Reporting > Kernel driver in use: sky2 > Kernel modules: sky2 > > And when I launch this guest with a 32-bit DomU: > kernel="/mnt/lab/latest/vmlinuz" > ramdisk="/mnt/lab/latest/initramfs.cpio.gz" > extra="console=hvc0 debug iommu=soft" > memory=320 > vcpus=1 > cpu='2' > on_crash="preserve" > #vif = [ 'bridge=switch' ] > pci = ["04:00.0"] > vfb = [ 'vnc=1, vnclisten=0.0.0.0,vncunused=1'] > > And played around with the 'extra' to add 'swiotlb=force'. > > The moment I had 'swiotlb=force' I could not get any DHCP > address from the NIC. If I did not have 'swiotlb=force' it would > work just fine (can ping any size, etc, this is with 320MB) > > > For fun, I upped the memory (320->720) and kept 'swiotlb=force' in effect. > Same effect: can't do DHCP. > > I look to have a different issue than you, which is that whenever I use > swiotlb=force, things go haywire. > > Fyi, this is what DomU tells me: > > 12:22:41 # 9 :~/ >> dmesg |grep Memor > [ 0.000000] Memory: 145640k/335872k available (3731k kernel code, 189784k reserved, 1565k data, 436k init, 0k highmem) > > 12:22:56 # 10 :~/ >> uname -a > Linux (none) 2.6.38-rc2-00028-gf2a2d8b #2 SMP Wed Feb 2 12:10:25 EST 2011 i686 i686 i386 GNU/Linux > rnet driver. > [ 0.921406] udevd (1126): /proc/1126/oom_adj is deprecated, please use /proc/1126/oom_score_adj instead. > [ 0.984886] sky2: driver version 1.28 > [ 0.995595] sky2 0000:04:00.0: BAR 0: set to [mem 0xfdefc000-0xfdefffff 64bit] (PCI address [0xfdefc000-0xfdefffff]) > [ 0.995662] sky2 0000:04:00.0: BAR 2: set to [io 0xbe00-0xbeff] (PCI address [0xbe00-0xbeff]) > [ 0.995697] sky2 0000:04:00.0: enabling device (0000 -> 0003) > [ 0.996440] sky2 0000:04:00.0: Xen PCI mapped GSI18 to IRQ27 > > (This is the #master branch from git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git) > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 8:43 ` Jean Baptiste Favre @ 2011-02-04 8:53 ` Ian Campbell 2011-02-04 8:54 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-04 8:53 UTC (permalink / raw) To: xen-devel On Fri, 2011-02-04 at 08:43 +0000, Jean Baptiste Favre wrote: > >From Konrad: > diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c > index 7d85a38..37c0631 100644 > --- a/drivers/net/sky2.c > +++ b/drivers/net/sky2.c > @@ -2331,7 +2331,7 @@ static struct sk_buff *receive_copy(struct > sky2_port *sky2, > if (likely(skb)) { > pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, > length, PCI_DMA_FROMDEVICE); > - skb_copy_from_linear_data(re->skb, skb->data, length); > + skb_copy_from_linear_data(skb, re->skb->data, length); > skb->ip_summed = re->skb->ip_summed; > skb->csum = re->skb->csum; > pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, Please don't apply this bit, the argument order to skb_copy.... can be a bit surprising if you assume it it is like memcpy. The original code is correct. This is probably where your L2 issue came from. Please can you try again with the just the WARN patch. Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 8:53 ` Ian Campbell @ 2011-02-04 8:54 ` Jean Baptiste Favre 2011-02-04 10:12 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-04 8:54 UTC (permalink / raw) To: xen-devel Hello, Le 04/02/2011 09:53, Ian Campbell a écrit : > On Fri, 2011-02-04 at 08:43 +0000, Jean Baptiste Favre wrote: > >> >From Konrad: >> diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c >> index 7d85a38..37c0631 100644 >> --- a/drivers/net/sky2.c >> +++ b/drivers/net/sky2.c >> @@ -2331,7 +2331,7 @@ static struct sk_buff *receive_copy(struct >> sky2_port *sky2, >> if (likely(skb)) { >> pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, >> length, PCI_DMA_FROMDEVICE); >> - skb_copy_from_linear_data(re->skb, skb->data, length); >> + skb_copy_from_linear_data(skb, re->skb->data, length); >> skb->ip_summed = re->skb->ip_summed; >> skb->csum = re->skb->csum; >> pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, > > Please don't apply this bit, the argument order to skb_copy.... can be a > bit surprising if you assume it it is like memcpy. The original code is > correct. This is probably where your L2 issue came from. > > Please can you try again with the just the WARN patch. Sure I can :) Did not understood Konrad's patch was incorrect, sorry. Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 8:54 ` Jean Baptiste Favre @ 2011-02-04 10:12 ` Jean Baptiste Favre 2011-02-04 11:04 ` Ian Campbell 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-04 10:12 UTC (permalink / raw) To: xen-devel Hello Ian, Applyed your patches. Now, I've: # ping -s86 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 86 data bytes __netif_receive_skb dropping skb proto 0x20 So problem seems to occur in net/core/dev.c file, according to the patch bellow @@ -3125,6 +3127,7 @@ ncls: if (pt_prev) { ret = pt_prev->func(skb, skb->dev, pt_prev, orig_dev); } else { printk(KERN_CRIT "__netif_receive_skb dropping skb proto %#x\n", type); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); /* Jamal, now you will not able to escape explaining Regards, JB Le 04/02/2011 09:54, Jean Baptiste Favre a écrit : > Hello, > > Le 04/02/2011 09:53, Ian Campbell a écrit : >> On Fri, 2011-02-04 at 08:43 +0000, Jean Baptiste Favre wrote: >> >>> >From Konrad: >>> diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c >>> index 7d85a38..37c0631 100644 >>> --- a/drivers/net/sky2.c >>> +++ b/drivers/net/sky2.c >>> @@ -2331,7 +2331,7 @@ static struct sk_buff *receive_copy(struct >>> sky2_port *sky2, >>> if (likely(skb)) { >>> pci_dma_sync_single_for_cpu(sky2->hw->pdev, re->data_addr, >>> length, PCI_DMA_FROMDEVICE); >>> - skb_copy_from_linear_data(re->skb, skb->data, length); >>> + skb_copy_from_linear_data(skb, re->skb->data, length); >>> skb->ip_summed = re->skb->ip_summed; >>> skb->csum = re->skb->csum; >>> pci_dma_sync_single_for_device(sky2->hw->pdev, re->data_addr, >> >> Please don't apply this bit, the argument order to skb_copy.... can be a >> bit surprising if you assume it it is like memcpy. The original code is >> correct. This is probably where your L2 issue came from. >> >> Please can you try again with the just the WARN patch. > Sure I can :) > Did not understood Konrad's patch was incorrect, sorry. > > Regards, > JB > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 10:12 ` Jean Baptiste Favre @ 2011-02-04 11:04 ` Ian Campbell 2011-02-04 11:25 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-04 11:04 UTC (permalink / raw) To: xen-devel On Fri, 2011-02-04 at 10:12 +0000, Jean Baptiste Favre wrote: > Hello Ian, > Applyed your patches. Thanks. > Now, I've: > # ping -s86 10.0.0.1 > PING 10.0.0.1 (10.0.0.1): 86 data bytes > __netif_receive_skb dropping skb proto 0x20 > > > So problem seems to occur in net/core/dev.c file, according to the patch > bellow Interesting. the number printed in the warning is type == skb->protocol == 0x20 which is not a valid protocol that I can find anywhere (nor is 0x2000 in case I'm mixing my endianesses up). Neither is 0x20 it a valid Ethernet frame length (min 64) so it's not that sort of confusion AFAICT. skb->protocol is initialised in sky2_status_intr with the return value of "eth_type_trans(skb, dev)" which as far as I can tell cannot return 0x20. The domU network configuration is using the sky2 device directly, no bridging, VLAN, tunnels or anything else like that? Please can you post the output of "ethtool -k <sky2-device>". If either tx or rx checksumming are enabled please can you try disabling them ("ethtool -K <sky2-device> [tx|rx] off"). This setting controls the code path in sky2_skb_rx which can go either to netif_receive_skb or napi_gro_receive. The tcpdump captures from both ends would still be interesting to see. The following patch enhances the previous one with a few checks for protocol 0x20 getting set. Ian. diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..727922f 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -2339,6 +2339,7 @@ static struct sk_buff *receive_copy(struct sky2_port *sky2, re->skb->ip_summed = CHECKSUM_NONE; skb_put(skb, length); } + WARN(skb == NULL, "sky2: receive_copy failed netdev_alloc_skb_ip_align length %u\n", length); return skb; } @@ -2386,10 +2387,15 @@ static struct sk_buff *receive_new(struct sky2_port *sky2, nre.skb = sky2_rx_alloc(sky2); if (unlikely(!nre.skb)) + { + WARN(1, "sky2: receive_new failed sky2_rx_alloc\n"); goto nobuf; - + } if (sky2_rx_map_skb(sky2->hw->pdev, &nre, hdr_space)) + { + WARN(1, "sky2: receive_new failed sky2_rx_map_skb\n"); goto nomap; + } skb = re->skb; sky2_rx_unmap_skb(sky2->hw->pdev, re); @@ -2600,6 +2606,7 @@ static int sky2_status_intr(struct sky2_hw *hw, int to_do, u16 idx) } skb->protocol = eth_type_trans(skb, dev); + WARN_ONCE(skb->protocol == 0x20, "sky2_status_intr: skb->protocol is 0x20. ip_summed is %d\n", skb->ip_summed); sky2_skb_rx(sky2, status, skb); diff --git a/net/core/dev.c b/net/core/dev.c index 54277df..5d786b6 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -1516,6 +1516,7 @@ int dev_forward_skb(struct net_device *dev, struct sk_buff *skb) if (unlikely(!(dev->flags & IFF_UP) || (skb->len > (dev->mtu + dev->hard_header_len + VLAN_HLEN)))) { + printk(KERN_CRIT "dev_forward_skb dropping skb\n"); atomic_long_inc(&dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -1524,6 +1525,7 @@ int dev_forward_skb(struct net_device *dev, struct sk_buff *skb) skb->tstamp.tv64 = 0; skb->pkt_type = PACKET_HOST; skb->protocol = eth_type_trans(skb, dev); + WARN_ONCE(skb->protocol == 0x20, "dev_forward_skb protocol 0x20 ip_summed %d\n", skb->ip_summed); return netif_rx(skb); } EXPORT_SYMBOL_GPL(dev_forward_skb); @@ -2700,6 +2702,7 @@ enqueue: local_irq_restore(flags); + printk(KERN_CRIT "enqueue_to_backlog dropping skb\n"); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); return NET_RX_DROP; @@ -3010,6 +3013,8 @@ static int __netif_receive_skb(struct sk_buff *skb) int ret = NET_RX_DROP; __be16 type; + WARN_ONCE(type == 0x20, "__netif_receive_skb protocol 0x20 ip summed %d at line %d\n", skb->ip_summed, __LINE__); + if (!netdev_tstamp_prequeue) net_timestamp_check(skb); @@ -3111,6 +3116,7 @@ ncls: } type = skb->protocol; + WARN_ONCE(type == 0x20, "__netif_receive_skb protocol 0x20 ip summed %d at line %d\n", skb->ip_summed, __LINE__); list_for_each_entry_rcu(ptype, &ptype_base[ntohs(type) & PTYPE_HASH_MASK], list) { if (ptype->type == type && (ptype->dev == null_or_orig || @@ -3125,6 +3131,7 @@ ncls: if (pt_prev) { ret = pt_prev->func(skb, skb->dev, pt_prev, orig_dev); } else { + printk(KERN_CRIT "__netif_receive_skb dropping skb proto %#x ip summed %d\n", type, skb->ip_summed); atomic_long_inc(&skb->dev->rx_dropped); kfree_skb(skb); /* Jamal, now you will not able to escape explaining @@ -3447,6 +3454,7 @@ gro_result_t napi_frags_finish(struct napi_struct *napi, struct sk_buff *skb, case GRO_NORMAL: case GRO_HELD: skb->protocol = eth_type_trans(skb, skb->dev); + WARN_ONCE(skb->protocol == 0x20, "napi_frags_finish protocol 0x20 ip_summed %d\n", skb->ip_summed); if (ret == GRO_HELD) skb_gro_pull(skb, -ETH_HLEN); @@ -3498,6 +3506,7 @@ struct sk_buff *napi_frags_skb(struct napi_struct *napi) * special handling. We'll fix it up properly at the end. */ skb->protocol = eth->h_proto; + WARN_ONCE(skb->protocol == 0x20, "napi_frags_skb protocol 0x20 ip_summed %d\n", skb->ip_summed); out: return skb; ^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 11:04 ` Ian Campbell @ 2011-02-04 11:25 ` Jean Baptiste Favre 2011-02-04 11:28 ` Ian Campbell 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-04 11:25 UTC (permalink / raw) To: xen-devel Hello, Le 04/02/2011 12:04, Ian Campbell a écrit : > On Fri, 2011-02-04 at 10:12 +0000, Jean Baptiste Favre wrote: >> Hello Ian, >> Applyed your patches. > > Thanks. > >> Now, I've: >> # ping -s86 10.0.0.1 >> PING 10.0.0.1 (10.0.0.1): 86 data bytes >> __netif_receive_skb dropping skb proto 0x20 >> >> >> So problem seems to occur in net/core/dev.c file, according to the patch >> bellow > > Interesting. the number printed in the warning is type == skb->protocol > == 0x20 which is not a valid protocol that I can find anywhere (nor is > 0x2000 in case I'm mixing my endianesses up). Neither is 0x20 it a valid > Ethernet frame length (min 64) so it's not that sort of confusion > AFAICT. > > skb->protocol is initialised in sky2_status_intr with the return value > of "eth_type_trans(skb, dev)" which as far as I can tell cannot return > 0x20. > > The domU network configuration is using the sky2 device directly, no > bridging, VLAN, tunnels or anything else like that? At boot it uses bridge. For the test, I delete bridge and set IP address directly on eth0 root@OpenWrt:/# ifconfig br-lan down br-lan: port 1(eth0) entering forwarding state root@OpenWrt:/# brctl delbr br-lan device eth0 left promiscuous mode br-lan: port 1(eth0) entering disabled state root@OpenWrt:/# ifconfig eth0 10.0.0.8 netmask 255.255.255.0 root@OpenWrt:/# ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: seq=0 ttl=64 time=10.455 ms 64 bytes from 10.0.0.1: seq=1 ttl=64 time=0.870 ms ^C --- 10.0.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.870/5.662/10.455 ms root@OpenWrt:/# ping -s86 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 86 data bytes ^C --- 10.0.0.1 ping statistics --- 12 packets transmitted, 0 packets received, 100% packet loss > > Please can you post the output of "ethtool -k <sky2-device>". root@OpenWrt:/# ethtool -k eth0 Offload parameters for eth0: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp-segmentation-offload: on udp-fragmentation-offload: off generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off ntuple-filters: off receive-hashing: on > If either tx or rx checksumming are enabled please can you try disabling > them ("ethtool -K <sky2-device> [tx|rx] off"). This setting controls the > code path in sky2_skb_rx which can go either to netif_receive_skb or > napi_gro_receive. root@OpenWrt:/# ethtool -K eth0 tx off root@OpenWrt:/# ethtool -K eth0 rx off root@OpenWrt:/# ping -s86 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 86 data bytes ^C --- 10.0.0.1 ping statistics --- 3 packets transmitted, 0 packets received, 100% packet loss root@OpenWrt:/# ethtool -k eth0 Offload parameters for eth0: rx-checksumming: off tx-checksumming: off scatter-gather: off tcp-segmentation-offload: off udp-fragmentation-offload: off generic-segmentation-offload: on generic-receive-offload: off large-receive-offload: off ntuple-filters: off receive-hashing: on > The tcpdump captures from both ends would still be interesting to see. > > The following patch enhances the previous one with a few checks for > protocol 0x20 getting set. > > Ian. What is a bit strange here is that I don't any more the KERN_CRIT printk message. Could be a false positive ? I'm currently compiling new kernel with your last patch. will keep you updated Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 11:25 ` Jean Baptiste Favre @ 2011-02-04 11:28 ` Ian Campbell 2011-02-04 13:15 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-04 11:28 UTC (permalink / raw) To: xen-devel On Fri, 2011-02-04 at 11:25 +0000, Jean Baptiste Favre wrote: > Hello, > > Le 04/02/2011 12:04, Ian Campbell a écrit : > > On Fri, 2011-02-04 at 10:12 +0000, Jean Baptiste Favre wrote: > >> Hello Ian, > >> Applyed your patches. > > > > Thanks. > > > >> Now, I've: > >> # ping -s86 10.0.0.1 > >> PING 10.0.0.1 (10.0.0.1): 86 data bytes > >> __netif_receive_skb dropping skb proto 0x20 > >> > >> > >> So problem seems to occur in net/core/dev.c file, according to the patch > >> bellow > > > > Interesting. the number printed in the warning is type == skb->protocol > > == 0x20 which is not a valid protocol that I can find anywhere (nor is > > 0x2000 in case I'm mixing my endianesses up). Neither is 0x20 it a valid > > Ethernet frame length (min 64) so it's not that sort of confusion > > AFAICT. > > > > skb->protocol is initialised in sky2_status_intr with the return value > > of "eth_type_trans(skb, dev)" which as far as I can tell cannot return > > 0x20. > > > > The domU network configuration is using the sky2 device directly, no > > bridging, VLAN, tunnels or anything else like that? > At boot it uses bridge. > For the test, I delete bridge and set IP address directly on eth0 OK, good. [...] > generic-segmentation-offload: on [...] > receive-hashing: on Can you also try turning these two off (independently and together). > What is a bit strange here is that I don't any more the KERN_CRIT printk > message. > Could be a false positive ? Worth bearing in mind, lets see what the next test run produces. > I'm currently compiling new kernel with your last patch. will keep you > updated Thanks. Please gather the tcpdump's too. Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 11:28 ` Ian Campbell @ 2011-02-04 13:15 ` Jean Baptiste Favre 2011-02-04 13:50 ` Ian Campbell 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-04 13:15 UTC (permalink / raw) To: xen-devel [-- Attachment #1: Type: text/plain, Size: 2447 bytes --] Le 04/02/2011 12:28, Ian Campbell a écrit : > On Fri, 2011-02-04 at 11:25 +0000, Jean Baptiste Favre wrote: >> Hello, >> >> Le 04/02/2011 12:04, Ian Campbell a écrit : >>> On Fri, 2011-02-04 at 10:12 +0000, Jean Baptiste Favre wrote: >>>> Hello Ian, >>>> Applyed your patches. >>> >>> Thanks. >>> >>>> Now, I've: >>>> # ping -s86 10.0.0.1 >>>> PING 10.0.0.1 (10.0.0.1): 86 data bytes >>>> __netif_receive_skb dropping skb proto 0x20 >>>> >>>> >>>> So problem seems to occur in net/core/dev.c file, according to the patch >>>> bellow >>> >>> Interesting. the number printed in the warning is type == skb->protocol >>> == 0x20 which is not a valid protocol that I can find anywhere (nor is >>> 0x2000 in case I'm mixing my endianesses up). Neither is 0x20 it a valid >>> Ethernet frame length (min 64) so it's not that sort of confusion >>> AFAICT. >>> >>> skb->protocol is initialised in sky2_status_intr with the return value >>> of "eth_type_trans(skb, dev)" which as far as I can tell cannot return >>> 0x20. >>> >>> The domU network configuration is using the sky2 device directly, no >>> bridging, VLAN, tunnels or anything else like that? >> At boot it uses bridge. >> For the test, I delete bridge and set IP address directly on eth0 > > OK, good. > > [...] >> generic-segmentation-offload: on > [...] >> receive-hashing: on > > Can you also try turning these two off (independently and together). No change with ethtool -K gso off Can not change receive-hashing: # ethtool -K eth1 rxhash off Cannot set device flag settings: Invalid argument >> What is a bit strange here is that I don't any more the KERN_CRIT printk >> message. >> Could be a false positive ? > > Worth bearing in mind, lets see what the next test run produces. Seems that I got this messge only with copybreak=0. With default value (128), no such message More, with copybreak=0, all packets are dropped (even a ping with default packet size is dropped. Same with ping -s1) >> I'm currently compiling new kernel with your last patch. will keep you >> updated No new messages, despite your patch :( > Thanks. > > Please gather the tcpdump's too. Both tcpdump from GW and domU are Attached. Commands were: domU# tcpdump -n -w domU.cap -s0 -i eth0 ether host 00:1f:c6:eb:71:43 or ether broadcast gw# tcpdump -n -w gw.cap -s0 -i br0 ether host 00:1f:c6:eb:71:43 or ether broadcast [-- Attachment #2: gw.cap --] [-- Type: application/octet-stream, Size: 8906 bytes --] [-- Attachment #3: domU.cap --] [-- Type: application/octet-stream, Size: 7484 bytes --] [-- Attachment #4: Type: text/plain, Size: 138 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 13:15 ` Jean Baptiste Favre @ 2011-02-04 13:50 ` Ian Campbell 2011-02-04 14:01 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-04 13:50 UTC (permalink / raw) To: xen-devel On Fri, 2011-02-04 at 13:15 +0000, Jean Baptiste Favre wrote: > > >> What is a bit strange here is that I don't any more the KERN_CRIT printk > >> message. > >> Could be a false positive ? > > > > Worth bearing in mind, lets see what the next test run produces. > Seems that I got this messge only with copybreak=0. > With default value (128), no such message > > More, with copybreak=0, all packets are dropped (even a ping with > default packet size is dropped. Same with ping -s1) Hang on, I thought you previously said copybreak=0 made everything work ok. If that isn't definitely the case then we may be following a red herring. Are you saying that copybreak=0 + this patch breaks? That would be very surprising since the patch doesn't cause any flow control differences. Perhaps there is some difference between your self-built kernels and the Debian kernels you started with? Perhaps you should try the self built kernel with no patches, just to confirm it behaves the same as the Debian kernels? > > Thanks. > > > > Please gather the tcpdump's too. > Both tcpdump from GW and domU are Attached. Were these collected with or without patches? With or without ethtool -K options? With or without copybreak? Please try and be explicit about everything you post, there are lots of variables in the air. > Commands were: > > domU# tcpdump -n -w domU.cap -s0 -i eth0 ether host 00:1f:c6:eb:71:43 or > ether broadcast > > gw# tcpdump -n -w gw.cap -s0 -i br0 ether host 00:1f:c6:eb:71:43 or > ether broadcast So no 0x20 anywhere in what the gw sent as the reply and still no clue where the value came from... Ian. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 13:50 ` Ian Campbell @ 2011-02-04 14:01 ` Jean Baptiste Favre 2011-02-09 9:59 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-04 14:01 UTC (permalink / raw) To: xen-devel Le 04/02/2011 14:50, Ian Campbell a écrit : > On Fri, 2011-02-04 at 13:15 +0000, Jean Baptiste Favre wrote: > >> >>>> What is a bit strange here is that I don't any more the KERN_CRIT printk >>>> message. >>>> Could be a false positive ? >>> >>> Worth bearing in mind, lets see what the next test run produces. >> Seems that I got this messge only with copybreak=0. >> With default value (128), no such message >> >> More, with copybreak=0, all packets are dropped (even a ping with >> default packet size is dropped. Same with ping -s1) > > Hang on, I thought you previously said copybreak=0 made everything work > ok. If that isn't definitely the case then we may be following a red > herring. That's something I'm investigating. Under Debian, copybreak=0 solve the problem Under OpenWRT, copybreak=0 + patch breaks. Will try without patch. > Are you saying that copybreak=0 + this patch breaks? That would be very > surprising since the patch doesn't cause any flow control differences. > > Perhaps there is some difference between your self-built kernels and the > Debian kernels you started with? Perhaps you should try the self built > kernel with no patches, just to confirm it behaves the same as the > Debian kernels? Under Debian, I use 2.6.37 from experimental Under OpenWRT, use legacy 2.6.37, build env applies patches for OpenWRT and compile. OpenWRT provides complete build env, as I still have problem compiling Debian 32bits kernel from 64bits env. That's why I switched back to openWRT for testing. >>> Thanks. >>> >>> Please gather the tcpdump's too. >> Both tcpdump from GW and domU are Attached. > > Were these collected with or without patches? With or without ethtool -K > options? With or without copybreak? > > Please try and be explicit about everything you post, there are lots of > variables in the air. OK, sorry. Will redo all tests Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-04 14:01 ` Jean Baptiste Favre @ 2011-02-09 9:59 ` Jean Baptiste Favre 2011-02-18 21:14 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-09 9:59 UTC (permalink / raw) To: xen-devel Hello, Sorry for long silent period. I'm taking night classes to get an engineering degree and am in the midst of exams this whole week. I'll be back available next week to perform required tests and will provide you an update as soon as possible. Regards, JB Le 04/02/2011 15:01, Jean Baptiste Favre a écrit : > Le 04/02/2011 14:50, Ian Campbell a écrit : >> On Fri, 2011-02-04 at 13:15 +0000, Jean Baptiste Favre wrote: >> >>> >>>>> What is a bit strange here is that I don't any more the KERN_CRIT printk >>>>> message. >>>>> Could be a false positive ? >>>> >>>> Worth bearing in mind, lets see what the next test run produces. >>> Seems that I got this messge only with copybreak=0. >>> With default value (128), no such message >>> >>> More, with copybreak=0, all packets are dropped (even a ping with >>> default packet size is dropped. Same with ping -s1) >> >> Hang on, I thought you previously said copybreak=0 made everything work >> ok. If that isn't definitely the case then we may be following a red >> herring. > That's something I'm investigating. > Under Debian, copybreak=0 solve the problem > Under OpenWRT, copybreak=0 + patch breaks. Will try without patch. > >> Are you saying that copybreak=0 + this patch breaks? That would be very >> surprising since the patch doesn't cause any flow control differences. >> >> Perhaps there is some difference between your self-built kernels and the >> Debian kernels you started with? Perhaps you should try the self built >> kernel with no patches, just to confirm it behaves the same as the >> Debian kernels? > Under Debian, I use 2.6.37 from experimental > Under OpenWRT, use legacy 2.6.37, build env applies patches for OpenWRT > and compile. > > OpenWRT provides complete build env, as I still have problem compiling > Debian 32bits kernel from 64bits env. That's why I switched back to > openWRT for testing. > > >>>> Thanks. >>>> >>>> Please gather the tcpdump's too. >>> Both tcpdump from GW and domU are Attached. >> >> Were these collected with or without patches? With or without ethtool -K >> options? With or without copybreak? >> >> Please try and be explicit about everything you post, there are lots of >> variables in the air. > OK, sorry. Will redo all tests > > Regards, > JB > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-09 9:59 ` Jean Baptiste Favre @ 2011-02-18 21:14 ` Jean Baptiste Favre 2011-02-25 14:40 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-18 21:14 UTC (permalink / raw) To: xen-devel Hello, Back online after my exams :) Had some time to perform tests with my Debian Squeeze 32bits domU and 2.6.37 kernel from experimental. DomU config is: ***************************************************************** kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' #kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem-sky2' #ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem-sky2' builder = 'linux' memory=268 vcpus = '1' cpus = '2' localtime = 0 serial = 'pty' boot = 'cdn' disk = [ 'drbd:xps-106,xvda,w' ] on_poweroff = 'destroy' on_reboot = 'restart' on_crash = 'restart' name = 'xps-106' hostname = 'xps-106.clichy.jbfavre.org' extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force console=hvc0 xencons=tty" pci = [ '04:00.0' ] ***************************************************************** Tests were done first with "legacy" kernel from Debian and with patched kernel (patches from Ian to get verbose debug). Each test consist in executing following command: ping -c5 -s150 10.0.0.1 ping -c5 -s85 10.0.0.1 10.0.0.1 is my gateway As you can see bellow, all verbose debug display "__netif_receive_skb dropping skb" with various "proto" value and always "ip summed 2". I can provide full detailled output if you want. Here are the results: Memory 2.6.37 2.6.37-patched 128 Works Works 130 NoBoot NoBoot 132 Works Works 134 NoBoot NoBoot 136 Works Works 138 NoBoot NoBoot 140 Fails __netif_receive_skb dropping skb 142 NoBoot NoBoot 144 Fails __netif_receive_skb dropping skb 146 NoBoot NoBoot 148 Fails __netif_receive_skb dropping skb 150 NoBoot NoBoot 152 Fails __netif_receive_skb dropping skb 154 NoBoot NoBoot 156 Fails __netif_receive_skb dropping skb 158 NoBoot NoBoot 160 Fails __netif_receive_skb dropping skb 192 Fails __netif_receive_skb dropping skb 224 Fails __netif_receive_skb dropping skb 256 Fails __netif_receive_skb dropping skb 264 Fails __netif_receive_skb dropping skb 266 NoBoot NoBoot 268 Works Works 272 Works Works 288 Works Works 320 Works Works 352 Works Works 384 Works Works 416 Works Works 448 Works Works 480 Works Works 512 Works Works Now, I'll try to do the same tests with OpenWRT. I hope I'll have similar results :) Regards, JB Le 09/02/2011 10:59, Jean Baptiste Favre a écrit : > Hello, > Sorry for long silent period. > > I'm taking night classes to get an engineering degree and am in the > midst of exams this whole week. > > I'll be back available next week to perform required tests and will > provide you an update as soon as possible. > > Regards, > JB > > Le 04/02/2011 15:01, Jean Baptiste Favre a écrit : >> Le 04/02/2011 14:50, Ian Campbell a écrit : >>> On Fri, 2011-02-04 at 13:15 +0000, Jean Baptiste Favre wrote: >>> >>>> >>>>>> What is a bit strange here is that I don't any more the KERN_CRIT printk >>>>>> message. >>>>>> Could be a false positive ? >>>>> >>>>> Worth bearing in mind, lets see what the next test run produces. >>>> Seems that I got this messge only with copybreak=0. >>>> With default value (128), no such message >>>> >>>> More, with copybreak=0, all packets are dropped (even a ping with >>>> default packet size is dropped. Same with ping -s1) >>> >>> Hang on, I thought you previously said copybreak=0 made everything work >>> ok. If that isn't definitely the case then we may be following a red >>> herring. >> That's something I'm investigating. >> Under Debian, copybreak=0 solve the problem >> Under OpenWRT, copybreak=0 + patch breaks. Will try without patch. >> >>> Are you saying that copybreak=0 + this patch breaks? That would be very >>> surprising since the patch doesn't cause any flow control differences. >>> >>> Perhaps there is some difference between your self-built kernels and the >>> Debian kernels you started with? Perhaps you should try the self built >>> kernel with no patches, just to confirm it behaves the same as the >>> Debian kernels? >> Under Debian, I use 2.6.37 from experimental >> Under OpenWRT, use legacy 2.6.37, build env applies patches for OpenWRT >> and compile. >> >> OpenWRT provides complete build env, as I still have problem compiling >> Debian 32bits kernel from 64bits env. That's why I switched back to >> openWRT for testing. >> >> >>>>> Thanks. >>>>> >>>>> Please gather the tcpdump's too. >>>> Both tcpdump from GW and domU are Attached. >>> >>> Were these collected with or without patches? With or without ethtool -K >>> options? With or without copybreak? >>> >>> Please try and be explicit about everything you post, there are lots of >>> variables in the air. >> OK, sorry. Will redo all tests >> >> Regards, >> JB >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-18 21:14 ` Jean Baptiste Favre @ 2011-02-25 14:40 ` Konrad Rzeszutek Wilk 2011-02-28 10:00 ` Jean Baptiste Favre 2011-02-28 12:18 ` Jean Baptiste Favre 0 siblings, 2 replies; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-02-25 14:40 UTC (permalink / raw) To: xen-devel, Jean Baptiste Favre On Fri, Feb 18, 2011 at 10:14:13PM +0100, Jean Baptiste Favre wrote: > Hello, > Back online after my exams :) > > Had some time to perform tests with my Debian Squeeze 32bits domU and > 2.6.37 kernel from experimental. I am bit lost now.. Can you refresh my memory whether the 'copy_break' parameter worked or not? Experimental is the stock kernel or was that a new proposed kernel? Where are the sources for the experimental kernel? > DomU config is: > ***************************************************************** > kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' > ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' > #kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem-sky2' > #ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem-sky2' > builder = 'linux' > memory=268 > vcpus = '1' > cpus = '2' > localtime = 0 > serial = 'pty' > boot = 'cdn' > disk = [ 'drbd:xps-106,xvda,w' ] > on_poweroff = 'destroy' > on_reboot = 'restart' > on_crash = 'restart' > name = 'xps-106' > hostname = 'xps-106.clichy.jbfavre.org' I had a box with a sky2 adapter that looked to have a similar issue but found the culprit to be the switch. So at this point I am having no luck reproducing this. Would it be possible for you to stick the kernel + dist image somewhere so I can try it out on my box? > > extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force > console=hvc0 xencons=tty" Try without 'swiotlb=force' on any kernel that is PVOPS. Only the older ones (lenny) required that. And you don't need 'xencons=tty' either with PVOPS kernels. ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-25 14:40 ` Konrad Rzeszutek Wilk @ 2011-02-28 10:00 ` Jean Baptiste Favre 2011-02-28 12:18 ` Jean Baptiste Favre 1 sibling, 0 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-28 10:00 UTC (permalink / raw) To: xen-devel, konrad.wilk Le 25/02/2011 15:40, Konrad Rzeszutek Wilk a écrit : > On Fri, Feb 18, 2011 at 10:14:13PM +0100, Jean Baptiste Favre wrote: >> Hello, >> Back online after my exams :) >> >> Had some time to perform tests with my Debian Squeeze 32bits domU and >> 2.6.37 kernel from experimental. > > I am bit lost now.. Can you refresh my memory whether the 'copy_break' > parameter worked or not? Copybreak param still solve problem: # ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. __netif_receive_skb dropping skb proto 0x6aac ip summed 2 __netif_receive_skb dropping skb proto 0x620c ip summed 2 __netif_receive_skb dropping skb proto 0x6aac ip summed 2 ^C --- 10.0.0.1 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms root@xps-106:~# rmmod sky2 [ 23.137515] sky2 0000:00:00.0: eth0: disabling interface root@xps-106:~# modprobe sky2 copybreak=0 sky2: driver version 1.28 sky2 0000:00:00.0: Xen PCI enabling IRQ: 18 xen_map_pirq_gsi: returning irq 18 for gsi 18 sky2 0000:00:00.0: Yukon-2 EC Ultra chip revision 3 xen_map_pirq_gsi: returning irq 55 for gsi 55 sky2 0000:00:00.0: eth0: addr 00:1f:c6:eb:71:43 sky2 0000:00:00.0: eth0: enabling interface ADDRCONF(NETDEV_UP): eth0: link is not ready sky2 0000:00:00.0: eth0: Link is up at 1000 Mbps, full duplex, flow control both ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready root@xps-106:~# ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data. 64 bytes from 10.0.0.1: icmp_req=1 ttl=64 time=4.09 ms 64 bytes from 10.0.0.1: icmp_req=2 ttl=64 time=0.814 ms ^C --- 10.0.0.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 0.814/2.456/4.098/1.642 ms > > Experimental is the stock kernel or was that a new proposed kernel? > Where are the sources for the experimental kernel? Kernel 2.6.37 has been installed from Debian experimental repository. Did not have time to compile my own from vanilla sources. Maybe will have some time this week, but not sure. >> DomU config is: >> ***************************************************************** >> kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' >> ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' >> #kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem-sky2' >> #ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem-sky2' >> builder = 'linux' >> memory=268 >> vcpus = '1' >> cpus = '2' >> localtime = 0 >> serial = 'pty' >> boot = 'cdn' >> disk = [ 'drbd:xps-106,xvda,w' ] >> on_poweroff = 'destroy' >> on_reboot = 'restart' >> on_crash = 'restart' >> name = 'xps-106' >> hostname = 'xps-106.clichy.jbfavre.org' > > I had a box with a sky2 adapter that looked to have a similar issue but > found the culprit to be the switch. So at this point I am having no luck > reproducing this. Would it be possible for you to stick the kernel + dist > image somewhere so I can try it out on my box? As requested by you (or Ian, I don't remeber exactly), I've tried booting the server with 32bits 2.6.37 kernel and 32bits Debian Squeeze. Even with mem=256 boot param, everything worked fine, so I'm not sure switch can be the culprit. >> extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force >> console=hvc0 xencons=tty" > > Try without 'swiotlb=force' on any kernel that is PVOPS. Only the older > ones (lenny) required that. And you don't need 'xencons=tty' either with > PVOPS kernels. OK, will try that. Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-25 14:40 ` Konrad Rzeszutek Wilk 2011-02-28 10:00 ` Jean Baptiste Favre @ 2011-02-28 12:18 ` Jean Baptiste Favre 2011-02-28 15:01 ` Konrad Rzeszutek Wilk 1 sibling, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-28 12:18 UTC (permalink / raw) To: xen-devel Hello, Le 25/02/2011 15:40, Konrad Rzeszutek Wilk a écrit : > On Fri, Feb 18, 2011 at 10:14:13PM +0100, Jean Baptiste Favre wrote: >> Hello, >> Back online after my exams :) >> >> Had some time to perform tests with my Debian Squeeze 32bits domU and >> 2.6.37 kernel from experimental. > > I am bit lost now.. Can you refresh my memory whether the 'copy_break' > parameter worked or not? > > Experimental is the stock kernel or was that a new proposed kernel? > Where are the sources for the experimental kernel? > >> DomU config is: >> ***************************************************************** >> kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem' >> ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem' >> #kernel = '/cluster/kernels/vmlinuz-2.6.37-trunk-686-bigmem-sky2' >> #ramdisk = '/cluster/kernels/initrd.img-2.6.37-trunk-686-bigmem-sky2' >> builder = 'linux' >> memory=268 >> vcpus = '1' >> cpus = '2' >> localtime = 0 >> serial = 'pty' >> boot = 'cdn' >> disk = [ 'drbd:xps-106,xvda,w' ] >> on_poweroff = 'destroy' >> on_reboot = 'restart' >> on_crash = 'restart' >> name = 'xps-106' >> hostname = 'xps-106.clichy.jbfavre.org' > > I had a box with a sky2 adapter that looked to have a similar issue but > found the culprit to be the switch. So at this point I am having no luck > reproducing this. Would it be possible for you to stick the kernel + dist > image somewhere so I can try it out on my box? > >> >> extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force >> console=hvc0 xencons=tty" > > Try without 'swiotlb=force' on any kernel that is PVOPS. Only the older > ones (lenny) required that. And you don't need 'xencons=tty' either with > PVOPS kernels. Thought I already tried without 'swiotlb=force', but seems not... Just tried it on my Debian domU, and everything works now, whatever copybreak value can be. But trying the same with OpenWRT gives following results: - With 'swiotlb=force': fails for size equal or greater than 128 bytes (or ping -s86) - Without 'swiotlb=force': always fails I can provide OpenWRT disk img and kernel. For Debian domU, I can provide a dd from LVM and kernel. Let me know which ones you want. Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-28 12:18 ` Jean Baptiste Favre @ 2011-02-28 15:01 ` Konrad Rzeszutek Wilk 2011-02-28 15:17 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-02-28 15:01 UTC (permalink / raw) To: xen-devel, Jean Baptiste Favre > >> extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force > >> console=hvc0 xencons=tty" > > > > Try without 'swiotlb=force' on any kernel that is PVOPS. Only the older > > ones (lenny) required that. And you don't need 'xencons=tty' either with > > PVOPS kernels. > Thought I already tried without 'swiotlb=force', but seems not... > > Just tried it on my Debian domU, and everything works now, whatever > copybreak value can be. > > But trying the same with OpenWRT gives following results: > - With 'swiotlb=force': fails for size equal or greater than 128 bytes > (or ping -s86) > - Without 'swiotlb=force': always fails > > I can provide OpenWRT disk img and kernel. For Debian domU, I can > provide a dd from LVM and kernel. Let me know which ones you want. Lets do both. Do you know where the sources for OpenWRT are located? ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-28 15:01 ` Konrad Rzeszutek Wilk @ 2011-02-28 15:17 ` Jean Baptiste Favre [not found] ` <4D6CB851.40103@jbfavre.org> 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-28 15:17 UTC (permalink / raw) To: xen-devel Hello, Le 28/02/2011 16:01, Konrad Rzeszutek Wilk a écrit : >>>> extra = "root=/dev/mapper/xps--106-root ro iommu=soft swiotlb=force >>>> console=hvc0 xencons=tty" >>> >>> Try without 'swiotlb=force' on any kernel that is PVOPS. Only the older >>> ones (lenny) required that. And you don't need 'xencons=tty' either with >>> PVOPS kernels. >> Thought I already tried without 'swiotlb=force', but seems not... >> >> Just tried it on my Debian domU, and everything works now, whatever >> copybreak value can be. >> >> But trying the same with OpenWRT gives following results: >> - With 'swiotlb=force': fails for size equal or greater than 128 bytes >> (or ping -s86) >> - Without 'swiotlb=force': always fails >> >> I can provide OpenWRT disk img and kernel. For Debian domU, I can >> provide a dd from LVM and kernel. Let me know which ones you want. > > Lets do both. Do you know where the sources for OpenWRT are located? Openwrt .img and PV kernel are available at http://downloads.jbfavre.org/openwrt.tar.gz I'm making dd from my debian DomU and will upload it as debian.tar.gz at the same place. Will update you when completed. Openwrt source tree is available here: svn://svn.openwrt.org/openwrt/trunk You have all information to build it here: http://wiki.openwrt.org/doc/howto/build Or you have all steps I followed here: http://publications.jbfavre.org/virtualisation/xen_openwrt_domu_pci_passthrough.en Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
[parent not found: <4D6CB851.40103@jbfavre.org>]
* Re: PCI passthrough issue [not found] ` <4D6CB851.40103@jbfavre.org> @ 2011-03-03 22:12 ` Konrad Rzeszutek Wilk 2011-03-03 22:47 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-03-03 22:12 UTC (permalink / raw) To: xen-devel, xen-devel > >> Lets do both. Do you know where the sources for OpenWRT are located? > > > > Openwrt .img and PV kernel are available at > > http://downloads.jbfavre.org/openwrt.tar.gz Using that, and this xm file kernel="/mnt/tmp/openwrt/openwrt-x86-xen_domu-vmlinuz" root='/dev/xvda2 rw' memory=256 vcpus=1 localtime=0 disk=['phy:/dev/sdc,xvda,w'] extra="console=hvc0 debug loglevel=10 iommu=soft" name="openwrt" on_crash="preserve" vfb = [ 'vnc=1, vnclisten=0.0.0.0,vncunused=1'] pci = ['04:00.0'] where 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 13) I can't get the sky2 adapter to work at all. Just to make sure it wasn't your build ... > > > > I'm making dd from my debian DomU and will upload it as debian.tar.gz at > > the same place. Will update you when completed. > > > > Openwrt source tree is available here: > > svn://svn.openwrt.org/openwrt/trunk > > > > You have all information to build it here: > > http://wiki.openwrt.org/doc/howto/build > > > > Or you have all steps I followed here: > > http://publications.jbfavre.org/virtualisation/xen_openwrt_domu_pci_passthrough.en ... I tried to follow those directions and found that it would not work. I can't get the xen-pcifront.ko file at all on any of the *combined.img images. I made this patch thinking it was due to the name of the module being different: Index: target/linux/x86/Makefile =================================================================== --- target/linux/x86/Makefile (revision 25855) +++ target/linux/x86/Makefile (working copy) @@ -12,7 +12,7 @@ FEATURES:=squashfs jffs2 ext4 vdi vmdk pcmcia targz SUBTARGETS=generic olpc xen_domu ep80579 net5501 kvm_guest geos -LINUX_VERSION:=2.6.32.29 +LINUX_VERSION:=2.6.37 include $(INCLUDE_DIR)/target.mk Index: package/kernel/modules/virtual.mk =================================================================== --- package/kernel/modules/virtual.mk (revision 25855) +++ package/kernel/modules/virtual.mk (working copy) @@ -168,7 +168,7 @@ TITLE:=Xen PCI device frontend DEPENDS:=@TARGET_x86_xen_domu @LINUX_2_6_37||LINUX_2_6_38 KCONFIG:=CONFIG_XEN_PCIDEV_FRONTEND - FILES:=$(LINUX_DIR)/drivers/xen/platform-pci.ko + FILES:=$(LINUX_DIR)/drivers/pci/xen-pcifront.ko AUTOLOAD:=$(call AutoLoad,10,xen-pcifront) endef but it still would not include the xen-pcifront.ko file on the *combined-ext4.img.gz file. Any ideas what I am doing wrong? ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-03-03 22:12 ` Konrad Rzeszutek Wilk @ 2011-03-03 22:47 ` Jean Baptiste Favre 2011-03-03 22:58 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-03-03 22:47 UTC (permalink / raw) To: Konrad Rzeszutek Wilk; +Cc: xen-devel Hello, Le 03/03/2011 23:12, Konrad Rzeszutek Wilk a écrit : >>>> Lets do both. Do you know where the sources for OpenWRT are located? >>> >>> Openwrt .img and PV kernel are available at >>> http://downloads.jbfavre.org/openwrt.tar.gz > > Using that, and this xm file > > kernel="/mnt/tmp/openwrt/openwrt-x86-xen_domu-vmlinuz" > root='/dev/xvda2 rw' > memory=256 > vcpus=1 > localtime=0 > disk=['phy:/dev/sdc,xvda,w'] > extra="console=hvc0 debug loglevel=10 iommu=soft" > name="openwrt" > on_crash="preserve" > vfb = [ 'vnc=1, vnclisten=0.0.0.0,vncunused=1'] > pci = ['04:00.0'] > > where > 04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8056 PCI-E Gigabit Ethernet Controller (rev 13) > > I can't get the sky2 adapter to work at all. > > Just to make sure it wasn't your build ... >>> >>> I'm making dd from my debian DomU and will upload it as debian.tar.gz at >>> the same place. Will update you when completed. >>> >>> Openwrt source tree is available here: >>> svn://svn.openwrt.org/openwrt/trunk >>> >>> You have all information to build it here: >>> http://wiki.openwrt.org/doc/howto/build >>> >>> Or you have all steps I followed here: >>> http://publications.jbfavre.org/virtualisation/xen_openwrt_domu_pci_passthrough.en > > ... I tried to follow those directions and found that it would not work. > I can't get the xen-pcifront.ko file at all on any of the *combined.img images. > > > I made this patch thinking it was due to the name of the module being different: > > Index: target/linux/x86/Makefile > =================================================================== > --- target/linux/x86/Makefile (revision 25855) > +++ target/linux/x86/Makefile (working copy) > @@ -12,7 +12,7 @@ > FEATURES:=squashfs jffs2 ext4 vdi vmdk pcmcia targz > SUBTARGETS=generic olpc xen_domu ep80579 net5501 kvm_guest geos > > -LINUX_VERSION:=2.6.32.29 > +LINUX_VERSION:=2.6.37 > > include $(INCLUDE_DIR)/target.mk > > Index: package/kernel/modules/virtual.mk > =================================================================== > --- package/kernel/modules/virtual.mk (revision 25855) > +++ package/kernel/modules/virtual.mk (working copy) > @@ -168,7 +168,7 @@ > TITLE:=Xen PCI device frontend > DEPENDS:=@TARGET_x86_xen_domu @LINUX_2_6_37||LINUX_2_6_38 > KCONFIG:=CONFIG_XEN_PCIDEV_FRONTEND > - FILES:=$(LINUX_DIR)/drivers/xen/platform-pci.ko > + FILES:=$(LINUX_DIR)/drivers/pci/xen-pcifront.ko > AUTOLOAD:=$(call AutoLoad,10,xen-pcifront) > endef > > but it still would not include the xen-pcifront.ko file on the *combined-ext4.img.gz > file. Any ideas what I am doing wrong? I don't remember such problem. Could this module be BTW, I'm checking that rebuilding OpenWRT. When you execute make menuconfig, choose Target=x86 and SubTarget="Xen paravirt Guest". Then go to Kernel Modules -> Virtualisation Support Xen PCI frontend is disabled by default. Did you activated it ? OpenWRT global config cat be found in .config: # grep xen -i .config CONFIG_TARGET_x86_xen_domu=y CONFIG_TARGET_x86_xen_domu_Default=y CONFIG_DEFAULT_kmod-xen-evtchn=y CONFIG_DEFAULT_kmod-xen-fs=y CONFIG_DEFAULT_kmod-xen-kbddev=y CONFIG_DEFAULT_kmod-xen-netdev=y CONFIG_X86_GRUB_BOOTOPTS="xencons=hvc" CONFIG_PACKAGE_kmod-xen-evtchn=y # CONFIG_PACKAGE_kmod-xen-fbdev is not set CONFIG_PACKAGE_kmod-xen-fs=y CONFIG_PACKAGE_kmod-xen-kbddev=y CONFIG_PACKAGE_kmod-xen-netdev=y CONFIG_PACKAGE_kmod-xen-pcidev=y Other way may be to fill target/linux/x86/xen_domu/config-default with Xen CONFIG_* values. This will be used as kernel config Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-03-03 22:47 ` Jean Baptiste Favre @ 2011-03-03 22:58 ` Konrad Rzeszutek Wilk 2011-03-04 7:25 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-03-03 22:58 UTC (permalink / raw) To: xen-devel > I don't remember such problem. Could this module be > > BTW, I'm checking that rebuilding OpenWRT. OK. To eliminate this being the kernel, I took the trunk/build_dir/toolchain-i386_gcc-linaro_uClibc-0.9.32/linux-2.6.37 (which has a bunch of patches on top of stock 2.6.37) compiled it outside the OpenWRT and tried to passthrough the same adapter. It worked. But let me try to follow your directions below tomorrow and see what progress I can make. > > When you execute make menuconfig, choose Target=x86 and SubTarget="Xen > paravirt Guest". Then go to Kernel Modules -> Virtualisation Support > Xen PCI frontend is disabled by default. Did you activated it ? > > OpenWRT global config cat be found in .config: > # grep xen -i .config > CONFIG_TARGET_x86_xen_domu=y > CONFIG_TARGET_x86_xen_domu_Default=y > CONFIG_DEFAULT_kmod-xen-evtchn=y > CONFIG_DEFAULT_kmod-xen-fs=y > CONFIG_DEFAULT_kmod-xen-kbddev=y > CONFIG_DEFAULT_kmod-xen-netdev=y > CONFIG_X86_GRUB_BOOTOPTS="xencons=hvc" > CONFIG_PACKAGE_kmod-xen-evtchn=y > # CONFIG_PACKAGE_kmod-xen-fbdev is not set > CONFIG_PACKAGE_kmod-xen-fs=y > CONFIG_PACKAGE_kmod-xen-kbddev=y > CONFIG_PACKAGE_kmod-xen-netdev=y > CONFIG_PACKAGE_kmod-xen-pcidev=y > > > Other way may be to fill target/linux/x86/xen_domu/config-default with > Xen CONFIG_* values. This will be used as kernel config > > Regards, > JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-03-03 22:58 ` Konrad Rzeszutek Wilk @ 2011-03-04 7:25 ` Jean Baptiste Favre 2011-03-16 3:14 ` Konrad Rzeszutek Wilk 0 siblings, 1 reply; 73+ messages in thread From: Jean Baptiste Favre @ 2011-03-04 7:25 UTC (permalink / raw) To: xen-devel Hello Konrad, Le 03/03/2011 23:58, Konrad Rzeszutek Wilk a écrit : >> I don't remember such problem. Could this module be >> >> BTW, I'm checking that rebuilding OpenWRT. > > OK. > > To eliminate this being the kernel, I took the > > trunk/build_dir/toolchain-i386_gcc-linaro_uClibc-0.9.32/linux-2.6.37 > (which has a bunch of patches on top of stock 2.6.37) > > compiled it outside the OpenWRT and tried to passthrough the same > adapter. It worked. But let me try to follow your directions > below tomorrow and see what progress I can make. Hello, One tought or two between wake-up and leaving for work :-) trunk/build_dir/toolchain-i386_gcc-linaro_uClibc-0.9.32/linux-2.6.37 is kernel builddir. So, if it works from here, there's no reason it finaly doesn't. One tought: in "make menuconfig", you will setup the Openwrt's package/features list. If you have one package marked "M", that means it'll be build as separate package (*.opkg). If marked with "y", it'll be integrated within FS image. For you kernel module xen-pcifront, you have to make sure that: - it's marked in trunk/.config - it's marked in kernel config file You can check after compliation: ls -l build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/ total 12 drwxr-xr-x 2 jbfavre jbfavre 4096 Mar 4 01:09 CONTROL drwxr-xr-x 3 jbfavre jbfavre 4096 Mar 4 01:09 etc drwxr-xr-x 3 jbfavre jbfavre 4096 Mar 4 01:09 lib jbfavre@xps-103:~/openwrt/trunk$ ls -l build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/lib/modules/2.6.37/platform-pci.ko -rw-r--r-- 1 jbfavre jbfavre 4936 Mar 4 01:09 build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/lib/modules/2.6.37/platform-pci.ko So, not sure you patch is mandatory, is it ? Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-03-04 7:25 ` Jean Baptiste Favre @ 2011-03-16 3:14 ` Konrad Rzeszutek Wilk 2011-03-25 13:06 ` Jean Baptiste Favre 0 siblings, 1 reply; 73+ messages in thread From: Konrad Rzeszutek Wilk @ 2011-03-16 3:14 UTC (permalink / raw) To: xen-devel On Fri, Mar 04, 2011 at 08:25:49AM +0100, Jean Baptiste Favre wrote: > Hello Konrad, > > Le 03/03/2011 23:58, Konrad Rzeszutek Wilk a écrit : > >> I don't remember such problem. Could this module be > >> > >> BTW, I'm checking that rebuilding OpenWRT. > > > > OK. > > > > To eliminate this being the kernel, I took the > > > > trunk/build_dir/toolchain-i386_gcc-linaro_uClibc-0.9.32/linux-2.6.37 > > (which has a bunch of patches on top of stock 2.6.37) > > > > compiled it outside the OpenWRT and tried to passthrough the same > > adapter. It worked. But let me try to follow your directions > > below tomorrow and see what progress I can make. > Hello, > One tought or two between wake-up and leaving for work :-) I haven't gotten back on this - sorry about this. The other thing that might be going wrong is that CONFIG_PCI_QUIRKS is not turned on in the OpenWRT build. Perhaps there are some work-arounds for the driver that aren't enabled with OpenWRT build? You could try to build a debian kernel using the OpenWRT .config and seeing if it works right? > > trunk/build_dir/toolchain-i386_gcc-linaro_uClibc-0.9.32/linux-2.6.37 > is kernel builddir. So, if it works from here, there's no reason it > finaly doesn't. > > One tought: in "make menuconfig", you will setup the Openwrt's > package/features list. If you have one package marked "M", that means > it'll be build as separate package (*.opkg). If marked with "y", it'll > be integrated within FS image. > > For you kernel module xen-pcifront, you have to make sure that: > - it's marked in trunk/.config > - it's marked in kernel config file > > You can check after compliation: > ls -l build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/ > total 12 > drwxr-xr-x 2 jbfavre jbfavre 4096 Mar 4 01:09 CONTROL > drwxr-xr-x 3 jbfavre jbfavre 4096 Mar 4 01:09 etc > drwxr-xr-x 3 jbfavre jbfavre 4096 Mar 4 01:09 lib > jbfavre@xps-103:~/openwrt/trunk$ ls -l > build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/lib/modules/2.6.37/platform-pci.ko > > -rw-r--r-- 1 jbfavre jbfavre 4936 Mar 4 01:09 > build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/lib/modules/2.6.37/platform-pci.ko > > So, not sure you patch is mandatory, is it ? Probably not. Will get back to this after the XenHackothon. > > Regards, > JB > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-03-16 3:14 ` Konrad Rzeszutek Wilk @ 2011-03-25 13:06 ` Jean Baptiste Favre 0 siblings, 0 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-03-25 13:06 UTC (permalink / raw) To: xen-devel Hello, Sorry for long silence: had some hollidays :) On 16/03/2011 04:14, Konrad Rzeszutek Wilk wrote: > On Fri, Mar 04, 2011 at 08:25:49AM +0100, Jean Baptiste Favre wrote: >> Hello Konrad, >> >> Le 03/03/2011 23:58, Konrad Rzeszutek Wilk a écrit : >>>> I don't remember such problem. Could this module be >>>> >>>> BTW, I'm checking that rebuilding OpenWRT. >>> >>> OK. >>> >>> To eliminate this being the kernel, I took the >>> >>> trunk/build_dir/toolchain-i386_gcc-linaro_uClibc-0.9.32/linux-2.6.37 >>> (which has a bunch of patches on top of stock 2.6.37) >>> >>> compiled it outside the OpenWRT and tried to passthrough the same >>> adapter. It worked. But let me try to follow your directions >>> below tomorrow and see what progress I can make. >> Hello, >> One tought or two between wake-up and leaving for work :-) > > I haven't gotten back on this - sorry about this. > > The other thing that might be going wrong is that CONFIG_PCI_QUIRKS is not > turned on in the OpenWRT build. Perhaps there are some work-arounds for the > driver that aren't enabled with OpenWRT build? You could try to > build a debian kernel using the OpenWRT .config and seeing if > it works right? In my setup, CONFIG_PCI_QUIRKS is enabled. As default compilation did not worked, I took all XEN and PCI related config options from Debian kernel and manually added them in target/linux/x86/xen_domu/config-default As a consequence, its activated in final kernel but with no change :( >> trunk/build_dir/toolchain-i386_gcc-linaro_uClibc-0.9.32/linux-2.6.37 >> is kernel builddir. So, if it works from here, there's no reason it >> finaly doesn't. >> >> One tought: in "make menuconfig", you will setup the Openwrt's >> package/features list. If you have one package marked "M", that means >> it'll be build as separate package (*.opkg). If marked with "y", it'll >> be integrated within FS image. >> >> For you kernel module xen-pcifront, you have to make sure that: >> - it's marked in trunk/.config >> - it's marked in kernel config file >> >> You can check after compliation: >> ls -l build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/ >> total 12 >> drwxr-xr-x 2 jbfavre jbfavre 4096 Mar 4 01:09 CONTROL >> drwxr-xr-x 3 jbfavre jbfavre 4096 Mar 4 01:09 etc >> drwxr-xr-x 3 jbfavre jbfavre 4096 Mar 4 01:09 lib >> jbfavre@xps-103:~/openwrt/trunk$ ls -l >> build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/lib/modules/2.6.37/platform-pci.ko >> >> -rw-r--r-- 1 jbfavre jbfavre 4936 Mar 4 01:09 >> build_dir/linux-x86_xen_domu/packages/ipkg-x86/kmod-xen-pcidev/lib/modules/2.6.37/platform-pci.ko >> >> So, not sure you patch is mandatory, is it ? > > Probably not. > > Will get back to this after the XenHackothon. Hope this goes well :) Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 15:14 ` Jean Baptiste Favre 2011-02-01 15:38 ` Jean Baptiste Favre @ 2011-02-01 15:38 ` Ian Campbell 2011-02-01 16:16 ` Jean Baptiste Favre 1 sibling, 1 reply; 73+ messages in thread From: Ian Campbell @ 2011-02-01 15:38 UTC (permalink / raw) To: xen-devel On Tue, 2011-02-01 at 15:14 +0000, Jean Baptiste Favre wrote: > Le 01/02/2011 15:18, Ian Campbell a écrit : > > Assuming the driver is modular: > > "modprobe sky2 copybreak=<N>" > > > > Depending on your distro there will be somewhere in /etc you can add > > this. e.g. on Debian you can create a file in /etc/modprobe.d/ > > containing "option sky copybreak=<N>" other distros > > use /etc/modprobe.conf etc. > OK I see but it doesn't seems to have any effect. > I tried "option sky copybreak=0" to get all packet copied with no change. The driver is called sky2 not sky so this won't have done anything. I typo'd it above, sorry. > But I have to say that I'm a bit confused: as I run a PV domU, kernel > and initrd are provided by dom0. > So basically, I had no module related binaries installed. After > installation, I tried to remove module and reload it with different > configuration without changes. > Is there any way to provide this sort of option in kernel commandline so > that it 'll be taken into account even in initrd ? It depends on your distro and/or initrd tool, I don't know a generic answer. If the driver is statically compiled and not modular at all you can do <module>.<param>=<value> on the kernel command line, e.g. "sky2.copybreak=256". There doesn't appear to be any way to find out what copybreak the driver actually uses, short of hacking something into the driver itself. e.g. diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c index 7d85a38..786b8c6 100644 --- a/drivers/net/sky2.c +++ b/drivers/net/sky2.c @@ -87,7 +87,7 @@ module_param(debug, int, 0); MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)"); static int copybreak __read_mostly = 128; -module_param(copybreak, int, 0); +module_param(copybreak, int, 0444); MODULE_PARM_DESC(copybreak, "Receive copy threshold"); static int disable_msi = 0; Allow you to see the current active value in /sys/module/sky2/parameters/copybreak. If you are going to do that you might as well just change the constant above though ;-) Ian. ^ permalink raw reply related [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-02-01 15:38 ` Ian Campbell @ 2011-02-01 16:16 ` Jean Baptiste Favre 0 siblings, 0 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-02-01 16:16 UTC (permalink / raw) To: xen-devel Le 01/02/2011 16:38, Ian Campbell a écrit : > On Tue, 2011-02-01 at 15:14 +0000, Jean Baptiste Favre wrote: > > Le 01/02/2011 15:18, Ian Campbell a écrit : > > >> Assuming the driver is modular: > >> "modprobe sky2 copybreak=<N>" > >> > >> Depending on your distro there will be somewhere in /etc you can add > >> this. e.g. on Debian you can create a file in /etc/modprobe.d/ > >> containing "option sky copybreak=<N>" other distros > >> use /etc/modprobe.conf etc. > > OK I see but it doesn't seems to have any effect. > > I tried "option sky copybreak=0" to get all packet copied with no > change. > > The driver is called sky2 not sky so this won't have done anything. I > typo'd it above, sorry. My bad, I fixed your typo during tests, but I just copied it while writing the mail. So basically, using: # cat /etc/modprobe.d/sky2.conf option sky2 copybreak=0 does not help. I have to remove module and load it with right option. BTW it's not a real issue for me, I can add it in init script :) > > But I have to say that I'm a bit confused: as I run a PV domU, kernel > > and initrd are provided by dom0. > > So basically, I had no module related binaries installed. After > > installation, I tried to remove module and reload it with different > > configuration without changes. > > Is there any way to provide this sort of option in kernel commandline so > > that it 'll be taken into account even in initrd ? > > It depends on your distro and/or initrd tool, I don't know a generic > answer. Both are Debian ones. I've tried to compile kernel from Jeremy's git repo, but face some problems: it's been a long time since my last kernel compilation and I did not find all configuration options I need :-/ > If the driver is statically compiled and not modular at all you can do > <module>.<param>=<value> on the kernel command line, e.g. > "sky2.copybreak=256". I may use this way because, as a firewall, I prefer using static kernels :) > There doesn't appear to be any way to find out what copybreak the driver > actually uses, short of hacking something into the driver itself. e.g. > > diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c > index 7d85a38..786b8c6 100644 > --- a/drivers/net/sky2.c > +++ b/drivers/net/sky2.c > @@ -87,7 +87,7 @@ module_param(debug, int, 0); > MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)"); > > static int copybreak __read_mostly = 128; > -module_param(copybreak, int, 0); > +module_param(copybreak, int, 0444); > MODULE_PARM_DESC(copybreak, "Receive copy threshold"); > > static int disable_msi = 0; > > Allow you to see the current active value > in /sys/module/sky2/parameters/copybreak. If you are going to do that > you might as well just change the constant above though ;-) Will try this as well Regards, JB ^ permalink raw reply [flat|nested] 73+ messages in thread
* Re: PCI passthrough issue 2011-01-12 20:07 ` Jean Baptiste Favre 2011-01-12 21:40 ` Konrad Rzeszutek Wilk @ 2011-01-12 21:44 ` Jean Baptiste Favre 1 sibling, 0 replies; 73+ messages in thread From: Jean Baptiste Favre @ 2011-01-12 21:44 UTC (permalink / raw) To: xen-devel Strange, Just noticed some message I missed before: When starting domain: [175004.836398] pciback 0000:04:00.0: device has been assigned to another domain! Over-writting the ownership, but beware. and (XEN) traps.c:2302:d44 Domain attempted WRMSR 00000000c0010004 from 0000c827:4e390408 to 00000000:0000abcd. The first one is very strange as I'm absolutely sure device is NOT assigned to any domain (the only domU I have on this dom0 is my OpenWRT). The second one... well, I don't know what it's talking about :-/ Regards, JB Le 12/01/2011 21:07, Jean Baptiste Favre a écrit : > Le 12/01/2011 19:32, Konrad Rzeszutek Wilk a écrit : >>> >>> Finally, I think that my setup shall not be responsible for the fact >>> that ping works, small DNS answers are received but not bigger ones >> >> Aah, somehow I missed that fact. > > Seems yes :) > >>> (limit seems to be around 100 bytes). >> >> What does wireshark or tcpdump tell you when you send the larger packets? > > That's one of my problem: nothing special :( > > I can only run tcpdump on my gateway (no SSH available on my OpenWRT > domU and only xm console access is available). > I see DNS requests comming in, I see DNS answers going out. > ARP requests are fine. > > As an example: > dig free.fr works > dig jbfavre.org works > dig amazon.fr works > dig amazon.com works (quite long, but works) > dig google.fr fails > dig google.com fails > >>From another machine in the same LAN, all requests work fine. > > Thought this was due to some ARP restriction setted up by default in > OpenWRT, but in this case, I can not explain why some DNS lookup do work > and some don't. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > ^ permalink raw reply [flat|nested] 73+ messages in thread
end of thread, other threads:[~2011-03-25 13:06 UTC | newest] Thread overview: 73+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2011-01-12 15:38 PCI passthrough issue Jean Baptiste Favre 2011-01-12 15:43 ` Konrad Rzeszutek Wilk 2011-01-12 15:53 ` Jean Baptiste Favre 2011-01-12 16:09 ` Jean Baptiste Favre 2011-01-12 16:36 ` Konrad Rzeszutek Wilk 2011-01-12 16:56 ` Jean Baptiste Favre 2011-01-12 17:26 ` Konrad Rzeszutek Wilk 2011-01-12 17:44 ` Jean Baptiste Favre 2011-01-12 18:32 ` Konrad Rzeszutek Wilk 2011-01-12 20:07 ` Jean Baptiste Favre 2011-01-12 21:40 ` Konrad Rzeszutek Wilk 2011-01-12 21:46 ` Jean Baptiste Favre 2011-01-12 22:18 ` Jean Baptiste Favre 2011-01-13 11:28 ` Jean Baptiste Favre 2011-01-13 19:18 ` Jean Baptiste Favre 2011-01-13 20:19 ` Konrad Rzeszutek Wilk 2011-01-13 20:44 ` Jean Baptiste Favre 2011-01-14 14:53 ` Konrad Rzeszutek Wilk 2011-01-14 23:29 ` Jean Baptiste Favre 2011-01-17 8:59 ` Jean Baptiste Favre 2011-01-17 13:58 ` Jean Baptiste Favre 2011-01-22 10:22 ` Jean Baptiste Favre 2011-01-27 20:27 ` Konrad Rzeszutek Wilk 2011-01-27 21:47 ` Jean Baptiste Favre 2011-01-28 15:47 ` Jean Baptiste Favre 2011-02-01 11:34 ` Ian Campbell 2011-02-01 12:17 ` Jean Baptiste Favre 2011-02-01 13:20 ` Ian Campbell 2011-02-01 14:12 ` Jean Baptiste Favre 2011-02-01 14:18 ` Ian Campbell 2011-02-01 15:14 ` Jean Baptiste Favre 2011-02-01 15:38 ` Jean Baptiste Favre 2011-02-01 16:23 ` Ian Campbell 2011-02-01 19:37 ` Konrad Rzeszutek Wilk 2011-02-01 22:06 ` Jean Baptiste Favre 2011-02-02 9:12 ` Ian Campbell 2011-02-01 23:01 ` Jean Baptiste Favre 2011-02-02 9:44 ` Ian Campbell 2011-02-02 15:38 ` Konrad Rzeszutek Wilk 2011-02-02 15:55 ` Ian Campbell 2011-02-02 15:56 ` Jean Baptiste Favre 2011-02-01 22:04 ` Jean Baptiste Favre 2011-02-02 9:27 ` Ian Campbell 2011-02-02 10:24 ` Jean Baptiste Favre 2011-02-02 10:59 ` Ian Campbell 2011-02-02 11:33 ` Jean Baptiste Favre 2011-02-02 17:42 ` Konrad Rzeszutek Wilk 2011-02-04 8:43 ` Jean Baptiste Favre 2011-02-04 8:53 ` Ian Campbell 2011-02-04 8:54 ` Jean Baptiste Favre 2011-02-04 10:12 ` Jean Baptiste Favre 2011-02-04 11:04 ` Ian Campbell 2011-02-04 11:25 ` Jean Baptiste Favre 2011-02-04 11:28 ` Ian Campbell 2011-02-04 13:15 ` Jean Baptiste Favre 2011-02-04 13:50 ` Ian Campbell 2011-02-04 14:01 ` Jean Baptiste Favre 2011-02-09 9:59 ` Jean Baptiste Favre 2011-02-18 21:14 ` Jean Baptiste Favre 2011-02-25 14:40 ` Konrad Rzeszutek Wilk 2011-02-28 10:00 ` Jean Baptiste Favre 2011-02-28 12:18 ` Jean Baptiste Favre 2011-02-28 15:01 ` Konrad Rzeszutek Wilk 2011-02-28 15:17 ` Jean Baptiste Favre [not found] ` <4D6CB851.40103@jbfavre.org> 2011-03-03 22:12 ` Konrad Rzeszutek Wilk 2011-03-03 22:47 ` Jean Baptiste Favre 2011-03-03 22:58 ` Konrad Rzeszutek Wilk 2011-03-04 7:25 ` Jean Baptiste Favre 2011-03-16 3:14 ` Konrad Rzeszutek Wilk 2011-03-25 13:06 ` Jean Baptiste Favre 2011-02-01 15:38 ` Ian Campbell 2011-02-01 16:16 ` Jean Baptiste Favre 2011-01-12 21:44 ` Jean Baptiste Favre
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.