From mboxrd@z Thu Jan 1 00:00:00 1970 From: Wei Liu Subject: Re: Xen-unstable Linux 3.14-rc3 and 3.13 Network troubles "bisected" Date: Wed, 12 Mar 2014 15:04:35 +0000 Message-ID: <20140312150435.GO19620@zion.uk.xensource.com> References: <20140311153616.GE19620@zion.uk.xensource.com> <1751914199.20140312024243@eikelenboom.it> <20140312113522.GG19620@zion.uk.xensource.com> <659214230.20140312124516@eikelenboom.it> <20140312120444.GH19620@zion.uk.xensource.com> <751560446.20140312152336@eikelenboom.it> <20140312144826.GK19620@zion.uk.xensource.com> <1241369584.20140312154946@eikelenboom.it> <20140312145915.GM19620@zion.uk.xensource.com> <309265573.20140312160156@eikelenboom.it> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Content-Disposition: inline In-Reply-To: <309265573.20140312160156@eikelenboom.it> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Sander Eikelenboom Cc: annie li , Paul Durrant , Wei Liu , Zoltan Kiss , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On Wed, Mar 12, 2014 at 04:01:56PM +0100, Sander Eikelenboom wrote: > > Wednesday, March 12, 2014, 3:59:15 PM, you wrote: > > > On Wed, Mar 12, 2014 at 03:49:46PM +0100, Sander Eikelenboom wrote: > >> > >> Wednesday, March 12, 2014, 3:48:26 PM, you wrote: > >> > >> > On Wed, Mar 12, 2014 at 03:23:36PM +0100, Sander Eikelenboom wrote: > >> > [...] > >> >> > From the look of the code, 0 looks correct to me. Netback won't complain > >> >> > about "bad gref" because it has no idea what a "good gref" looks like. > >> >> > Only Xen has the knowledge whether a gref is legit. All netback sees is > >> >> > the hypercall fails and it tries to push corresponding response to > >> >> > netfront. But you can probably safely assume a gref larger than several > >> >> > thousands be bad. > >> >> > >> >> OK it's indeed netback setting the status to -1: > >> >> > >> >> [ 976.431585] vif vif-7-0 vif7.0: ?!? xenvif_rx_action status err? status:-1 meta_slots_used:1 flags:3 size:958 > >> >> [ 1030.855057] vif vif-7-0 vif7.0: ?!? xenvif_rx_action status err? status:-1 meta_slots_used:2 flags:7 size:2974 > >> >> [ 1030.861759] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:0 nr_meta_slots:1 flags:0 size:1460 > >> >> [ 1030.868499] vif vif-7-0 vif7.0: ?!? xenvif_rx_action status err? status:-1 meta_slots_used:1 flags:3 size:958 > >> >> [ 1030.875278] vif vif-7-0 vif7.0: ?!? xenvif_rx_action status err? status:-1 meta_slots_used:1 flags:3 size:2974 > >> >> [ 1068.199650] vif vif-7-0 vif7.0: ?!? xenvif_rx_action status err? status:-1 meta_slots_used:8 flags:7 size:1634 > >> >> [ 1068.206479] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:0 nr_meta_slots:7 flags:4 size:2848 > >> >> [ 1068.213158] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:1 nr_meta_slots:7 flags:4 size:4096 > >> >> [ 1068.219701] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:2 nr_meta_slots:7 flags:4 size:4096 > >> >> [ 1068.226194] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:3 nr_meta_slots:7 flags:4 size:4096 > >> >> [ 1068.232728] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:4 nr_meta_slots:7 flags:4 size:4096 > >> >> [ 1068.239163] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:5 nr_meta_slots:7 flags:4 size:4096 > >> >> [ 1068.245397] vif vif-7-0 vif7.0: ?!? xenvif_add_frag_responses status err? status:-1 i:6 nr_meta_slots:7 flags:0 size:1168 > >> >> [ 1068.251457] vif vif-7-0 vif7.0: ?!? xenvif_rx_action status err? status:-1 meta_slots_used:1 flags:3 size:958 > >> >> > >> >> Now to find out why .. > >> >> > >> > >> > -1 is the error code documented in netif.h, XEN_NETIF_RSP_ERROR. > >> > >> grant copy fails ->> netback sets response status to -1. > >> > >> > Assuming the traffic volumn is quite high both way (TX and RX), probably > >> > worth checking if RX ring is overflowed. That is, if consumer pointer(s) > >> > advance before producer pointer(s). > >> > >> In what function ? > >> > > > That should be in RX path. Look for "RING_GET_REQUESTS(&vif->rx". That's > > the macro to get a slot in RX ring; second argument is consumer index. > > You can see if it advances before producer index. > > grep -r -i -n "RING_GET_REQUESTS" drivers/net/xen-netback/* delivers 0.0 results .. > Sorry, remove the trailing "S". Actually you only need to look at netback.c. > > Wei. > > >> > Wei. > >> >