From mboxrd@z Thu Jan  1 00:00:00 1970
From: Wei Liu <wei.liu2@citrix.com>
Subject: Re: Xen-unstable Linux 3.14-rc3 and 3.13 Network
	troubles
Date: Thu, 27 Feb 2014 15:57:26 +0000
Message-ID: <20140227155726.GI16241@zion.uk.xensource.com>
References: <1772884781.20140218222513@eikelenboom.it>
	<5305CFC6.3080502@oracle.com>
	<587238484.20140220121842@eikelenboom.it>
	<5306F2E8.5090509@oracle.com>
	<824074181.20140226101442@eikelenboom.it>
	<59358334.20140226161123@eikelenboom.it>
	<20140227141812.GE16241@zion.uk.xensource.com>
	<529743590.20140227154351@eikelenboom.it>
	<20140227151538.GG16241@zion.uk.xensource.com>
	<1982379440.20140227162655@eikelenboom.it>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Content-Disposition: inline
In-Reply-To: <1982379440.20140227162655@eikelenboom.it>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Sander Eikelenboom <linux@eikelenboom.it>
Cc: annie li <annie.li@oracle.com>, Paul Durrant <Paul.Durrant@citrix.com>, Wei Liu <wei.liu2@citrix.com>, Zoltan Kiss <zoltan.kiss@citrix.com>, xen-devel@lists.xen.org
List-Id: xen-devel@lists.xenproject.org

On Thu, Feb 27, 2014 at 04:26:55PM +0100, Sander Eikelenboom wrote:
[...]
> >> Added some more printk's:
> >> 
> >> @@ -2072,7 +2076,11 @@ __gnttab_copy(
> >>                                        &s_frame, &s_pg,
> >>                                        &source_off, &source_len, 1);
> >>          if ( rc != GNTST_okay )
> >> -            goto error_out;
> >> +            PIN_FAIL(error_out, GNTST_general_error,
> >> +                     "?!?!? src_is_gref: aquire grant for copy failed current_dom_id:%d src_dom_id:%d dest_dom_id:%d\n",
> >> +                     current->domain->domain_id, op->source.domid, op->dest.domid);
> >> +
> >> +
> >>          have_s_grant = 1;
> >>          if ( op->source.offset < source_off ||
> >>               op->len > source_len )
> >> @@ -2096,7 +2104,11 @@ __gnttab_copy(
> >>                                        current->domain->domain_id, 0,
> >>                                        &d_frame, &d_pg, &dest_off, &dest_len, 1);
> >>          if ( rc != GNTST_okay )
> >> -            goto error_out;
> >> +            PIN_FAIL(error_out, GNTST_general_error,
> >> +                     "?!?!? dest_is_gref: aquire grant for copy failed current_dom_id:%d src_dom_id:%d dest_dom_id:%d\n",
> >> +                     current->domain->domain_id, op->source.domid, op->dest.domid);
> >> +
> >> +
> >>          have_d_grant = 1;
> >> 
> >> 
> >> this comes out:
> >> 
> >> (XEN) [2014-02-27 02:34:37] grant_table.c:2109:d0 ?!?!? dest_is_gref: aquire grant for copy failed current_dom_id:0 src_dom_id:32752 dest_dom_id:7
> >> 
> 
> > If it fails in gnttab_copy then I very much suspects this is a network
> > driver problem as persistent grant in blk driver doesn't use grant
> > copy.
> 
> Does the dest_gref or src_is_gref by any chance give some sort of direction ?
> 

Yes, there's indication. For network driver, dest_is_gref means DomU RX
path, src_is_gref means DomU TX path.

In the particular error message you mentioned, it means that this
happens in DomU's RX path, but it would not give us clear idea what had
happened. As the ring is skewed any way it's not surprised to see a
garbage gref in hypervisor. 

Wei.

> >> 
> >> > My suggestion is, if you have a working base line, you can try to setup
> >> > different frontend / backend combination to help narrow down the
> >> > problem.
> >> 
> >> Will see what i can do after the weekend
> >> 
> 
> > Thanks
> 
> >> > Wei.
> >> 
> >> <snip>
> >> 
>