From mboxrd@z Thu Jan  1 00:00:00 1970
From: Tim Deegan <tim@xen.org>
Subject: Re: [PATCH RFC 1/4] x86/mm: Shadow and p2m changes for
 PV mem_access
Date: Thu, 1 May 2014 16:18:35 +0200
Message-ID: <20140501141835.GD77240@deinos.phlegethon.org>
References: <1398746705-6658-1-git-send-email-aravindp@cisco.com>
	<1398746705-6658-2-git-send-email-aravindp@cisco.com>
	<535F83FE020000780000D397@nat28.tlf.novell.com>
	<97A500D504438F4ABC02EBA81613CC63317E7AC4@xmb-aln-x02.cisco.com>
	<5360B51D020000780000D9A2@g0-1-119.ukb-fw-asa.gns.novell.com>
	<97A500D504438F4ABC02EBA81613CC63317E9527@xmb-aln-x02.cisco.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Received: from mail6.bemta5.messagelabs.com ([195.245.231.135])
	by lists.xen.org with esmtp (Exim 4.72) (envelope-from <tim@xen.org>)
	id 1Wfroz-0000CL-GE
	for xen-devel@lists.xenproject.org; Thu, 01 May 2014 14:18:45 +0000
Content-Disposition: inline
In-Reply-To: <97A500D504438F4ABC02EBA81613CC63317E9527@xmb-aln-x02.cisco.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: "Aravindh Puthiyaparambil (aravindp)" <aravindp@cisco.com>
Cc: Keir Fraser <keir@xen.org>, Ian Campbell <ian.campbell@citrix.com>, "Andres Lagar-Cavilla (andreslc@gridcentric.ca)" <andreslc@gridcentric.ca>, Ian Jackson <ian.jackson@eu.citrix.com>, Jan Beulich <JBeulich@suse.com>, "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
List-Id: xen-devel@lists.xenproject.org

At 22:20 +0000 on 30 Apr (1398892838), Aravindh Puthiyaparambil (aravindp) wrot> >>>> @@ -2443,7 +2443,8 @@ int sh_remove_all_mappings(struct vcpu *v,
> >>>mfn_t gmfn)
> >>>>          if ( !(shadow_mode_external(v->domain)
> >>>>                 && (page->count_info & PGC_count_mask) <= 3
> >>>>                 && ((page->u.inuse.type_info & PGT_count_mask)
> >>>> -                   == !!is_xen_heap_page(page))) )
> >>>> +                       == !!is_xen_heap_page(page)))
> >>>> +                    && !(shadow_mode_mem_access(v->domain)) )
> >>>
> >>>You're breaking indentation, there are pointless parentheses again,
> >>>but most importantly - why?
> >>
> >> Sorry, I meant to ask a question about this in my patch message and
> >> mark this as a workaround. I am seeing the message on all
> >> sh_remove_all_mappings() and I was not able to figure out why this was
> >> happening. I just added this as a work around. I was hoping you or Tim
> >would shed more light on this.
> >
> >I'm afraid that without detail on which pages the triggers on, and you at least
> >having spent some time finding out where the stray/extra references may be
> >coming from it's going to be hard to help.
> 
> Here are some of the prints I saw. I typically saw it for every set_entry() call.
> 
> (XEN) sh error: sh_remove_all_mappings(): can't find all mappings of mfn 33960: c=8000000000000002 t=7400000000000000 <most common>
>  (XEN) sh error: sh_remove_all_mappings(): can't find all mappings of mfn 134b8: c=8000000000000038 t=7400000000000001

This is because you are not in shadow_mode_refcounts() (i.e. the
page's refcount and typecount are based on the _guest_ pagetables
rather than the _shadow_ ones).  That means that sh_remove_all_mappings()
can't use the refcounts to figure out when it's removed the last
shadow mapping.

That also means that sh_remove_all_mappings() will have had to search
every sl1e in the system, which is expensive.  If there will be a lot
of these updates, you might consider batching them and using
shadow_blow_tables() after each batch instead.

Tim.