From mboxrd@z Thu Jan  1 00:00:00 1970
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: crash in is_xen_swiotlb_buffer
Date: Wed, 2 May 2012 15:50:36 -0400
Message-ID: <20120502195035.GA31755@phenom.dumpdata.com>
References: <20120313173239.GA27920@eire.uk.xensource.com>
	<20120313173504.GB13033@phenom.dumpdata.com>
	<20120313180422.GA28716@eire.uk.xensource.com>
	<20120313190917.GA31144@eire.uk.xensource.com>
	<20120313192541.GA4969@phenom.dumpdata.com>
	<20120313223437.GA5180@eire.uk.xensource.com>
	<20120313234503.GA18924@phenom.dumpdata.com>
	<20120316191127.GA6668@eire.uk.xensource.com>
	<20120316195907.GA6228@phenom.dumpdata.com>
	<20120318155034.GA24315@eire.uk.xensource.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Content-Disposition: inline
In-Reply-To: <20120318155034.GA24315@eire.uk.xensource.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Goncalo Gomes <Goncalo.Gomes@EU.CITRIX.COM>
Cc: Konrad Rzeszutek Wilk <konrad@darnok.org>, "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>, "dave.mccracken@oracle.com" <dave.mccracken@oracle.com>
List-Id: xen-devel@lists.xenproject.org

On Sun, Mar 18, 2012 at 03:50:35PM +0000, Goncalo Gomes wrote:
> On Fri, 16 Mar 2012, Konrad Rzeszutek Wilk wrote:
> 
> > On Fri, Mar 16, 2012 at 07:11:27PM +0000, Goncalo Gomes wrote:
> > > Any luck with this one? :)
> > 
> > Can you try with a 64-bit hypervisor please?
> 
> I tried and I can still trigger the bug.
> 
> I've attached the log.
> 
> Goncalo

.. snip..
> [    0.000000] No NUMA configuration found
> [    0.000000] Faking a node at 0000000000000000-0000000240000000
> (XEN) mm.c:943:d0 Attempt to map superpage without allowsuperpage flag in hypervisor
> [    0.000000] ------------[ cut here ]------------
> [    0.000000] WARNING: at arch/x86/xen/multicalls.c:129 xen_mc_issue+0x34/0x62()
> [    0.000000] Hardware name: PowerEdge R310
> [    0.000000] Modules linked in:
> [    0.000000] Pid: 0, comm: swapper Not tainted 3.2.9 #9
> [    0.000000] Call Trace:
> [    0.000000]  [<c104236b>] ? warn_slowpath_common+0x6a/0x7b
> [    0.000000]  [<c1005442>] ? xen_mc_issue+0x34/0x62
> [    0.000000]  [<c1042389>] ? warn_slowpath_null+0xd/0x10
> [    0.000000]  [<c1005442>] ? xen_mc_issue+0x34/0x62
> [    0.000000]  [<c1005f3b>] ? xen_set_pmd_hyper+0x3c/0x42
> [    0.000000]  [<c102edbe>] ? set_pmd_pfn+0xde/0xf9
> [    0.000000]  [<c168b652>] ? init_alloc_remap+0x1b3/0x216
> [    0.000000]  [<c168aa48>] ? setup_node_data+0x4c/0x22f
> [    0.000000]  [<c168b203>] ? T.744+0x290/0x2c2
> [    0.000000]  [<c168b2ac>] ? T.743+0x77/0x1a1
> [    0.000000]  [<c1025290>] ? default_get_apic_id+0x14/0x33
> [    0.000000]  [<c168b3ed>] ? initmem_init+0x5/0xb7
> [    0.000000]  [<c167cef4>] ? setup_arch+0x5bf/0x694
> [    0.000000]  [<c100b840>] ? __spin_time_accum+0x26/0x36
> [    0.000000]  [<c167852c>] ? start_kernel+0x81/0x34d
> [    0.000000]  [<c167a258>] ? xen_start_kernel+0x554/0x55b
> [    0.000000] ---[ end trace 4eaa2a86a8e2da22 ]---

So I've been able to reproduce this and it is due to CONFIG_NUMA=y
set on 32-bit builds.

>>From a brief look, it looks as this is happening:

        /* perform actual remap */
        for (pfn = 0; pfn < size >> PAGE_SHIFT; pfn += PTRS_PER_PTE)
                set_pmd_pfn((unsigned long)remap_va + (pfn << PAGE_SHIFT),
                            (node_pa >> PAGE_SHIFT) + pfn,
                            PAGE_KERNEL_LARGE);

The PAGE_KERNEL_LARGE means that the PSE bit (so the 2MB) is set
which is a no-no. What is weird is that acpi_numa is disabled when booting
under Xen, but somehow this (which in my case was the 'fake_numa' code)
still gets turned on. Even doing 'numa=off' on the command line causes
this to appear.