Re: vcpu-pin cause dom0 kernel panic

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Andrew Xu <xu.an@cloudex.cn>
Cc: xen-devel@lists.xensource.com, yunhong.jiang@intel.com
Subject: Re: vcpu-pin cause dom0 kernel panic
Date: Tue, 31 May 2011 10:12:49 -0400	[thread overview]
Message-ID: <20110531141249.GH14641@dumpdata.com> (raw)
In-Reply-To: <20110530091216.E863.3A8D29D5@cloudex.cn>

On Mon, May 30, 2011 at 09:12:16AM +0800, Andrew Xu wrote:
> The panic reason of pin-1-core-dom0 is DMA-remapping error.
> 
> Dom0 (pv) does not require VT-d , although it can use.
> So, adding "iommu=no-pv,passthrough" to xen bootoption 
> can avoid this problem, obviously, bug still exist ...

Or the firmware on the LSI card is doing something icky..

Is the 3:0.0 your LSI/PERC card?

This was not a problem with the 12.10.1.001 firmware right?
> 
> /*
>  * The 'iommu' parameter enables the IOMMU.  Optional comma separated
>  * value may contain:
>  *
>  *   off|no|false|disable       Disable IOMMU (default)
>  *   pv                         Enable IOMMU for PV domains
>  *   no-pv                      Disable IOMMU for PV domains (default)
>  *   force|required             Don't boot unless IOMMU is enabled
>  *   workaround_bios_bug        Workaround some bios issue to still enable
>                                 VT-d, don't guarantee security
>  *   passthrough                Enable VT-d DMA passthrough (no DMA
>  *                              translation for Dom0)
>  *   no-snoop                   Disable VT-d Snoop Control
>  *   no-qinval                  Disable VT-d Queued Invalidation
>  *   no-intremap                Disable VT-d Interrupt Remapping
>  */
> On Fri, 27 May 2011 17:38:23 +0800
> Andrew Xu <xu.an@cloudex.cn> wrote:
> 
> > I update update RAID firmware from 12.10.1.001 to 12.12.0.048.
> > pin-2 dom0 can find disk,
> > but, pin-1 dom0 still panic.
> > ------------------------------------------------
> > (XEN) [VT-D]iommu.c:821: iommu_fault_status: Primary Pending Fault
> > (XEN) [VT-D]iommu.c:796: DMAR:[DMA Read] Request device [03:00.0] fault addr bf173000, iommu reg = ffff82c3fff57000
> > (XEN) DMAR:[fault reason 06h] PTE Read access is not set
> > (XEN) print_vtd_entries: iommu = ffff83103fdf7ea0 bdf = 3:0.0 gmfn = bf173
> > (XEN)     root_entry = ffff83103fd38000
> > (XEN)     root_entry[3] = 1011b5d001
> > (XEN)     context = ffff831011b5d000
> > (XEN)     context[0] = 2_1019da3005
> > (XEN)     l4 = ffff831019da3000
> > (XEN)     l4_index = 0
> > (XEN)     l4[0] = 1019da2003
> > (XEN)     l3 = ffff831019da2000
> > (XEN)     l3_index = 2
> > (XEN)     l3[2] = 101999f003
> > (XEN)     l2 = ffff83101999f000
> > (XEN)     l2_index = 1f8
> > (XEN)     l2[1f8] = 10197a6003
> > (XEN)     l1 = ffff8310197a6000
> > (XEN)     l1_index = 173
> > (XEN)     l1[173] = 0
> > (XEN)     l1[173] not present
> > -------------------------------------------------
> > 
> > When I add "loglvl=none" to xen bootoptions, pin-1 dom0 run OK.
> > It is so !!!strange!!!
> > 
> > 
> > On Tue, 24 May 2011 10:01:50 -0400
> > Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > 
> > > On Thu, May 19, 2011 at 09:08:24PM +0800, Andrew Xu wrote:
> > > > I use xen 4.0(dom0 is suse11.sp1,2.6.32 x86_64) on 
> > > 
> > > Did you open a Bugzilla with SLES?
> > > 
> > > > Dell R710 with PERC H700 RAID adapter.
> > > > --Intel(R) Xeon(R) CPU E5620 @ 2.40GHz
> > > > --8 CPU cores.
> > > > --Memory 64G
> > > > --RAID5 4.5T
> > > > 
> > > > When I dedicated (pin) a CPU core only for dom0 use.
> > > > (I specify "dom0_max_vcpus=1 dom0_vcpus_pin" options for Xen)
> > > > I got dom0 kernel panic error(pin-1-5.30.bmp)
> > > > 
> > > > When I pin 2 core to dom0, the dom0 system can boot up,
> > > > but it can't find any disk devices.
> > > > 
> > > > But when I pin 4 core to dom0, it run OK.
> > > > 
> > > > Does anyone can help me, please.
> > > > 
> > > > I have read this mail, and updated my megaraid_sas to version 05.30.
> > > > http://xen.1045712.n5.nabble.com/HOWTO-Update-megaraid-sas-driver-in-Linux-2-6-32-x-to-make-it-boot-and-work-with-Dell-LSI-RAID-adapts-td3254332.html
> > > > Anyone can help me?
> > > > 
> > > > I download megaraid_sas_05.30 driver from LSI web:
> > > > http://www.lsi.com/storage_home/products_home/internal_raid/megaraid_sas/6gb_s_value_line/sas9260-8i/index.html
> > > 
> > > 
> > > > _______________________________________________
> > > > Xen-devel mailing list
> > > > Xen-devel@lists.xensource.com
> > > > http://lists.xensource.com/xen-devel
> > > 
> > > 
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel
> > 
> > **************************************************
> > 徐安（Andrew Xu）
> > 部门  ：云快线 - 运营支撑中心 - 研发中心
> > 手机  : 18910391796
> > E-mail：xu.an@cloudex.cn
> > 地址  ：北京市朝阳区酒仙桥东路1号M5楼
> > **************************************************
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> **************************************************
> 徐安（Andrew Xu）
> 部门  ：云快线 - 运营支撑中心 - 研发中心
> 手机  : 18910391796
> E-mail：xu.an@cloudex.cn
> 地址  ：北京市朝阳区酒仙桥东路1号M5楼
> **************************************************