From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Hao, Xudong" Subject: Re: [BUG 1747]Guest could't find bootable device with memory more than 3600M Date: Sat, 8 Jun 2013 07:27:32 +0000 Message-ID: <403610A45A2B5242BD291EDAE8B37D3010E56731@SHSMSX102.ccr.corp.intel.com> References: <51B1FF50.90406@eu.citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Stefano Stabellini , George Dunlap Cc: "Ren, Yongjie" , "xen-devel@lists.xensource.com" , Keir Fraser , Tim Deegan , Paolo Bonzini , "Xu, YongweiX" , "Liu, SongtaoX" List-Id: xen-devel@lists.xenproject.org > -----Original Message----- > From: xen-devel-bounces@lists.xen.org > [mailto:xen-devel-bounces@lists.xen.org] On Behalf Of Stefano Stabellini > Sent: Friday, June 07, 2013 11:57 PM > To: George Dunlap > Cc: Ren, Yongjie; xen-devel@lists.xensource.com; Keir Fraser; Stefano Stabellini; > Tim Deegan; Paolo Bonzini; Xu, YongweiX; Liu, SongtaoX > Subject: Re: [Xen-devel] [BUG 1747]Guest could't find bootable device with > memory more than 3600M > > On Fri, 7 Jun 2013, George Dunlap wrote: > > On 06/07/2013 01:15 PM, Stefano Stabellini wrote: > > > On Fri, 7 Jun 2013, Xu, YongweiX wrote: > > > > Hi Stefano Stabellini, > > > > > > > > I found a new bug "Guest could't find bootable device with memory more > > > > than 3600M". > > > > Attach the link: > > > > http://bugzilla-archived.xenproject.org//bugzilla/show_bug.cgi?id=1857 > > > > > > > > When booting up guest(include Linux&Windows guest) with memory more > than > > > > 3600M,the guest will show "No bootable device" and could not boot up. > Then > > > > the guest will continuous reboot automatically and never found bootable > > > > device. But with guest memory less than or equal to 3600M, boot up > > > > successfully. > > > > > > > > I found this is the qemu(qemu-upstream-unstable) issue, the latest > version > > > > (commit:4597594c61add43725bd207bb498268a058f9cfb) caused this > issue: > > > > xen: start PCI hole at 0xe0000000 (same as pc_init1 and > > > > qemu-xen-traditional) > > > > author Stefano Stabellini > > > > Wed, 5 Jun 2013 19:36:10 +0800 (11:36 +0000) > > > > committer Stefano Stabellini > > > > Wed, 5 Jun 2013 19:36:10 +0800 (11:36 +0000) > > > > commit 4597594c61add43725bd207bb498268a058f9cfb > > > > tree d6831f75f4a7d4ad7a94bd4e33584ac358808ee6 > > > > parent 25adf763933faddcc6a62bf55e1c52909a9bafbb > > > > > > > > Can you debug this issue soon? Thanks. > > > > > > Thank you very much for testing and bisecting the issue! > > > > > > The problem is that by default hvmloader sets PCI_MEM_START to > > > 0xf0000000, then dynamically expands it backwards if needed. > > > It works with qemu-xen-traditional, because it doesn't do any checking > > > when registering memory regions. > > > It doesn't work with qemu-xen because it needs to know where the ram and > > > where the pci hole are when building the emulated machine. > > > We can't have hvmloader increasing the pci hole in a way that overlaps > > > with the guest ram (or where qemu thinks that the guest ram is). > > > It is also worth noting thet seabios has its own view of where the pci > > > hole starts and at the moment is set to 0xe0000000 at build time. > > > > > > I can see two ways of fixing this now: > > > > > > 1) have everybody agree about the pci hole starting at 0xe0000000 > > > > > > - set PCI_MEM_START in hvmloader to 0xe0000000 to match qemu's view > of > > > the pci hole and have a bigger pci hole by default > > > > > > - remove the loop in hvmloader to increase the pci hole > > > > > > - set HVM_BELOW_4G_RAM_END to 0xe0000000, so that > low_mem_pgend is set > > > accordingly in tools/libxc/xc_hvm_build_x86.c:build_hvm_info > > > > > > > > > 2) have everybody agree about the pci hole starting at 0xf0000000 > > > > > > - revert 4597594c61add43725bd207bb498268a058f9cfb in qemu-xen > > > > > > - set BUILD_PCIMEM_START to 0xf0000000 in seabios > > > > > > - remove the loop in hvmloader to increase the pci hole > > > > > > > > > Given that in both cases we need to remove the loop to increase the pci > > > hole in hvmloader I would rather go with 1) and have a bigger pci hole > > > by default to avoid problems with the pci hole being too small. > > > > > > > Did we ever figure out what actually happens on real hardware? Or on KVM? > I > > have a hard time believing that real hardware is hard-coded -- who would be > > the person to ask about that, do you reckon? > > For KVM I am CC'ing Paolo, for real hardware I'll let the Intel guys > speak. > To be clear the question is: > > What happens on real hardware when the BIOS boots and finds out that the > PCI hole is too small to contain all the MMIO regions of the PCI devices? > I do not know what does BIOS do details, but I think it's in 4G above if pci hole is too small. I saw native OS program the pci hole to a very high addr 0x3c0c00000000 if the pci device has large bar size. For Xen hvmloader, it already policy that is: if PCI hole is too small, another MMIO region will be created starting from the high_mem_pgend.