From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53096) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UmfQs-0002Fv-1Y for qemu-devel@nongnu.org; Wed, 12 Jun 2013 03:25:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UmfQp-0000iF-Bt for qemu-devel@nongnu.org; Wed, 12 Jun 2013 03:25:25 -0400 Received: from nat28.tlf.novell.com ([130.57.49.28]:50299) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UmfQp-0000i0-2L for qemu-devel@nongnu.org; Wed, 12 Jun 2013 03:25:23 -0400 Message-Id: <51B83E7A02000078000DD6E9@nat28.tlf.novell.com> Date: Wed, 12 Jun 2013 08:25:14 +0100 From: "Jan Beulich" References: <51B1FF50.90406@eu.citrix.com> <403610A45A2B5242BD291EDAE8B37D3010E56731@SHSMSX102.ccr.corp.intel.com> In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: Re: [Qemu-devel] [Xen-devel] [BUG 1747]Guest could't find bootable device with memory more than 3600M List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: George Dunlap , Stefano Stabellini Cc: Tim Deegan , Yongjie Ren , "xen-devel@lists.xensource.com" , Keir Fraser , Ian Campbell , hanweidong@huawei.com, Xudong Hao , yanqiangjun@huawei.com, luonengjun@huawei.com, qemu-devel@nongnu.org, wangzhenguo@huawei.com, xiaowei.yang@huawei.com, arei.gonglei@huawei.com, Paolo Bonzini , YongweiX Xu , SongtaoX Liu >>> On 11.06.13 at 19:26, Stefano Stabellini wrote: > I went through the code that maps the PCI MMIO regions in hvmloader > (tools/firmware/hvmloader/pci.c:pci_setup) and it looks like it already > maps the PCI region to high memory if the PCI bar is 64-bit and the MMIO > region is larger than 512MB. >=20 > Maybe we could just relax this condition and map the device memory to > high memory no matter the size of the MMIO region if the PCI bar is > 64-bit? I can only recommend not to: For one, guests not using PAE or PSE-36 can't map such space at all (and older OSes may not properly deal with 64-bit BARs at all). And then one would generally expect this allocation to be done top down (to minimize risk of running into RAM), and doing so is going to present further risks of incompatibilities with guest OSes (Linux for example learned only in 2.6.36 that PFNs in ioremap() can exceed 32 bits, but even in 3.10-rc5 ioremap_pte_range(), while using "u64 pfn", passes the PFN to pfn_pte(), the respective parameter of which is "unsigned long"). I think this ought to be done in an iterative process - if all MMIO regions together don't fit below 4G, the biggest one should be moved up beyond 4G first, followed by the next to biggest one etc. And, just like many BIOSes have, there ought to be a guest (config) controlled option to shrink the RAM portion below 4G allowing more MMIO blocks to fit. Finally we shouldn't forget the option of not doing any assignment at all in the BIOS, allowing/forcing the OS to use suitable address ranges. Of course any OS is permitted to re-assign resources, but I think they will frequently prefer to avoid re-assignment if already done by the BIOS. Jan From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Jan Beulich" Subject: Re: [Xen-devel] [BUG 1747]Guest could't find bootable device with memory more than 3600M Date: Wed, 12 Jun 2013 08:25:14 +0100 Message-ID: <51B83E7A02000078000DD6E9@nat28.tlf.novell.com> References: <51B1FF50.90406@eu.citrix.com> <403610A45A2B5242BD291EDAE8B37D3010E56731@SHSMSX102.ccr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: Content-Disposition: inline List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org To: George Dunlap , Stefano Stabellini Cc: Tim Deegan , Yongjie Ren , "xen-devel@lists.xensource.com" , Keir Fraser , Ian Campbell , hanweidong@huawei.com, Xudong Hao , yanqiangjun@huawei.com, luonengjun@huawei.com, qemu-devel@nongnu.org, wangzhenguo@huawei.com, xiaowei.yang@huawei.com, arei.gonglei@huawei.com, Paolo Bonzini , YongweiX Xu , SongtaoX Liu List-Id: xen-devel@lists.xenproject.org >>> On 11.06.13 at 19:26, Stefano Stabellini wrote: > I went through the code that maps the PCI MMIO regions in hvmloader > (tools/firmware/hvmloader/pci.c:pci_setup) and it looks like it already > maps the PCI region to high memory if the PCI bar is 64-bit and the MMIO > region is larger than 512MB. >=20 > Maybe we could just relax this condition and map the device memory to > high memory no matter the size of the MMIO region if the PCI bar is > 64-bit? I can only recommend not to: For one, guests not using PAE or PSE-36 can't map such space at all (and older OSes may not properly deal with 64-bit BARs at all). And then one would generally expect this allocation to be done top down (to minimize risk of running into RAM), and doing so is going to present further risks of incompatibilities with guest OSes (Linux for example learned only in 2.6.36 that PFNs in ioremap() can exceed 32 bits, but even in 3.10-rc5 ioremap_pte_range(), while using "u64 pfn", passes the PFN to pfn_pte(), the respective parameter of which is "unsigned long"). I think this ought to be done in an iterative process - if all MMIO regions together don't fit below 4G, the biggest one should be moved up beyond 4G first, followed by the next to biggest one etc. And, just like many BIOSes have, there ought to be a guest (config) controlled option to shrink the RAM portion below 4G allowing more MMIO blocks to fit. Finally we shouldn't forget the option of not doing any assignment at all in the BIOS, allowing/forcing the OS to use suitable address ranges. Of course any OS is permitted to re-assign resources, but I think they will frequently prefer to avoid re-assignment if already done by the BIOS. Jan