From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:56045) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e33XI-0006Lj-GV for qemu-devel@nongnu.org; Fri, 13 Oct 2017 13:14:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e33XF-0000ZT-3r for qemu-devel@nongnu.org; Fri, 13 Oct 2017 13:14:12 -0400 Received: from mx1.redhat.com ([209.132.183.28]:56218) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e33XE-0000Yf-Tj for qemu-devel@nongnu.org; Fri, 13 Oct 2017 13:14:09 -0400 Date: Fri, 13 Oct 2017 11:14:03 -0600 From: Alex Williamson Message-ID: <20171013111403.293919fe@t450s.home> In-Reply-To: <20171013170143.GB3370@work-vm> References: <20171013170143.GB3370@work-vm> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] host physical address width issues/questions for x86_64 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: Prasad Singamsetty , qemu-devel@nongnu.org, pbonzini@redhat.com, Sunit Jain , ehabkost@redhat.com, rth@twiddle.net, Peter Xu On Fri, 13 Oct 2017 18:01:44 +0100 "Dr. David Alan Gilbert" wrote: > * Prasad Singamsetty (prasad.singamsetty@oracle.com) wrote: > > Hi, > > > > I am new to the alias. I have some questions on this subject > > and seek some clarifications from the experts in the team. > > I ran into a couple of issues when I tried with large configuration > > ( >= 1TB memory, > 255 CPUs) for x86_64 guest machine. > > > > 1. QEMU uses the default value of 40 (TCG_PHYS_ADDR_BITS) for address > > width if user has not specified phys-bits or host-phys-bits=true > > property. The default value is obviously not sufficient and > > causing guest kernel to crash if configured with >= 1TB > > memory. Depending on the linux kernel version in the guest the > > panic was in different code paths. The workaround is for the > > user to specify the phys-bits property or set the property > > host-phys-bits=true. > > > > QUESTIONS: ... > > 2. host_address_width in DMAR table structure > > > > In this case, the default value is set to 39 > > (VTD_HOST_ADDRESS_WIDTH - 1). With interrupt remapping > > enabled for the intel iommu and the guest is configured > > with > 255 cpus and >= 1TB memory, the guest kernel hangs > > during boot up. This need to be fixed. > > > > QUESTION: > > The question here again is can we fix this to use the > > real address width from the host as the default? > > I don't know DMAR stuff; chatting to Alex (cc'd) it does sound > like that's an ommission that should be fixed. [CC +Peter] On physical hardware VT-d supports either 39 or 48 bit address widths and generally you'd expect a sufficiently capable IOMMU to be matched with the CPU. Seems QEMU has only implemented a lower bit width and it should probably be forcing phys bits of the VM to 39 to match until the extended width can be implemented. Thanks, Alex