From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:40824) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atXzG-0005Rj-3i for qemu-devel@nongnu.org; Fri, 22 Apr 2016 06:06:59 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1atXzB-0000fk-VK for qemu-devel@nongnu.org; Fri, 22 Apr 2016 06:06:58 -0400 Received: from mx2.suse.de ([195.135.220.15]:46859) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1atXzB-0000fP-OD for qemu-devel@nongnu.org; Fri, 22 Apr 2016 06:06:53 -0400 References: <20160421162348.GA24178@cbox> <57192EED.2040501@suse.de> <20160422100118.GD25288@cbox> From: Alexander Graf Message-ID: <5719F7BC.4090703@suse.de> Date: Fri, 22 Apr 2016 12:06:52 +0200 MIME-Version: 1.0 In-Reply-To: <20160422100118.GD25288@cbox> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] Performance regression using KVM/ARM List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Christoffer Dall Cc: Peter Maydell , "Michael S. Tsirkin" , qemu-devel@nongnu.org, Marc Zyngier , Paolo Bonzini On 04/22/2016 12:01 PM, Christoffer Dall wrote: > On Thu, Apr 21, 2016 at 09:50:05PM +0200, Alexander Graf wrote: >> >> On 21.04.16 18:23, Christoffer Dall wrote: >>> Hi, >>> >>> Commit 9fac18f (oslib: allocate PROT_NONE pages on top of RAM, >>> 2015-09-10) had the unfortunate side effect that memory slots registered >>> with KVM no longer contain a userspace address that is aligned to a 2M >>> boundary, causing the use of THP to fail in the kernel. >>> >>> I fail to see where in the QEMU code we should be asking for a 2M >>> alignment of our memory region. Can someone help pointing me to the >>> right place to fix this or suggest a patch? >>> >>> This causes a performance regssion of hackbench on KVM/ARM of about 62% >>> compared to the workload running with THP. >>> >>> We have verified that this is indeed the cause of the failure by adding >>> various prints to QEMU and the kernel, but unfortunatley my QEMU >>> knowledge is not sufficient for me to fix it myself. >>> >>> Any help would be much appreciated! >> The code changed quite heavily since I last looked at it, but could you >> please try whether the (untested) patch below makes a difference? >> >> > Unfortunately this doesn't make any difference. It feels to me like > we're missing specifying a 2M alignemnt in QEMU somewhere, but I can't > properly understand the links between the actual allocation, registering > mem slots with the KVM part of QEMU, and actually setting up KVM user > memory regions. > > What has to happen is that the resulting struct > kvm_userspace_memory_region() has the same alignment offset from 2M (the > huge page size) of the ->guest_phys_addr and ->userspace-addr fields. Well, I would expect that the guest address space is always very big aligned - and definitely at least 2MB - so we're safe there. That means we only need to align the qemu virtual address. There used to be a memalign() call for that, but it got replaced with direct mmap() and then a lot of code changed on top. Looking at the logs, I'm sure Paolo knows the answer though :) Alex