From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Andres Lagar-Cavilla" Subject: Re: [PATCH 0 of 3] RFC: x86 memory sharing performance improvements Date: Fri, 13 Apr 2012 09:11:46 -0700 Message-ID: <697daeafdd55160711d03e0c8da33359.squirrel@webmail.lagarcavilla.org> Reply-To: andres@lagarcavilla.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org Cc: andres@gridcentric.ca, Keir Fraser , Tim , Adin Scannell List-Id: xen-devel@lists.xenproject.org Please scratch the RFC tag. I'm done testing this and now I'm asking for inclusion in the tree. Thanks, Andres ----------------------------------------------------------------------- This is an RFC series. I haven't fully tested it yet, but I want the concept to be known as I intend this to be merged prior to the closing of the 4.2 window. The sharing subsystem does not scale elegantly with high degrees of page sharing. The culprit is a reverse map that each shared frame maintains, resolving to all domain pages pointing to the shared frame. Because the rmap is implemented with a O(n) search linked-list, CoW unsharing can result in prolonged search times. The place where this becomes most obvious is during domain destruction, during which all shared p2m entries need to be unshared. Destroying a domain with a lot of sharing could result in minutes of hypervisor freeze-up! Solutions proposed: - Make the p2m clean up of shared entries part of the preemptible, synchronous, domain_kill domctl (as opposed to executing monolithically in the finalize destruction RCU callback) - When a shared frame exceeds an arbitrary ref count, mutate the rmap from a linked list to a hash table. Signed-off-by: Andres Lagar-Cavilla xen/arch/x86/domain.c | 16 +++- xen/arch/x86/mm/mem_sharing.c | 45 ++++++++++ xen/arch/x86/mm/p2m.c | 4 + xen/include/asm-arm/mm.h | 4 + xen/include/asm-x86/domain.h | 1 + xen/include/asm-x86/mem_sharing.h | 10 ++ xen/include/asm-x86/p2m.h | 4 + xen/arch/x86/mm/mem_sharing.c | 142 +++++++++++++++++++++++-------- xen/arch/x86/mm/mem_sharing.c | 170 +++++++++++++++++++++++++++++++++++-- xen/include/asm-x86/mem_sharing.h | 13 ++- 10 files changed, 354 insertions(+), 55 deletions(-)