From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933013Ab1BYWfO (ORCPT ); Fri, 25 Feb 2011 17:35:14 -0500 Received: from zene.cmpxchg.org ([85.214.230.12]:42015 "EHLO zene.cmpxchg.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932105Ab1BYWfN (ORCPT ); Fri, 25 Feb 2011 17:35:13 -0500 Date: Fri, 25 Feb 2011 23:34:44 +0100 From: Johannes Weiner To: Mel Gorman Cc: Andrew Morton , Arthur Marsh , Clemens Ladisch , Andrea Arcangeli , Linux-MM , Linux Kernel Mailing List Subject: Re: [PATCH 1/2] mm: compaction: Minimise the time IRQs are disabled while isolating free pages Message-ID: <20110225223444.GX25382@cmpxchg.org> References: <1298664299-10270-1-git-send-email-mel@csn.ul.ie> <1298664299-10270-2-git-send-email-mel@csn.ul.ie> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1298664299-10270-2-git-send-email-mel@csn.ul.ie> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Feb 25, 2011 at 08:04:58PM +0000, Mel Gorman wrote: > compaction_alloc() isolates free pages to be used as migration targets. > While its scanning, IRQs are disabled on the mistaken assumption the scanning > should be short. Analysis showed that IRQs were in fact being disabled for > substantial time. A simple test was run using large anonymous mappings with > transparent hugepage support enabled to trigger frequent compactions. A > monitor sampled what the worst IRQ-off latencies were and a post-processing > tool found the following; > > Total sampled time IRQs off (not real total time): 22355 > Event compaction_alloc..compaction_alloc 8409 us count 1 > Event compaction_alloc..compaction_alloc 7341 us count 1 > Event compaction_alloc..compaction_alloc 2463 us count 1 > Event compaction_alloc..compaction_alloc 2054 us count 1 > Event shrink_inactive_list..shrink_zone 1864 us count 1 > Event shrink_inactive_list..shrink_zone 88 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __make_request..__blk_run_queue 24 us count 1 > Event __alloc_pages_nodemask..__alloc_pages_nodemask 6 us count 1 > > i.e. compaction is disabled IRQs for a prolonged period of time - 8ms in > one instance. The full report generated by the tool can be found at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-vanilla-micro.report . > This patch reduces the time IRQs are disabled by simply disabling IRQs > at the last possible minute. An updated IRQs-off summary report then > looks like; > > Total sampled time IRQs off (not real total time): 5493 > Event shrink_inactive_list..shrink_zone 1596 us count 1 > Event shrink_inactive_list..shrink_zone 1530 us count 1 > Event shrink_inactive_list..shrink_zone 956 us count 1 > Event shrink_inactive_list..shrink_zone 541 us count 1 > Event shrink_inactive_list..shrink_zone 531 us count 1 > Event split_huge_page..add_to_swap 232 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __wake_up..__wake_up 1 us count 1 > > A full report is again available at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-minimiseirq-free-v1r4-micro.report . > . As should be obvious, IRQ disabled latencies due to compaction are > almost elimimnated for this particular test. > > [aarcange@redhat.com: Fix initialisation of isolated] > Signed-off-by: Mel Gorman Acked-by: Johannes Weiner From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail143.messagelabs.com (mail143.messagelabs.com [216.82.254.35]) by kanga.kvack.org (Postfix) with ESMTP id DB8FE8D0039 for ; Fri, 25 Feb 2011 17:35:12 -0500 (EST) Date: Fri, 25 Feb 2011 23:34:44 +0100 From: Johannes Weiner Subject: Re: [PATCH 1/2] mm: compaction: Minimise the time IRQs are disabled while isolating free pages Message-ID: <20110225223444.GX25382@cmpxchg.org> References: <1298664299-10270-1-git-send-email-mel@csn.ul.ie> <1298664299-10270-2-git-send-email-mel@csn.ul.ie> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1298664299-10270-2-git-send-email-mel@csn.ul.ie> Sender: owner-linux-mm@kvack.org List-ID: To: Mel Gorman Cc: Andrew Morton , Arthur Marsh , Clemens Ladisch , Andrea Arcangeli , Linux-MM , Linux Kernel Mailing List On Fri, Feb 25, 2011 at 08:04:58PM +0000, Mel Gorman wrote: > compaction_alloc() isolates free pages to be used as migration targets. > While its scanning, IRQs are disabled on the mistaken assumption the scanning > should be short. Analysis showed that IRQs were in fact being disabled for > substantial time. A simple test was run using large anonymous mappings with > transparent hugepage support enabled to trigger frequent compactions. A > monitor sampled what the worst IRQ-off latencies were and a post-processing > tool found the following; > > Total sampled time IRQs off (not real total time): 22355 > Event compaction_alloc..compaction_alloc 8409 us count 1 > Event compaction_alloc..compaction_alloc 7341 us count 1 > Event compaction_alloc..compaction_alloc 2463 us count 1 > Event compaction_alloc..compaction_alloc 2054 us count 1 > Event shrink_inactive_list..shrink_zone 1864 us count 1 > Event shrink_inactive_list..shrink_zone 88 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __make_request..__blk_run_queue 24 us count 1 > Event __alloc_pages_nodemask..__alloc_pages_nodemask 6 us count 1 > > i.e. compaction is disabled IRQs for a prolonged period of time - 8ms in > one instance. The full report generated by the tool can be found at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-vanilla-micro.report . > This patch reduces the time IRQs are disabled by simply disabling IRQs > at the last possible minute. An updated IRQs-off summary report then > looks like; > > Total sampled time IRQs off (not real total time): 5493 > Event shrink_inactive_list..shrink_zone 1596 us count 1 > Event shrink_inactive_list..shrink_zone 1530 us count 1 > Event shrink_inactive_list..shrink_zone 956 us count 1 > Event shrink_inactive_list..shrink_zone 541 us count 1 > Event shrink_inactive_list..shrink_zone 531 us count 1 > Event split_huge_page..add_to_swap 232 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __wake_up..__wake_up 1 us count 1 > > A full report is again available at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-minimiseirq-free-v1r4-micro.report . > . As should be obvious, IRQ disabled latencies due to compaction are > almost elimimnated for this particular test. > > [aarcange@redhat.com: Fix initialisation of isolated] > Signed-off-by: Mel Gorman Acked-by: Johannes Weiner -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org