From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752188Ab1B1CBx (ORCPT ); Sun, 27 Feb 2011 21:01:53 -0500 Received: from fgwmail6.fujitsu.co.jp ([192.51.44.36]:40261 "EHLO fgwmail6.fujitsu.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752049Ab1B1CBw (ORCPT ); Sun, 27 Feb 2011 21:01:52 -0500 X-SecurityPolicyCheck-FJ: OK by FujitsuOutboundMailChecker v1.3.1 Date: Mon, 28 Feb 2011 10:55:35 +0900 From: KAMEZAWA Hiroyuki To: Mel Gorman Cc: Andrew Morton , Arthur Marsh , Clemens Ladisch , Andrea Arcangeli , Linux-MM , Linux Kernel Mailing List Subject: Re: [PATCH 1/2] mm: compaction: Minimise the time IRQs are disabled while isolating free pages Message-Id: <20110228105535.cd7bde35.kamezawa.hiroyu@jp.fujitsu.com> In-Reply-To: <1298664299-10270-2-git-send-email-mel@csn.ul.ie> References: <1298664299-10270-1-git-send-email-mel@csn.ul.ie> <1298664299-10270-2-git-send-email-mel@csn.ul.ie> Organization: FUJITSU Co. LTD. X-Mailer: Sylpheed 3.1.0 (GTK+ 2.10.14; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 25 Feb 2011 20:04:58 +0000 Mel Gorman wrote: > compaction_alloc() isolates free pages to be used as migration targets. > While its scanning, IRQs are disabled on the mistaken assumption the scanning > should be short. Analysis showed that IRQs were in fact being disabled for > substantial time. A simple test was run using large anonymous mappings with > transparent hugepage support enabled to trigger frequent compactions. A > monitor sampled what the worst IRQ-off latencies were and a post-processing > tool found the following; > > Total sampled time IRQs off (not real total time): 22355 > Event compaction_alloc..compaction_alloc 8409 us count 1 > Event compaction_alloc..compaction_alloc 7341 us count 1 > Event compaction_alloc..compaction_alloc 2463 us count 1 > Event compaction_alloc..compaction_alloc 2054 us count 1 > Event shrink_inactive_list..shrink_zone 1864 us count 1 > Event shrink_inactive_list..shrink_zone 88 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __make_request..__blk_run_queue 24 us count 1 > Event __alloc_pages_nodemask..__alloc_pages_nodemask 6 us count 1 > > i.e. compaction is disabled IRQs for a prolonged period of time - 8ms in > one instance. The full report generated by the tool can be found at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-vanilla-micro.report . > This patch reduces the time IRQs are disabled by simply disabling IRQs > at the last possible minute. An updated IRQs-off summary report then > looks like; > > Total sampled time IRQs off (not real total time): 5493 > Event shrink_inactive_list..shrink_zone 1596 us count 1 > Event shrink_inactive_list..shrink_zone 1530 us count 1 > Event shrink_inactive_list..shrink_zone 956 us count 1 > Event shrink_inactive_list..shrink_zone 541 us count 1 > Event shrink_inactive_list..shrink_zone 531 us count 1 > Event split_huge_page..add_to_swap 232 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __wake_up..__wake_up 1 us count 1 > > A full report is again available at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-minimiseirq-free-v1r4-micro.report . > . As should be obvious, IRQ disabled latencies due to compaction are > almost elimimnated for this particular test. > > [aarcange@redhat.com: Fix initialisation of isolated] > Signed-off-by: Mel Gorman Reviewed-by: KAMEZAWA Hiroyuki From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail137.messagelabs.com (mail137.messagelabs.com [216.82.249.19]) by kanga.kvack.org (Postfix) with ESMTP id 9EC088D0039 for ; Sun, 27 Feb 2011 21:01:54 -0500 (EST) Received: from m1.gw.fujitsu.co.jp (unknown [10.0.50.71]) by fgwmail5.fujitsu.co.jp (Postfix) with ESMTP id 34C593EE0C2 for ; Mon, 28 Feb 2011 11:01:51 +0900 (JST) Received: from smail (m1 [127.0.0.1]) by outgoing.m1.gw.fujitsu.co.jp (Postfix) with ESMTP id 1913845DE56 for ; Mon, 28 Feb 2011 11:01:51 +0900 (JST) Received: from s1.gw.fujitsu.co.jp (s1.gw.fujitsu.co.jp [10.0.50.91]) by m1.gw.fujitsu.co.jp (Postfix) with ESMTP id EBA1445DE54 for ; Mon, 28 Feb 2011 11:01:50 +0900 (JST) Received: from s1.gw.fujitsu.co.jp (localhost.localdomain [127.0.0.1]) by s1.gw.fujitsu.co.jp (Postfix) with ESMTP id D6685E08003 for ; Mon, 28 Feb 2011 11:01:50 +0900 (JST) Received: from m107.s.css.fujitsu.com (m107.s.css.fujitsu.com [10.249.87.107]) by s1.gw.fujitsu.co.jp (Postfix) with ESMTP id 82D7BE38003 for ; Mon, 28 Feb 2011 11:01:50 +0900 (JST) Date: Mon, 28 Feb 2011 10:55:35 +0900 From: KAMEZAWA Hiroyuki Subject: Re: [PATCH 1/2] mm: compaction: Minimise the time IRQs are disabled while isolating free pages Message-Id: <20110228105535.cd7bde35.kamezawa.hiroyu@jp.fujitsu.com> In-Reply-To: <1298664299-10270-2-git-send-email-mel@csn.ul.ie> References: <1298664299-10270-1-git-send-email-mel@csn.ul.ie> <1298664299-10270-2-git-send-email-mel@csn.ul.ie> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Mel Gorman Cc: Andrew Morton , Arthur Marsh , Clemens Ladisch , Andrea Arcangeli , Linux-MM , Linux Kernel Mailing List On Fri, 25 Feb 2011 20:04:58 +0000 Mel Gorman wrote: > compaction_alloc() isolates free pages to be used as migration targets. > While its scanning, IRQs are disabled on the mistaken assumption the scanning > should be short. Analysis showed that IRQs were in fact being disabled for > substantial time. A simple test was run using large anonymous mappings with > transparent hugepage support enabled to trigger frequent compactions. A > monitor sampled what the worst IRQ-off latencies were and a post-processing > tool found the following; > > Total sampled time IRQs off (not real total time): 22355 > Event compaction_alloc..compaction_alloc 8409 us count 1 > Event compaction_alloc..compaction_alloc 7341 us count 1 > Event compaction_alloc..compaction_alloc 2463 us count 1 > Event compaction_alloc..compaction_alloc 2054 us count 1 > Event shrink_inactive_list..shrink_zone 1864 us count 1 > Event shrink_inactive_list..shrink_zone 88 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __make_request..__blk_run_queue 24 us count 1 > Event __alloc_pages_nodemask..__alloc_pages_nodemask 6 us count 1 > > i.e. compaction is disabled IRQs for a prolonged period of time - 8ms in > one instance. The full report generated by the tool can be found at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-vanilla-micro.report . > This patch reduces the time IRQs are disabled by simply disabling IRQs > at the last possible minute. An updated IRQs-off summary report then > looks like; > > Total sampled time IRQs off (not real total time): 5493 > Event shrink_inactive_list..shrink_zone 1596 us count 1 > Event shrink_inactive_list..shrink_zone 1530 us count 1 > Event shrink_inactive_list..shrink_zone 956 us count 1 > Event shrink_inactive_list..shrink_zone 541 us count 1 > Event shrink_inactive_list..shrink_zone 531 us count 1 > Event split_huge_page..add_to_swap 232 us count 1 > Event save_args..call_softirq 36 us count 1 > Event save_args..call_softirq 35 us count 2 > Event __wake_up..__wake_up 1 us count 1 > > A full report is again available at > http://www.csn.ul.ie/~mel/postings/minfree-20110225/irqsoff-minimiseirq-free-v1r4-micro.report . > . As should be obvious, IRQ disabled latencies due to compaction are > almost elimimnated for this particular test. > > [aarcange@redhat.com: Fix initialisation of isolated] > Signed-off-by: Mel Gorman Reviewed-by: KAMEZAWA Hiroyuki -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org