From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753211Ab1GNAdo (ORCPT ); Wed, 13 Jul 2011 20:33:44 -0400 Received: from ipmail07.adl2.internode.on.net ([150.101.137.131]:4150 "EHLO ipmail07.adl2.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752758Ab1GNAdn (ORCPT ); Wed, 13 Jul 2011 20:33:43 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqcCALs4Hk55LCkBgWdsb2JhbABTG6cmFQEBFiYliHrCKw6GLASjQw Date: Thu, 14 Jul 2011 10:33:40 +1000 From: Dave Chinner To: Mel Gorman Cc: Linux-MM , LKML , XFS , Christoph Hellwig , Johannes Weiner , Wu Fengguang , Jan Kara , Rik van Riel , Minchan Kim Subject: Re: [RFC PATCH 0/5] Reduce filesystem writeback from page reclaim (again) Message-ID: <20110714003340.GZ23038@dastard> References: <1310567487-15367-1-git-send-email-mgorman@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1310567487-15367-1-git-send-email-mgorman@suse.de> User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 13, 2011 at 03:31:22PM +0100, Mel Gorman wrote: > (Revisting this from a year ago and following on from the thread > "Re: [PATCH 03/27] xfs: use write_cache_pages for writeback > clustering". Posting an prototype to see if anything obvious is > being missed) Hi Mel, Thanks for picking this up again. The results are definitely promising, but I'd like to see a comparison against simply not doing IO from memory reclaim at all combined with the enhancements in this patchset. After all, that's what I keep asking for (so we can get rid of .writepage altogether), and if the numbers don't add up, then I'll shut up about it. ;) ..... > use-once LRU logic). The command line for fs_mark looked something like > > ./fs_mark -d /tmp/fsmark-2676 -D 100 -N 150 -n 150 -L 25 -t 1 -S0 -s 10485760 > > The machine was booted with "nr_cpus=1 mem=512M" as according to Dave > this triggers the worst behaviour. .... > During testing, a number of monitors were running to gather information > from ftrace in particular. This disrupts the results of course because > recording the information generates IO in itself but I'm ignoring > that for the moment so the effect of the patches can be seen. > > I've posted the raw reports for each filesystem at > > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-ext3/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-ext4/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-btrfs/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-xfs/sandy/comparison.html ..... > Average files per second is increased by a nice percentage albeit > just within the standard deviation. Consider the type of test this is, > variability was inevitable but will double check without monitoring. > > The overhead (time spent in non-filesystem-related activities) is > reduced a *lot* and is a lot less variable. Given that userspace is doing the same amount of work in all test runs, that implies that the userspace process is retaining it's working set hot in the cache over syscalls with this patchset. > Direct reclaim work is significantly reduced going from 37% of all > pages scanned to 1% with all patches applied. This implies that > processes are getting stalled less. And that directly implicates page scanning during direct reclaim as the prime contributor to turfing the application's working set out of the CPU cache.... > Page writes by reclaim is what is motivating this series. It goes > from 14511 pages to 4084 which is a big improvement. We'll see later > if these were anonymous or file-backed pages. Which were anon pages, so this is a major improvement. However, given that there were no dirty pages writen directly by memory reclaim, perhaps we don't need to do IO at all from here and throttling is all that is needed? ;) > Direct reclaim writes were never a problem according to this. That's true. but we disable direct reclaim for other reasons, namely that writeback from direct reclaim blows the stack. Cheers, Dave. -- Dave Chinner david@fromorbit.com From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id p6E0XlZF107278 for ; Wed, 13 Jul 2011 19:33:47 -0500 Received: from ipmail07.adl2.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 3CD476B283 for ; Wed, 13 Jul 2011 17:33:43 -0700 (PDT) Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id nZeHMJOzKsVlzHOX for ; Wed, 13 Jul 2011 17:33:43 -0700 (PDT) Date: Thu, 14 Jul 2011 10:33:40 +1000 From: Dave Chinner Subject: Re: [RFC PATCH 0/5] Reduce filesystem writeback from page reclaim (again) Message-ID: <20110714003340.GZ23038@dastard> References: <1310567487-15367-1-git-send-email-mgorman@suse.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1310567487-15367-1-git-send-email-mgorman@suse.de> List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: xfs-bounces@oss.sgi.com Errors-To: xfs-bounces@oss.sgi.com To: Mel Gorman Cc: Rik van Riel , Jan Kara , LKML , XFS , Christoph Hellwig , Linux-MM , Minchan Kim , Wu Fengguang , Johannes Weiner On Wed, Jul 13, 2011 at 03:31:22PM +0100, Mel Gorman wrote: > (Revisting this from a year ago and following on from the thread > "Re: [PATCH 03/27] xfs: use write_cache_pages for writeback > clustering". Posting an prototype to see if anything obvious is > being missed) Hi Mel, Thanks for picking this up again. The results are definitely promising, but I'd like to see a comparison against simply not doing IO from memory reclaim at all combined with the enhancements in this patchset. After all, that's what I keep asking for (so we can get rid of .writepage altogether), and if the numbers don't add up, then I'll shut up about it. ;) ..... > use-once LRU logic). The command line for fs_mark looked something like > > ./fs_mark -d /tmp/fsmark-2676 -D 100 -N 150 -n 150 -L 25 -t 1 -S0 -s 10485760 > > The machine was booted with "nr_cpus=1 mem=512M" as according to Dave > this triggers the worst behaviour. .... > During testing, a number of monitors were running to gather information > from ftrace in particular. This disrupts the results of course because > recording the information generates IO in itself but I'm ignoring > that for the moment so the effect of the patches can be seen. > > I've posted the raw reports for each filesystem at > > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-ext3/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-ext4/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-btrfs/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-xfs/sandy/comparison.html ..... > Average files per second is increased by a nice percentage albeit > just within the standard deviation. Consider the type of test this is, > variability was inevitable but will double check without monitoring. > > The overhead (time spent in non-filesystem-related activities) is > reduced a *lot* and is a lot less variable. Given that userspace is doing the same amount of work in all test runs, that implies that the userspace process is retaining it's working set hot in the cache over syscalls with this patchset. > Direct reclaim work is significantly reduced going from 37% of all > pages scanned to 1% with all patches applied. This implies that > processes are getting stalled less. And that directly implicates page scanning during direct reclaim as the prime contributor to turfing the application's working set out of the CPU cache.... > Page writes by reclaim is what is motivating this series. It goes > from 14511 pages to 4084 which is a big improvement. We'll see later > if these were anonymous or file-backed pages. Which were anon pages, so this is a major improvement. However, given that there were no dirty pages writen directly by memory reclaim, perhaps we don't need to do IO at all from here and throttling is all that is needed? ;) > Direct reclaim writes were never a problem according to this. That's true. but we disable direct reclaim for other reasons, namely that writeback from direct reclaim blows the stack. Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail138.messagelabs.com (mail138.messagelabs.com [216.82.249.35]) by kanga.kvack.org (Postfix) with SMTP id 1B6976B004A for ; Wed, 13 Jul 2011 20:33:45 -0400 (EDT) Date: Thu, 14 Jul 2011 10:33:40 +1000 From: Dave Chinner Subject: Re: [RFC PATCH 0/5] Reduce filesystem writeback from page reclaim (again) Message-ID: <20110714003340.GZ23038@dastard> References: <1310567487-15367-1-git-send-email-mgorman@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1310567487-15367-1-git-send-email-mgorman@suse.de> Sender: owner-linux-mm@kvack.org List-ID: To: Mel Gorman Cc: Linux-MM , LKML , XFS , Christoph Hellwig , Johannes Weiner , Wu Fengguang , Jan Kara , Rik van Riel , Minchan Kim On Wed, Jul 13, 2011 at 03:31:22PM +0100, Mel Gorman wrote: > (Revisting this from a year ago and following on from the thread > "Re: [PATCH 03/27] xfs: use write_cache_pages for writeback > clustering". Posting an prototype to see if anything obvious is > being missed) Hi Mel, Thanks for picking this up again. The results are definitely promising, but I'd like to see a comparison against simply not doing IO from memory reclaim at all combined with the enhancements in this patchset. After all, that's what I keep asking for (so we can get rid of .writepage altogether), and if the numbers don't add up, then I'll shut up about it. ;) ..... > use-once LRU logic). The command line for fs_mark looked something like > > ./fs_mark -d /tmp/fsmark-2676 -D 100 -N 150 -n 150 -L 25 -t 1 -S0 -s 10485760 > > The machine was booted with "nr_cpus=1 mem=512M" as according to Dave > this triggers the worst behaviour. .... > During testing, a number of monitors were running to gather information > from ftrace in particular. This disrupts the results of course because > recording the information generates IO in itself but I'm ignoring > that for the moment so the effect of the patches can be seen. > > I've posted the raw reports for each filesystem at > > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-ext3/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-ext4/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-btrfs/sandy/comparison.html > http://www.csn.ul.ie/~mel/postings/reclaim-20110713/writeback-xfs/sandy/comparison.html ..... > Average files per second is increased by a nice percentage albeit > just within the standard deviation. Consider the type of test this is, > variability was inevitable but will double check without monitoring. > > The overhead (time spent in non-filesystem-related activities) is > reduced a *lot* and is a lot less variable. Given that userspace is doing the same amount of work in all test runs, that implies that the userspace process is retaining it's working set hot in the cache over syscalls with this patchset. > Direct reclaim work is significantly reduced going from 37% of all > pages scanned to 1% with all patches applied. This implies that > processes are getting stalled less. And that directly implicates page scanning during direct reclaim as the prime contributor to turfing the application's working set out of the CPU cache.... > Page writes by reclaim is what is motivating this series. It goes > from 14511 pages to 4084 which is a big improvement. We'll see later > if these were anonymous or file-backed pages. Which were anon pages, so this is a major improvement. However, given that there were no dirty pages writen directly by memory reclaim, perhaps we don't need to do IO at all from here and throttling is all that is needed? ;) > Direct reclaim writes were never a problem according to this. That's true. but we disable direct reclaim for other reasons, namely that writeback from direct reclaim blows the stack. Cheers, Dave. -- Dave Chinner david@fromorbit.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org