From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753033Ab1GMXmJ (ORCPT ); Wed, 13 Jul 2011 19:42:09 -0400 Received: from ipmail07.adl2.internode.on.net ([150.101.137.131]:3984 "EHLO ipmail07.adl2.internode.on.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751395Ab1GMXl7 (ORCPT ); Wed, 13 Jul 2011 19:41:59 -0400 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqYCAJsqHk55LCkBgWdsb2JhbABTp0EVAQEWJiXLQg6DJ4MFBKND Date: Thu, 14 Jul 2011 09:41:50 +1000 From: Dave Chinner To: Mel Gorman Cc: Linux-MM , LKML , XFS , Christoph Hellwig , Johannes Weiner , Wu Fengguang , Jan Kara , Rik van Riel , Minchan Kim Subject: Re: [PATCH 3/5] mm: vmscan: Throttle reclaim if encountering too many dirty pages under writeback Message-ID: <20110713234150.GW23038@dastard> References: <1310567487-15367-1-git-send-email-mgorman@suse.de> <1310567487-15367-4-git-send-email-mgorman@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1310567487-15367-4-git-send-email-mgorman@suse.de> User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 13, 2011 at 03:31:25PM +0100, Mel Gorman wrote: > Workloads that are allocating frequently and writing files place a > large number of dirty pages on the LRU. With use-once logic, it is > possible for them to reach the end of the LRU quickly requiring the > reclaimer to scan more to find clean pages. Ordinarily, processes that > are dirtying memory will get throttled by dirty balancing but this > is a global heuristic and does not take into account that LRUs are > maintained on a per-zone basis. This can lead to a situation whereby > reclaim is scanning heavily, skipping over a large number of pages > under writeback and recycling them around the LRU consuming CPU. > > This patch checks how many of the number of pages isolated from the > LRU were dirty. If a percentage of them are dirty, the process will be > throttled if a blocking device is congested or the zone being scanned > is marked congested. The percentage that must be dirty depends on > the priority. At default priority, all of them must be dirty. At > DEF_PRIORITY-1, 50% of them must be dirty, DEF_PRIORITY-2, 25% > etc. i.e. as pressure increases the greater the likelihood the process > will get throttled to allow the flusher threads to make some progress. It still doesn't take into account how many pages under writeback were skipped. If there are lots of pages that are under writeback, I think we still want to throttle to give IO a chance to complete and clean those pages before scanning again.... Cheers, Dave. -- Dave Chinner david@fromorbit.com From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15]) by oss.sgi.com (8.14.3/8.14.3/SuSE Linux 0.8) with ESMTP id p6DNg0Bk105452 for ; Wed, 13 Jul 2011 18:42:00 -0500 Received: from ipmail07.adl2.internode.on.net (localhost [127.0.0.1]) by cuda.sgi.com (Spam Firewall) with ESMTP id 5C35F1D8556F for ; Wed, 13 Jul 2011 16:41:58 -0700 (PDT) Received: from ipmail07.adl2.internode.on.net (ipmail07.adl2.internode.on.net [150.101.137.131]) by cuda.sgi.com with ESMTP id NhasqgCdyKGOaMMU for ; Wed, 13 Jul 2011 16:41:58 -0700 (PDT) Date: Thu, 14 Jul 2011 09:41:50 +1000 From: Dave Chinner Subject: Re: [PATCH 3/5] mm: vmscan: Throttle reclaim if encountering too many dirty pages under writeback Message-ID: <20110713234150.GW23038@dastard> References: <1310567487-15367-1-git-send-email-mgorman@suse.de> <1310567487-15367-4-git-send-email-mgorman@suse.de> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <1310567487-15367-4-git-send-email-mgorman@suse.de> List-Id: XFS Filesystem from SGI List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: xfs-bounces@oss.sgi.com Errors-To: xfs-bounces@oss.sgi.com To: Mel Gorman Cc: Rik van Riel , Jan Kara , LKML , XFS , Christoph Hellwig , Linux-MM , Minchan Kim , Wu Fengguang , Johannes Weiner On Wed, Jul 13, 2011 at 03:31:25PM +0100, Mel Gorman wrote: > Workloads that are allocating frequently and writing files place a > large number of dirty pages on the LRU. With use-once logic, it is > possible for them to reach the end of the LRU quickly requiring the > reclaimer to scan more to find clean pages. Ordinarily, processes that > are dirtying memory will get throttled by dirty balancing but this > is a global heuristic and does not take into account that LRUs are > maintained on a per-zone basis. This can lead to a situation whereby > reclaim is scanning heavily, skipping over a large number of pages > under writeback and recycling them around the LRU consuming CPU. > > This patch checks how many of the number of pages isolated from the > LRU were dirty. If a percentage of them are dirty, the process will be > throttled if a blocking device is congested or the zone being scanned > is marked congested. The percentage that must be dirty depends on > the priority. At default priority, all of them must be dirty. At > DEF_PRIORITY-1, 50% of them must be dirty, DEF_PRIORITY-2, 25% > etc. i.e. as pressure increases the greater the likelihood the process > will get throttled to allow the flusher threads to make some progress. It still doesn't take into account how many pages under writeback were skipped. If there are lots of pages that are under writeback, I think we still want to throttle to give IO a chance to complete and clean those pages before scanning again.... Cheers, Dave. -- Dave Chinner david@fromorbit.com _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail143.messagelabs.com (mail143.messagelabs.com [216.82.254.35]) by kanga.kvack.org (Postfix) with SMTP id 391B26B004A for ; Wed, 13 Jul 2011 19:42:00 -0400 (EDT) Date: Thu, 14 Jul 2011 09:41:50 +1000 From: Dave Chinner Subject: Re: [PATCH 3/5] mm: vmscan: Throttle reclaim if encountering too many dirty pages under writeback Message-ID: <20110713234150.GW23038@dastard> References: <1310567487-15367-1-git-send-email-mgorman@suse.de> <1310567487-15367-4-git-send-email-mgorman@suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1310567487-15367-4-git-send-email-mgorman@suse.de> Sender: owner-linux-mm@kvack.org List-ID: To: Mel Gorman Cc: Linux-MM , LKML , XFS , Christoph Hellwig , Johannes Weiner , Wu Fengguang , Jan Kara , Rik van Riel , Minchan Kim On Wed, Jul 13, 2011 at 03:31:25PM +0100, Mel Gorman wrote: > Workloads that are allocating frequently and writing files place a > large number of dirty pages on the LRU. With use-once logic, it is > possible for them to reach the end of the LRU quickly requiring the > reclaimer to scan more to find clean pages. Ordinarily, processes that > are dirtying memory will get throttled by dirty balancing but this > is a global heuristic and does not take into account that LRUs are > maintained on a per-zone basis. This can lead to a situation whereby > reclaim is scanning heavily, skipping over a large number of pages > under writeback and recycling them around the LRU consuming CPU. > > This patch checks how many of the number of pages isolated from the > LRU were dirty. If a percentage of them are dirty, the process will be > throttled if a blocking device is congested or the zone being scanned > is marked congested. The percentage that must be dirty depends on > the priority. At default priority, all of them must be dirty. At > DEF_PRIORITY-1, 50% of them must be dirty, DEF_PRIORITY-2, 25% > etc. i.e. as pressure increases the greater the likelihood the process > will get throttled to allow the flusher threads to make some progress. It still doesn't take into account how many pages under writeback were skipped. If there are lots of pages that are under writeback, I think we still want to throttle to give IO a chance to complete and clean those pages before scanning again.... Cheers, Dave. -- Dave Chinner david@fromorbit.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org