From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759351Ab1EMPVx (ORCPT ); Fri, 13 May 2011 11:21:53 -0400 Received: from smtp101.prem.mail.ac4.yahoo.com ([76.13.13.40]:26915 "HELO smtp101.prem.mail.ac4.yahoo.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with SMTP id S1756163Ab1EMPVv (ORCPT ); Fri, 13 May 2011 11:21:51 -0400 X-Yahoo-SMTP: _Dag8S.swBC1p4FJKLCXbs8NQzyse1SYSgnAbY0- X-YMail-OSG: Fj9NskAVM1kB05MUsIuDOvVkoqZAVluS7gNLbKQo5kea0dC Wrz56PH88MSdgCJaZhulpjeQ3ZLfvUNQZgz2gplZqEujI4F5RQaCUqVjkGvj lb8vDGstbuahN0WUGZUC_ZFeKp.Po4vmD0ZTJHKTvlM6PmOUPt8_6k6y55i1 g00jR0CfU47bhNZmAgg9OG9I7eMps_w4lfo4pD1.7TpTr0Q3kGIDXiKSEg55 IRe9hEk_IGJpDzFAbp2.WD3PZiQgFQO28CG98LOCvjapSRcl9pUfpJW2.wbJ WHYew2i9Xa5vID83zHiW1J6s8Cjf74RP6efi4t6uoG_8X7snSWHMrxt2H5gn GXXuMNErLJErv3L.Gub2_mL5O X-Yahoo-Newman-Property: ymail-3 Date: Fri, 13 May 2011 10:21:46 -0500 (CDT) From: Christoph Lameter X-X-Sender: cl@router.home To: Mel Gorman cc: Andrew Morton , James Bottomley , Colin King , Raghavendra D Prabhu , Jan Kara , Chris Mason , Pekka Enberg , Rik van Riel , Johannes Weiner , linux-fsdevel , linux-mm , linux-kernel , linux-ext4 Subject: Re: [PATCH 0/4] Reduce impact to overall system of SLUB using high-order allocations V2 In-Reply-To: <1305295404-12129-1-git-send-email-mgorman@suse.de> Message-ID: References: <1305295404-12129-1-git-send-email-mgorman@suse.de> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 13 May 2011, Mel Gorman wrote: > SLUB using high orders is the trigger but not the root cause as SLUB > has been using high orders for a while. The following four patches > aim to fix the problems in reclaim while reducing the cost for SLUB > using those high orders. > > Patch 1 corrects logic introduced by commit [1741c877: mm: > kswapd: keep kswapd awake for high-order allocations until > a percentage of the node is balanced] to allow kswapd to > go to sleep when balanced for high orders. The above looks good. > Patch 2 prevents kswapd waking up in response to SLUBs speculative > use of high orders. Not sure if that is necessary since it seems that we triggered kswapd before? Why not continue to do it? Once kswapd has enough higher order pages kswapd should no longer be triggered right? > Patch 3 further reduces the cost by prevent SLUB entering direct > compaction or reclaim paths on the grounds that falling > back to order-0 should be cheaper. Its cheaper for reclaim path true but more expensive in terms of SLUBs management costs of the data and it also increases the memory wasted. A higher order means denser packing of objects less page management overhead. Fallback is not for free. Reasonable effort should be made to allocate the page order requested. > Patch 4 notes that even when kswapd is failing to keep up with > allocation requests, it should still go to sleep when its > quota has expired to prevent it spinning. Looks good too. Overall, it looks like the compaction logic and the modifications to reclaim introduced recently with the intend to increase the amount of physically contiguous memory is not working as expected. SLUBs chance of getting higher order pages should be *increasing* as a result of these changes. The above looks like the chances are decreasing now. This is a matter of future concern. The metadata management overhead in the kernel is continually increasing since memory sizes keep growing and we typically manage memory in 4k chunks. Through large allocation sizes we can reduce that management overhead but we can only do this if we have an effective way of defragmenting memory to get longer contiguous chunks that can be managed to a single page struct. Please make sure that compaction and related measures really work properly. The patches suggest that the recent modifications are not improving the situation. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail203.messagelabs.com (mail203.messagelabs.com [216.82.254.243]) by kanga.kvack.org (Postfix) with SMTP id 4C3596B0012 for ; Fri, 13 May 2011 11:21:52 -0400 (EDT) Date: Fri, 13 May 2011 10:21:46 -0500 (CDT) From: Christoph Lameter Subject: Re: [PATCH 0/4] Reduce impact to overall system of SLUB using high-order allocations V2 In-Reply-To: <1305295404-12129-1-git-send-email-mgorman@suse.de> Message-ID: References: <1305295404-12129-1-git-send-email-mgorman@suse.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-linux-mm@kvack.org List-ID: To: Mel Gorman Cc: Andrew Morton , James Bottomley , Colin King , Raghavendra D Prabhu , Jan Kara , Chris Mason , Pekka Enberg , Rik van Riel , Johannes Weiner , linux-fsdevel , linux-mm , linux-kernel , linux-ext4 On Fri, 13 May 2011, Mel Gorman wrote: > SLUB using high orders is the trigger but not the root cause as SLUB > has been using high orders for a while. The following four patches > aim to fix the problems in reclaim while reducing the cost for SLUB > using those high orders. > > Patch 1 corrects logic introduced by commit [1741c877: mm: > kswapd: keep kswapd awake for high-order allocations until > a percentage of the node is balanced] to allow kswapd to > go to sleep when balanced for high orders. The above looks good. > Patch 2 prevents kswapd waking up in response to SLUBs speculative > use of high orders. Not sure if that is necessary since it seems that we triggered kswapd before? Why not continue to do it? Once kswapd has enough higher order pages kswapd should no longer be triggered right? > Patch 3 further reduces the cost by prevent SLUB entering direct > compaction or reclaim paths on the grounds that falling > back to order-0 should be cheaper. Its cheaper for reclaim path true but more expensive in terms of SLUBs management costs of the data and it also increases the memory wasted. A higher order means denser packing of objects less page management overhead. Fallback is not for free. Reasonable effort should be made to allocate the page order requested. > Patch 4 notes that even when kswapd is failing to keep up with > allocation requests, it should still go to sleep when its > quota has expired to prevent it spinning. Looks good too. Overall, it looks like the compaction logic and the modifications to reclaim introduced recently with the intend to increase the amount of physically contiguous memory is not working as expected. SLUBs chance of getting higher order pages should be *increasing* as a result of these changes. The above looks like the chances are decreasing now. This is a matter of future concern. The metadata management overhead in the kernel is continually increasing since memory sizes keep growing and we typically manage memory in 4k chunks. Through large allocation sizes we can reduce that management overhead but we can only do this if we have an effective way of defragmenting memory to get longer contiguous chunks that can be managed to a single page struct. Please make sure that compaction and related measures really work properly. The patches suggest that the recent modifications are not improving the situation. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/ Don't email: email@kvack.org