All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mel Gorman <mgorman@suse.de>
To: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: minchan.kim@gmail.com, James.Bottomley@hansenpartnership.com,
	akpm@linux-foundation.org, colin.king@canonical.com,
	raghu.prabhu13@gmail.com, jack@suse.cz, chris.mason@oracle.com,
	cl@linux.com, penberg@kernel.org, riel@redhat.com,
	hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	linux-ext4@vger.kernel.org
Subject: Re: [PATCH 4/4] mm: vmscan: If kswapd has been running too long, allow it to sleep
Date: Wed, 18 May 2011 10:57:20 +0100	[thread overview]
Message-ID: <20110518095720.GQ5279@suse.de> (raw)
In-Reply-To: <4DD31221.3060205@jp.fujitsu.com>

On Wed, May 18, 2011 at 09:26:09AM +0900, KOSAKI Motohiro wrote:
> >Lets see;
> >
> >shrink_page_list() only applies if inactive pages were isolated
> >	which in turn may not happen if all_unreclaimable is set in
> >	shrink_zones(). If for whatver reason, all_unreclaimable is
> >	set on all zones, we can miss calling cond_resched().
> >
> >shrink_slab only applies if we are reclaiming slab pages. If the first
> >	shrinker returns -1, we do not call cond_resched(). If that
> >	first shrinker is dcache and __GFP_FS is not set, direct
> >	reclaimers will not shrink at all. However, if there are
> >	enough of them running or if one of the other shrinkers
> >	is running for a very long time, kswapd could be starved
> >	acquiring the shrinker_rwsem and never reaching the
> >	cond_resched().
> 
> OK.
> 
> 
> >
> >balance_pgdat() only calls cond_resched if the zones are not
> >	balanced. For a high-order allocation that is balanced, it
> >	checks order-0 again. During that window, order-0 might have
> >	become unbalanced so it loops again for order-0 and returns
> >	that was reclaiming for order-0 to kswapd(). It can then find
> >	that a caller has rewoken kswapd for a high-order and re-enters
> >	balance_pgdat() without ever have called cond_resched().
> 
> Then, Shouldn't balance_pgdat() call cond_resched() unconditionally?
> The problem is NOT 100% cpu consumption. if kswapd will sleep, other
> processes need to reclaim old pages. The problem is, kswapd doesn't
> invoke context switch and other tasks hang-up.
> 

Which the shrink_slab patch does (either version). What's the gain from
sprinkling more cond_resched() around? If you think there is, submit
another pair of patches (include patch 1 from this series) but I'm not
seeing the advantage myself.

> 
> >While it appears unlikely, there are bad conditions which can result
> >in cond_resched() being avoided.
> >
> 
> 

-- 
Mel Gorman
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Mel Gorman <mgorman@suse.de>
To: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: minchan.kim@gmail.com, James.Bottomley@hansenpartnership.com,
	akpm@linux-foundation.org, colin.king@canonical.com,
	raghu.prabhu13@gmail.com, jack@suse.cz, chris.mason@oracle.com,
	cl@linux.com, penberg@kernel.org, riel@redhat.com,
	hannes@cmpxchg.org, linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	linux-ext4@vger.kernel.org
Subject: Re: [PATCH 4/4] mm: vmscan: If kswapd has been running too long, allow it to sleep
Date: Wed, 18 May 2011 10:57:20 +0100	[thread overview]
Message-ID: <20110518095720.GQ5279@suse.de> (raw)
In-Reply-To: <4DD31221.3060205@jp.fujitsu.com>

On Wed, May 18, 2011 at 09:26:09AM +0900, KOSAKI Motohiro wrote:
> >Lets see;
> >
> >shrink_page_list() only applies if inactive pages were isolated
> >	which in turn may not happen if all_unreclaimable is set in
> >	shrink_zones(). If for whatver reason, all_unreclaimable is
> >	set on all zones, we can miss calling cond_resched().
> >
> >shrink_slab only applies if we are reclaiming slab pages. If the first
> >	shrinker returns -1, we do not call cond_resched(). If that
> >	first shrinker is dcache and __GFP_FS is not set, direct
> >	reclaimers will not shrink at all. However, if there are
> >	enough of them running or if one of the other shrinkers
> >	is running for a very long time, kswapd could be starved
> >	acquiring the shrinker_rwsem and never reaching the
> >	cond_resched().
> 
> OK.
> 
> 
> >
> >balance_pgdat() only calls cond_resched if the zones are not
> >	balanced. For a high-order allocation that is balanced, it
> >	checks order-0 again. During that window, order-0 might have
> >	become unbalanced so it loops again for order-0 and returns
> >	that was reclaiming for order-0 to kswapd(). It can then find
> >	that a caller has rewoken kswapd for a high-order and re-enters
> >	balance_pgdat() without ever have called cond_resched().
> 
> Then, Shouldn't balance_pgdat() call cond_resched() unconditionally?
> The problem is NOT 100% cpu consumption. if kswapd will sleep, other
> processes need to reclaim old pages. The problem is, kswapd doesn't
> invoke context switch and other tasks hang-up.
> 

Which the shrink_slab patch does (either version). What's the gain from
sprinkling more cond_resched() around? If you think there is, submit
another pair of patches (include patch 1 from this series) but I'm not
seeing the advantage myself.

> 
> >While it appears unlikely, there are bad conditions which can result
> >in cond_resched() being avoided.
> >
> 
> 

-- 
Mel Gorman
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2011-05-18  9:57 UTC|newest]

Thread overview: 119+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-05-13 14:03 [PATCH 0/4] Reduce impact to overall system of SLUB using high-order allocations V2 Mel Gorman
2011-05-13 14:03 ` Mel Gorman
2011-05-13 14:03 ` [PATCH 1/4] mm: vmscan: Correct use of pgdat_balanced in sleeping_prematurely Mel Gorman
2011-05-13 14:03   ` Mel Gorman
2011-05-13 14:28   ` Johannes Weiner
2011-05-13 14:28     ` Johannes Weiner
2011-05-14 16:30   ` Minchan Kim
2011-05-14 16:30     ` Minchan Kim
2011-05-16 14:30   ` Rik van Riel
2011-05-16 14:30     ` Rik van Riel
2011-05-13 14:03 ` [PATCH 2/4] mm: slub: Do not wake kswapd for SLUBs speculative high-order allocations Mel Gorman
2011-05-13 14:03   ` Mel Gorman
2011-05-16 21:10   ` David Rientjes
2011-05-16 21:10     ` David Rientjes
2011-05-18  6:09     ` Pekka Enberg
2011-05-18  6:09       ` Pekka Enberg
2011-05-18 17:21       ` Christoph Lameter
2011-05-18 17:21         ` Christoph Lameter
2011-05-13 14:03 ` [PATCH 3/4] mm: slub: Do not take expensive steps " Mel Gorman
2011-05-13 14:03   ` Mel Gorman
2011-05-16 21:16   ` David Rientjes
2011-05-16 21:16     ` David Rientjes
2011-05-17  8:42     ` Mel Gorman
2011-05-17  8:42       ` Mel Gorman
2011-05-17 13:51       ` Christoph Lameter
2011-05-17 13:51         ` Christoph Lameter
2011-05-17 16:22         ` Mel Gorman
2011-05-17 16:22           ` Mel Gorman
2011-05-17 17:52           ` Christoph Lameter
2011-05-17 17:52             ` Christoph Lameter
2011-05-17 19:35             ` David Rientjes
2011-05-17 19:35               ` David Rientjes
2011-05-17 19:31       ` David Rientjes
2011-05-17 19:31         ` David Rientjes
2011-05-13 14:03 ` [PATCH 4/4] mm: vmscan: If kswapd has been running too long, allow it to sleep Mel Gorman
2011-05-13 14:03   ` Mel Gorman
2011-05-15 10:27   ` KOSAKI Motohiro
2011-05-15 10:27     ` KOSAKI Motohiro
2011-05-16  4:21     ` James Bottomley
2011-05-16  4:21       ` James Bottomley
2011-05-16  5:04       ` Minchan Kim
2011-05-16  5:04         ` Minchan Kim
2011-05-16  8:45         ` Mel Gorman
2011-05-16  8:45           ` Mel Gorman
2011-05-16  8:45           ` Mel Gorman
2011-05-16  8:58           ` Minchan Kim
2011-05-16  8:58             ` Minchan Kim
2011-05-16  8:58             ` Minchan Kim
2011-05-16 10:27             ` Mel Gorman
2011-05-16 10:27               ` Mel Gorman
2011-05-16 10:27               ` Mel Gorman
2011-05-16 23:50               ` Minchan Kim
2011-05-16 23:50                 ` Minchan Kim
2011-05-17  0:48                 ` Minchan Kim
2011-05-17  0:48                   ` Minchan Kim
2011-05-17  0:48                   ` Minchan Kim
2011-05-17 10:38                 ` Mel Gorman
2011-05-17 10:38                   ` Mel Gorman
2011-05-17 10:38                   ` Mel Gorman
2011-05-17 13:50                   ` Colin Ian King
2011-05-17 13:50                     ` Colin Ian King
2011-05-17 16:15                     ` [PATCH] mm: vmscan: Correctly check if reclaimer should schedule during shrink_slab Mel Gorman
2011-05-17 16:15                       ` Mel Gorman
2011-05-18  0:45                       ` KOSAKI Motohiro
2011-05-18  0:45                         ` KOSAKI Motohiro
2011-05-19  0:03                       ` Minchan Kim
2011-05-19  0:03                         ` Minchan Kim
2011-05-19  0:03                         ` Minchan Kim
2011-05-19  0:09                       ` Minchan Kim
2011-05-19  0:09                         ` Minchan Kim
2011-05-19  0:09                         ` Minchan Kim
2011-05-19 11:36                         ` Colin Ian King
2011-05-19 11:36                           ` Colin Ian King
2011-05-20  0:06                           ` Minchan Kim
2011-05-20  0:06                             ` Minchan Kim
2011-05-20  0:06                             ` Minchan Kim
2011-05-18  4:19                     ` [PATCH 4/4] mm: vmscan: If kswapd has been running too long, allow it to sleep Minchan Kim
2011-05-18  4:19                       ` Minchan Kim
2011-05-18  7:39                       ` Colin Ian King
2011-05-18  7:39                         ` Colin Ian King
2011-05-18  4:09                   ` James Bottomley
2011-05-18  4:09                     ` James Bottomley
2011-05-18  1:05                 ` KOSAKI Motohiro
2011-05-18  1:05                   ` KOSAKI Motohiro
2011-05-18  5:44                   ` Minchan Kim
2011-05-18  5:44                     ` Minchan Kim
2011-05-18  5:44                     ` Minchan Kim
2011-05-18  6:05                     ` KOSAKI Motohiro
2011-05-18  6:05                       ` KOSAKI Motohiro
2011-05-18  9:58                     ` Mel Gorman
2011-05-18  9:58                       ` Mel Gorman
2011-05-18  9:58                       ` Mel Gorman
2011-05-18 22:55                       ` Minchan Kim
2011-05-18 22:55                         ` Minchan Kim
2011-05-18 23:54                         ` KOSAKI Motohiro
2011-05-18 23:54                           ` KOSAKI Motohiro
2011-05-18  0:26               ` KOSAKI Motohiro
2011-05-18  0:26                 ` KOSAKI Motohiro
2011-05-18  9:57                 ` Mel Gorman [this message]
2011-05-18  9:57                   ` Mel Gorman
2011-05-16  8:45     ` Mel Gorman
2011-05-16  8:45       ` Mel Gorman
2011-05-16 14:30   ` Rik van Riel
2011-05-16 14:30     ` Rik van Riel
2011-05-13 15:19 ` [PATCH 0/4] Reduce impact to overall system of SLUB using high-order allocations V2 James Bottomley
2011-05-13 15:19   ` James Bottomley
2011-05-13 15:19   ` James Bottomley
2011-05-13 15:52   ` Mel Gorman
2011-05-13 15:52     ` Mel Gorman
2011-05-13 15:21 ` Christoph Lameter
2011-05-13 15:21   ` Christoph Lameter
2011-05-13 15:43   ` Mel Gorman
2011-05-13 15:43     ` Mel Gorman
2011-05-14  8:34 ` Colin Ian King
2011-05-14  8:34   ` Colin Ian King
2011-05-16  8:37   ` Mel Gorman
2011-05-16  8:37     ` Mel Gorman
2011-05-16 11:24     ` Colin Ian King
2011-05-16 11:24       ` Colin Ian King

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110518095720.GQ5279@suse.de \
    --to=mgorman@suse.de \
    --cc=James.Bottomley@hansenpartnership.com \
    --cc=akpm@linux-foundation.org \
    --cc=chris.mason@oracle.com \
    --cc=cl@linux.com \
    --cc=colin.king@canonical.com \
    --cc=hannes@cmpxchg.org \
    --cc=jack@suse.cz \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=minchan.kim@gmail.com \
    --cc=penberg@kernel.org \
    --cc=raghu.prabhu13@gmail.com \
    --cc=riel@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.