From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753187Ab3LJMiu (ORCPT ); Tue, 10 Dec 2013 07:38:50 -0500 Received: from mail-pb0-f47.google.com ([209.85.160.47]:62830 "EHLO mail-pb0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753112Ab3LJMir (ORCPT ); Tue, 10 Dec 2013 07:38:47 -0500 MIME-Version: 1.0 In-Reply-To: <52A6FFF0.6080207@parallels.com> References: <5287164773f8aade33ce17f3c91546c6e1afaf85.1386571280.git.vdavydov@parallels.com> <20131210041826.GB31386@dastard> <52A6FFF0.6080207@parallels.com> Date: Tue, 10 Dec 2013 16:38:46 +0400 Message-ID: Subject: Re: [PATCH v13 13/16] vmscan: take at least one pass with shrinkers From: Glauber Costa To: Vladimir Davydov Cc: Dave Chinner , dchinner@redhat.com, Johannes Weiner , Michal Hocko , Andrew Morton , LKML , linux-mm@kvack.org, cgroups@vger.kernel.org, devel@openvz.org, Glauber Costa , Glauber Costa , Mel Gorman , Rik van Riel Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Dec 10, 2013 at 3:50 PM, Vladimir Davydov wrote: > On 12/10/2013 08:18 AM, Dave Chinner wrote: >> On Mon, Dec 09, 2013 at 12:05:54PM +0400, Vladimir Davydov wrote: >>> From: Glauber Costa >>> >>> In very low free kernel memory situations, it may be the case that we >>> have less objects to free than our initial batch size. If this is the >>> case, it is better to shrink those, and open space for the new workload >>> then to keep them and fail the new allocations. >>> >>> In particular, we are concerned with the direct reclaim case for memcg. >>> Although this same technique can be applied to other situations just as >>> well, we will start conservative and apply it for that case, which is >>> the one that matters the most. >> This should be at the start of the series. > > Since Glauber wanted to introduce this only for memcg-reclaim first, > this can't be at the start of the series, but I'll move it to go > immediately after per-memcg shrinking core in the next iteration. > > Thanks. So, the reason for that being memcg only, is that the reclaim for small objects triggered a bunch of XFS regressions (I am sure the regressions are general, but I've tested them using ZFS). In theory they shouldn't, so we can try to make it global again, so long as it comes together with benchmarks demonstrating that it is a safe change. I am not sure the filesystem people would benefit from that directly, though. So it may not be worth the hassle... From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-yh0-f46.google.com (mail-yh0-f46.google.com [209.85.213.46]) by kanga.kvack.org (Postfix) with ESMTP id D5D186B0035 for ; Tue, 10 Dec 2013 07:38:48 -0500 (EST) Received: by mail-yh0-f46.google.com with SMTP id l109so3724804yhq.5 for ; Tue, 10 Dec 2013 04:38:48 -0800 (PST) Received: from mail-pb0-x22d.google.com (mail-pb0-x22d.google.com [2607:f8b0:400e:c01::22d]) by mx.google.com with ESMTPS id t39si8624514yhp.225.2013.12.10.04.38.47 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 10 Dec 2013 04:38:47 -0800 (PST) Received: by mail-pb0-f45.google.com with SMTP id rp16so7542545pbb.4 for ; Tue, 10 Dec 2013 04:38:46 -0800 (PST) MIME-Version: 1.0 In-Reply-To: <52A6FFF0.6080207@parallels.com> References: <5287164773f8aade33ce17f3c91546c6e1afaf85.1386571280.git.vdavydov@parallels.com> <20131210041826.GB31386@dastard> <52A6FFF0.6080207@parallels.com> Date: Tue, 10 Dec 2013 16:38:46 +0400 Message-ID: Subject: Re: [PATCH v13 13/16] vmscan: take at least one pass with shrinkers From: Glauber Costa Content-Type: text/plain; charset=UTF-8 Sender: owner-linux-mm@kvack.org List-ID: To: Vladimir Davydov Cc: Dave Chinner , dchinner@redhat.com, Johannes Weiner , Michal Hocko , Andrew Morton , LKML , linux-mm@kvack.org, cgroups@vger.kernel.org, devel@openvz.org, Glauber Costa , Glauber Costa , Mel Gorman , Rik van Riel On Tue, Dec 10, 2013 at 3:50 PM, Vladimir Davydov wrote: > On 12/10/2013 08:18 AM, Dave Chinner wrote: >> On Mon, Dec 09, 2013 at 12:05:54PM +0400, Vladimir Davydov wrote: >>> From: Glauber Costa >>> >>> In very low free kernel memory situations, it may be the case that we >>> have less objects to free than our initial batch size. If this is the >>> case, it is better to shrink those, and open space for the new workload >>> then to keep them and fail the new allocations. >>> >>> In particular, we are concerned with the direct reclaim case for memcg. >>> Although this same technique can be applied to other situations just as >>> well, we will start conservative and apply it for that case, which is >>> the one that matters the most. >> This should be at the start of the series. > > Since Glauber wanted to introduce this only for memcg-reclaim first, > this can't be at the start of the series, but I'll move it to go > immediately after per-memcg shrinking core in the next iteration. > > Thanks. So, the reason for that being memcg only, is that the reclaim for small objects triggered a bunch of XFS regressions (I am sure the regressions are general, but I've tested them using ZFS). In theory they shouldn't, so we can try to make it global again, so long as it comes together with benchmarks demonstrating that it is a safe change. I am not sure the filesystem people would benefit from that directly, though. So it may not be worth the hassle... -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Glauber Costa Subject: Re: [PATCH v13 13/16] vmscan: take at least one pass with shrinkers Date: Tue, 10 Dec 2013 16:38:46 +0400 Message-ID: References: <5287164773f8aade33ce17f3c91546c6e1afaf85.1386571280.git.vdavydov@parallels.com> <20131210041826.GB31386@dastard> <52A6FFF0.6080207@parallels.com> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lkrXguvgbWBXFR9tsRNVYqDxVzKto2NiUuy2WLTQmbA=; b=qNpQg9Dn4rnke81TiMBg7hFBGF7nDclCNRZNk+c3TtMWjGcKeFDalUa2YuR+OT20pD WySdrAuktnOQOcKvOoG3uxE5+pU68Pa74uMFPDbCLSfQf/vRaPr0yOYLmU4P48OZovKQ IN+T52oi2uok3b1PP6ncASqo5wHAeVlpw0Gx3/BMbweXLoIBs4o6vUlW+Y+C1Snem2kV ee9uSPON5nZQo+hoTvfFJNU3tHRv9wbfUXnuOq3GDVuIXyTrIYqJUw1sysgEI/Zetfc0 yuAIEBARZAFVi6/+6BYy/EatDS1nhxYgH7tiGnBf6DH2ZRUgOvjh5ruUcsGToc38xsUj WLXw== In-Reply-To: <52A6FFF0.6080207-bzQdu9zFT3WakBO8gow8eQ@public.gmane.org> Sender: cgroups-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Vladimir Davydov Cc: Dave Chinner , dchinner-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, Johannes Weiner , Michal Hocko , Andrew Morton , LKML , linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, devel-GEFAQzZX7r8dnm+yROfE0A@public.gmane.org, Glauber Costa , Glauber Costa , Mel Gorman , Rik van Riel On Tue, Dec 10, 2013 at 3:50 PM, Vladimir Davydov wrote: > On 12/10/2013 08:18 AM, Dave Chinner wrote: >> On Mon, Dec 09, 2013 at 12:05:54PM +0400, Vladimir Davydov wrote: >>> From: Glauber Costa >>> >>> In very low free kernel memory situations, it may be the case that we >>> have less objects to free than our initial batch size. If this is the >>> case, it is better to shrink those, and open space for the new workload >>> then to keep them and fail the new allocations. >>> >>> In particular, we are concerned with the direct reclaim case for memcg. >>> Although this same technique can be applied to other situations just as >>> well, we will start conservative and apply it for that case, which is >>> the one that matters the most. >> This should be at the start of the series. > > Since Glauber wanted to introduce this only for memcg-reclaim first, > this can't be at the start of the series, but I'll move it to go > immediately after per-memcg shrinking core in the next iteration. > > Thanks. So, the reason for that being memcg only, is that the reclaim for small objects triggered a bunch of XFS regressions (I am sure the regressions are general, but I've tested them using ZFS). In theory they shouldn't, so we can try to make it global again, so long as it comes together with benchmarks demonstrating that it is a safe change. I am not sure the filesystem people would benefit from that directly, though. So it may not be worth the hassle...