From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=3.0 tests=DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS,T_DKIM_INVALID, USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5385EC476E8 for ; Thu, 12 Jul 2018 16:49:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0D6042075C for ; Thu, 12 Jul 2018 16:49:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="WIjEoRWj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D6042075C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732413AbeGLQ74 (ORCPT ); Thu, 12 Jul 2018 12:59:56 -0400 Received: from bombadil.infradead.org ([198.137.202.133]:40864 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726925AbeGLQ74 (ORCPT ); Thu, 12 Jul 2018 12:59:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=In-Reply-To:Content-Type:MIME-Version :References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=N7wd4eUrfJZuG+lYRTZKZ1CMnvviF+b+4hdGweq4R2I=; b=WIjEoRWjyIn/yACKSe38FuwUJ iksD4Uk13H1cy9XJzQKI9cmaa9Nyewr7kZ8b7RkOBMz+lr0fKGRUN9bOV3HAeoIREL9xOm1rR2m1T 2WLoXwWsMJ49pEUhSqu1JLszYKnqPLiI72EB2n6tUQ+ShqcG6ETIBfs+wKIO7QZDuTOSKEA3G7dmP Emc1cR53PXs2MAfwIq+hwiW0QnU+4zxLwQM162EcLPzm0U36hHJzGzGbAUSuZDsUiqZKYSpTIfzPV zf+i0C7htlfQN3zJMfxbdBygyijkCqqLO4CuNZg6I6Rr6fSu5GG4FNelAPTZWSXPwZ+44q+wTzYg+ F8lsM74GA==; Received: from willy by bombadil.infradead.org with local (Exim 4.90_1 #2 (Red Hat Linux)) id 1fdema-0006cB-KQ; Thu, 12 Jul 2018 16:49:32 +0000 Date: Thu, 12 Jul 2018 09:49:32 -0700 From: Matthew Wilcox To: James Bottomley Cc: Waiman Long , Michal Hocko , Alexander Viro , Jonathan Corbet , "Luis R. Rodriguez" , Kees Cook , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-doc@vger.kernel.org, Linus Torvalds , Jan Kara , "Paul E. McKenney" , Andrew Morton , Ingo Molnar , Miklos Szeredi , Larry Woodman , "Wangkai (Kevin C)" Subject: Re: [PATCH v6 0/7] fs/dcache: Track & limit # of negative dentries Message-ID: <20180712164932.GA3475@bombadil.infradead.org> References: <62275711-e01d-7dbe-06f1-bf094b618195@redhat.com> <20180710142740.GQ14284@dhcp22.suse.cz> <20180711102139.GG20050@dhcp22.suse.cz> <9f24c043-1fca-ee86-d609-873a7a8f7a64@redhat.com> <1531330947.3260.13.camel@HansenPartnership.com> <18c5cbfe-403b-bb2b-1d11-19d324ec6234@redhat.com> <1531336913.3260.18.camel@HansenPartnership.com> <4d49a270-23c9-529f-f544-65508b6b53cc@redhat.com> <1531411494.18255.6.camel@HansenPartnership.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1531411494.18255.6.camel@HansenPartnership.com> User-Agent: Mutt/1.9.2 (2017-12-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jul 12, 2018 at 09:04:54AM -0700, James Bottomley wrote: > On Thu, 2018-07-12 at 11:54 -0400, Waiman Long wrote: > > It is not that dentry cache is harder to get rid of than the other > > memory. It is that the ability of generate unlimited number of > > negative dentries that will displace other useful memory from the > > system. What the patch is trying to do is to have a warning or > > notification system in place to spot unusual activities in regard to > > the number of negative dentries in the system. The system > > administrators can then decide on what to do next. > > But every cache has this property: I can cause the same effect by doing > a streaming read on a multi gigabyte file: the page cache will fill > with the clean pages belonging to the file until I run out of memory > and it has to start evicting older cache entries. Once we hit the > steady state of minimal free memory, the mm subsytem tries to balance > the cache requests (like my streaming read) against the existing pool > of cached objects. > > The question I'm trying to get an answer to is why does the dentry > cache need special limits when the mm handling of the page cache (and > other mm caches) just works? I don't know that it does work. Or that it works well. When we try to allocate something and there's no memory readily available, we ask all the shrinkers to shrink in order to free up memory. That leads to one kind of allocation (eg dentries) being able to easily kick all the page cache out of the machine. What we could do instead is first call the shrinker for the type of object being allocated. That is, assume the system is more or less in equilibrium between all the different things it could be allocating, and if something needs to be kicked out, it's better to kick out this kind of thing rather than changing the equilibrium. Of course, workloads change over time, and sometimes we should accept we need more dentries and less page cache, or vice versa. So we'd need a scheme for the shrinker to say "this is getting hard, I think we do need more dentries", and then we'd move on to calling the other shrinkers to reclaim inodes or page cache or whatever. I don't think we even try to work at this level today. But it would have the distinct advantage that we can implement this in the slab/slub code rather than touching the page allocator.