From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.5 required=3.0 tests=MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42B8FC433FF for ; Mon, 12 Aug 2019 08:24:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0EB4B2085A for ; Mon, 12 Aug 2019 08:24:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0EB4B2085A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9DBE86B0003; Mon, 12 Aug 2019 04:24:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9659D6B0005; Mon, 12 Aug 2019 04:24:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 853376B0006; Mon, 12 Aug 2019 04:24:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0147.hostedemail.com [216.40.44.147]) by kanga.kvack.org (Postfix) with ESMTP id 5D0096B0003 for ; Mon, 12 Aug 2019 04:24:16 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with SMTP id 00771180AD7C3 for ; Mon, 12 Aug 2019 08:24:15 +0000 (UTC) X-FDA: 75813088512.13.crate79_2535a52411752 X-HE-Tag: crate79_2535a52411752 X-Filterd-Recvd-Size: 3679 Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Mon, 12 Aug 2019 08:24:15 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 1D034AE04; Mon, 12 Aug 2019 08:24:14 +0000 (UTC) Date: Mon, 12 Aug 2019 10:24:11 +0200 From: Michal Hocko To: ndrw Cc: Johannes Weiner , Suren Baghdasaryan , Vlastimil Babka , "Artem S. Tashkinov" , Andrew Morton , LKML , linux-mm Subject: Re: Let's talk about the elephant in the room - the Linux kernel's inability to gracefully handle low memory pressure Message-ID: <20190812082411.GB5117@dhcp22.suse.cz> References: <20190808114826.GC18351@dhcp22.suse.cz> <806F5696-A8D6-481D-A82F-49DEC1F2B035@redhazel.co.uk> <20190808163228.GE18351@dhcp22.suse.cz> <5FBB0A26-0CFE-4B88-A4F2-6A42E3377EDB@redhazel.co.uk> <20190808185925.GH18351@dhcp22.suse.cz> <08e5d007-a41a-e322-5631-b89978b9cc20@redhazel.co.uk> <20190809085748.GN18351@dhcp22.suse.cz> <20190809105016.GP18351@dhcp22.suse.cz> <33407eca-3c05-5900-0353-761db62feeea@redhazel.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <33407eca-3c05-5900-0353-761db62feeea@redhazel.co.uk> User-Agent: Mutt/1.10.1 (2018-07-13) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat 10-08-19 13:34:06, ndrw wrote: > On 09/08/2019 11:50, Michal Hocko wrote: > > We try to protect low amount of cache. Have a look at get_scan_count > > function. But the exact amount of the cache to be protected is really > > hard to know wihtout a crystal ball or understanding of the workload. > > The kernel doesn't have neither of the two. > > Thank you. I'm familiarizing myself with the code. Is there anyone I could > discuss some details with? I don't want to create too much noise here. linux-mm mailing list sounds like a good fit. > For example, are file pages created by mmaping files and are anon page > exclusively allocated on heap (RW data)? If so, where do "streaming IO" > pages belong to? Page cache will be generated by both buffered IO (read/write) and file mmaps. Anonymous memory by MAP_PRIVATE of file backed or MAP_ANON. Streaming IO is generally referred to by an single data pass IO that is not reused later (e.g. a backup). > > We have been thinking about this problem for a long time and couldn't > > come up with anything much better than we have now. PSI is the most recent > > improvement in that area. If you have better ideas then patches are > > always welcome. > > In general, I found there are very few user accessible knobs for adjusting > caching, especially in the pre-OOM phase. On the other hand, swapping, dirty > page caching, have many options or can even be disabled completely. > > For example, I would like to try disabling/limiting eviction of some/all > file pages (for example exec pages) akin to disabling swapping, but there is > no such mechanism. Yes, there would likely be problems with large RO mmapped > files that would need to be addressed, but in many applications users would > be interested in having such options. > > Adjusting how aggressive/conservative the system should be with the OOM > killer also falls into this category. What would that mean and how it would be configured? -- Michal Hocko SUSE Labs