From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-23.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39947C433ED for ; Tue, 6 Apr 2021 17:48:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B12F9613CD for ; Tue, 6 Apr 2021 17:48:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B12F9613CD Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 24BE86B007E; Tue, 6 Apr 2021 13:48:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 223046B0080; Tue, 6 Apr 2021 13:48:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0C3EB6B0081; Tue, 6 Apr 2021 13:48:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id E41C06B007E for ; Tue, 6 Apr 2021 13:48:31 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9C6C7180ACF8F for ; Tue, 6 Apr 2021 17:48:31 +0000 (UTC) X-FDA: 78002676822.24.E32DBD0 Received: from mail-lf1-f42.google.com (mail-lf1-f42.google.com [209.85.167.42]) by imf24.hostedemail.com (Postfix) with ESMTP id E0660A000381 for ; Tue, 6 Apr 2021 17:48:27 +0000 (UTC) Received: by mail-lf1-f42.google.com with SMTP id d12so24077282lfv.11 for ; Tue, 06 Apr 2021 10:48:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=yOoUOFLQlh9034N4v2oxQm9jcLLVDu90iggICUK0ZwU=; b=Xj4N0+f8XD0nqzF0OAFU2U4HQQ/DjDUY32w6YY++lalFjLWRxLVEabRedpQUGzrwVz qr6s8T6wNbCAnllM9mSMDgF/j5vJ/z3Mt+U7cumXlq5uFuOTmSNVrJayVfpSunM0MdsM MMkQ4zfe7ZXBIyCKak30FfQSnxcti49a0sjbK0fRozzTFgQ/wWA3t/FXfxGMu1HiPuv9 GrCO/COPrBeNLy67WEFaYAT3S/J6xW4BxND74LUJ1s0tqgPHWzDGW+Bgb/utlu2iGsiE wCiIprAwpb05wtwxed/3JLdUNbkLHR/66fUc8+kqC5JRyteVZ+TMO7bKCnn1Y+4xkc92 z1aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=yOoUOFLQlh9034N4v2oxQm9jcLLVDu90iggICUK0ZwU=; b=L7Pb6WFP1qzZBXV8aRDa5+ulfpc4lmnFrUXwMM4VednRp80Tvt17gVxqi2j6ZTwczK mhwGcnl41nZjJxRxFdWRmcASERidG232XLvoA0Q2X/FX6t3NL/peH+d/P3j2u4f9MlGm udpkPzq95gZqdzqT7EcvNzhUN4jHaX7psaIPQvbXmhTZzKCh329Q9uJYWBOSXYThY+na XtRMfIVEv9fw3ZZ2ySeE87q9S8OU92IpC4f7B8d5LjWug7tbOf/HisdfqhmNTulYjcZA IIhU139Hcq4GvpQI7FN4zMH8MtlShJZFZ/4yMlk3rux2b65iDJDLpMhntB/dQ+gJbTNN ezww== X-Gm-Message-State: AOAM5303b/miCFLNnz1IeqG3ATct2KDdIbkr3yBK+knbebbtMNhZnYG1 WvFZ9AEL+AaSxb/eQ+fe+TvnO4I3aX8WEziHTKhcJA== X-Google-Smtp-Source: ABdhPJxDqtgpQyqgh72HvpE69l5zhTy4yTOsu9t6se5EOU92Iep9NLmH2O3TqOSCrYWSwyyZdbgKQKDddR6CrshbfB8= X-Received: by 2002:a05:6512:11c2:: with SMTP id h2mr21959123lfr.94.1617731309356; Tue, 06 Apr 2021 10:48:29 -0700 (PDT) MIME-Version: 1.0 References: <20210322215823.962758-1-cfijalkovich@google.com> <7CC369CF-66F6-4362-BCA8-0C1CAE350CDF@oracle.com> In-Reply-To: <7CC369CF-66F6-4362-BCA8-0C1CAE350CDF@oracle.com> From: Collin Fijalkovich Date: Tue, 6 Apr 2021 10:48:18 -0700 Message-ID: Subject: Re: [PATCH] mm, thp: Relax the VM_DENYWRITE constraint on file-backed THPs To: William Kucharski Cc: Song Liu , Suren Baghdasaryan , Hridya Valsaraju , Kalesh Singh , Hugh Dickins , Tim Murray , Alexander Viro , Andrew Morton , Linux-Fsdevel , open list , Linux-MM Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E0660A000381 X-Stat-Signature: jwhxjb9ywdtsc1fycebofc66tz364pxt Received-SPF: none (google.com>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail-lf1-f42.google.com; client-ip=209.85.167.42 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617731307-342198 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instrumenting filemap_nr_thps_inc() should be sufficient for ensuring writable file mappings will not be THP-backed. If filemap_nr_thps_dec() in unaccount_page_cache_page() and filemap_nr_thps() in do_dentry_open() race, the worst case is an unnecessary truncation. We could introduce a memory barrier in unaccount_page_cache_page(), but I'm unsure it would significantly mitigate the risk of spurious truncations. Barring synchronization between do_dentry_open() and ongoing page cache operations, there could be an in-progress delete_from_page_cache_batch() that has not yet updated accounting for THPs in its targeted range. -- Collin On Mon, Apr 5, 2021 at 7:05 PM William Kucharski wrote: > > > I saw a similar change a few years ago with my prototype: > > https://lore.kernel.org/linux-mm/5BB682E1-DD52-4AA9-83E9-DEF091E0C709@oracle.com/ > > the key being a very nice drop in iTLB-load-misses, so it looks like your code is > having the right effect. > > What about the call to filemap_nr_thps_dec() in unaccount_page_cache_page() - does > that need an smp_mb() as well? > > -- Bill > > > On Apr 5, 2021, at 6:15 PM, Collin Fijalkovich wrote: > > > > v2 has been uploaded with performance testing results: > > https://lore.kernel.org/patchwork/patch/1408266/ > > > > > > > > On Mon, Mar 22, 2021 at 2:59 PM Collin Fijalkovich > > wrote: > >> > >> Transparent huge pages are supported for read-only non-shmem filesystems, > >> but are only used for vmas with VM_DENYWRITE. This condition ensures that > >> file THPs are protected from writes while an application is running > >> (ETXTBSY). Any existing file THPs are then dropped from the page cache > >> when a file is opened for write in do_dentry_open(). Since sys_mmap > >> ignores MAP_DENYWRITE, this constrains the use of file THPs to vmas > >> produced by execve(). > >> > >> Systems that make heavy use of shared libraries (e.g. Android) are unable > >> to apply VM_DENYWRITE through the dynamic linker, preventing them from > >> benefiting from the resultant reduced contention on the TLB. > >> > >> This patch reduces the constraint on file THPs allowing use with any > >> executable mapping from a file not opened for write (see > >> inode_is_open_for_write()). It also introduces additional conditions to > >> ensure that files opened for write will never be backed by file THPs. > >> > >> Restricting the use of THPs to executable mappings eliminates the risk that > >> a read-only file later opened for write would encounter significant > >> latencies due to page cache truncation. > >> > >> The ld linker flag '-z max-page-size=(hugepage size)' can be used to > >> produce executables with the necessary layout. The dynamic linker must > >> map these file's segments at a hugepage size aligned vma for the mapping to > >> be backed with THPs. > >> > >> Signed-off-by: Collin Fijalkovich > >> --- > >> fs/open.c | 13 +++++++++++-- > >> mm/khugepaged.c | 16 +++++++++++++++- > >> 2 files changed, 26 insertions(+), 3 deletions(-) > >> > >> diff --git a/fs/open.c b/fs/open.c > >> index e53af13b5835..f76e960d10ea 100644 > >> --- a/fs/open.c > >> +++ b/fs/open.c > >> @@ -852,8 +852,17 @@ static int do_dentry_open(struct file *f, > >> * XXX: Huge page cache doesn't support writing yet. Drop all page > >> * cache for this file before processing writes. > >> */ > >> - if ((f->f_mode & FMODE_WRITE) && filemap_nr_thps(inode->i_mapping)) > >> - truncate_pagecache(inode, 0); > >> + if (f->f_mode & FMODE_WRITE) { > >> + /* > >> + * Paired with smp_mb() in collapse_file() to ensure nr_thps > >> + * is up to date and the update to i_writecount by > >> + * get_write_access() is visible. Ensures subsequent insertion > >> + * of THPs into the page cache will fail. > >> + */ > >> + smp_mb(); > >> + if (filemap_nr_thps(inode->i_mapping)) > >> + truncate_pagecache(inode, 0); > >> + } > >> > >> return 0; > >> > >> diff --git a/mm/khugepaged.c b/mm/khugepaged.c > >> index a7d6cb912b05..4c7cc877d5e3 100644 > >> --- a/mm/khugepaged.c > >> +++ b/mm/khugepaged.c > >> @@ -459,7 +459,8 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, > >> > >> /* Read-only file mappings need to be aligned for THP to work. */ > >> if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && vma->vm_file && > >> - (vm_flags & VM_DENYWRITE)) { > >> + !inode_is_open_for_write(vma->vm_file->f_inode) && > >> + (vm_flags & VM_EXEC)) { > >> return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, > >> HPAGE_PMD_NR); > >> } > >> @@ -1872,6 +1873,19 @@ static void collapse_file(struct mm_struct *mm, > >> else { > >> __mod_lruvec_page_state(new_page, NR_FILE_THPS, nr); > >> filemap_nr_thps_inc(mapping); > >> + /* > >> + * Paired with smp_mb() in do_dentry_open() to ensure > >> + * i_writecount is up to date and the update to nr_thps is > >> + * visible. Ensures the page cache will be truncated if the > >> + * file is opened writable. > >> + */ > >> + smp_mb(); > >> + if (inode_is_open_for_write(mapping->host)) { > >> + result = SCAN_FAIL; > >> + __mod_lruvec_page_state(new_page, NR_FILE_THPS, -nr); > >> + filemap_nr_thps_dec(mapping); > >> + goto xa_locked; > >> + } > >> } > >> > >> if (nr_none) { > >> -- > >> 2.31.0.rc2.261.g7f71774620-goog > >> > > >