From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-25.2 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,MISSING_HEADERS, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT,USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17207C433C1 for ; Mon, 22 Mar 2021 22:00:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 893506199F for ; Mon, 22 Mar 2021 22:00:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 893506199F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AA8AE6B010D; Mon, 22 Mar 2021 18:00:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A332F6B010F; Mon, 22 Mar 2021 18:00:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AC076B0111; Mon, 22 Mar 2021 18:00:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0051.hostedemail.com [216.40.44.51]) by kanga.kvack.org (Postfix) with ESMTP id 6BE236B010D for ; Mon, 22 Mar 2021 18:00:15 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 24C608249980 for ; Mon, 22 Mar 2021 22:00:15 +0000 (UTC) X-FDA: 77948879148.01.A13C8F8 Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf02.hostedemail.com (Postfix) with ESMTP id EB0AF40B8CE5 for ; Mon, 22 Mar 2021 21:59:58 +0000 (UTC) Received: by mail-pj1-f74.google.com with SMTP id l17so240115pjt.2 for ; Mon, 22 Mar 2021 14:59:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:cc; bh=voOmLAB+ChC2A4QEJf/yJCzWFfGe3YHCm7ZcDcYjjnU=; b=cd0M2VHOYGzQl8ZXH/1Sz3MjSzG2cudqMvnHDFR7ZzJq4unb+n72PKnDuWzta6zl56 BT1G5HDQbp9Yo8Ym/E/Ed9bxayOeNfsP3OSuab7gWEBii+DL5NwY7Nkci6JOoXtnTHlV u+8jkHVP35f7FsExNWvgKUeF5xzdyzsQSZVvV/3wDVq/gUt4CajAGEJhK3ce9GVifPdN MdnIBUMdevzZaCIN0LcH0J4zJgPwDJpyhZoZkgg/cVJn2t/BBRzakpwcJvig9QNFaNH5 4s/YDGKFKO/zGvlM7wWOFowipybMEr6brimqeTXqGP2eBz1Eh7yocLh7VM17Z/DLSKww OH8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:cc; bh=voOmLAB+ChC2A4QEJf/yJCzWFfGe3YHCm7ZcDcYjjnU=; b=PhxSdpGQ2UlytGuZ37Jl7JosP60+yObFXRiLvipGWaPnN1jfXNozgX/CzfR4c9ybH9 bk++Bqewdz23fJ74FjcoQWgzcypBaVwWzhz5Leks1wH9fJ5NqfTt2M/uM7502rUNrK0N BSh9cblujinSRylREiuk1YNbcC94zzU7CZj8QC1tqww6nC/Qj6NbJLaEh/2/wBbXPTGO 3Wjwr5uJCPGyz05Rndylx7dOGDjKQ1+UAW67LD3ySQElY6GXJXSJqYwnMmr6Fv158SbZ iHV5pZ2dzuzBv9L/k9y2/4+sNf6LP+aPyQojSPTQR0RSA/455+Vq4xOwQEppb43flCmp Qucw== X-Gm-Message-State: AOAM531mjb8FGgA83N03XD1hzmkhWwPrxwfzWr5384Sid/OSUvugtePX dbTQ7IzypxplXIAMvMGL+CGSeIz2a9DKdmNfw24= X-Google-Smtp-Source: ABdhPJwD6jhMxNK5nm1rj1xbPlntW8CX8SX1XOzhve/8dPDbQRmmU11dpDAzjKSMpeq7I2jfAjx+SWaxs9+WVLq47Hk= X-Received: from cfijalkovich.mtv.corp.google.com ([2620:15c:211:202:3d09:6020:a2b1:f8fb]) (user=cfijalkovich job=sendgmr) by 2002:a17:90a:ce0c:: with SMTP id f12mr1126811pju.11.1616450394091; Mon, 22 Mar 2021 14:59:54 -0700 (PDT) Date: Mon, 22 Mar 2021 14:58:23 -0700 Message-Id: <20210322215823.962758-1-cfijalkovich@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.31.0.rc2.261.g7f71774620-goog Subject: [PATCH] mm, thp: Relax the VM_DENYWRITE constraint on file-backed THPs From: Collin Fijalkovich Cc: songliubraving@fb.com, surenb@google.com, hridya@google.com, kaleshsingh@google.com, hughd@google.com, timmurray@google.com, Collin Fijalkovich , Alexander Viro , Andrew Morton , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: EB0AF40B8CE5 X-Stat-Signature: 4tzkettonuwajiy4f3e3z9jrq14do7t1 Received-SPF: none (flex--cfijalkovich.bounces.google.com>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from="<3WhNZYAwKCO0RUXYPaZdkXRWVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--cfijalkovich.bounces.google.com>"; helo=mail-pj1-f74.google.com; client-ip=209.85.216.74 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1616450398-992197 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Transparent huge pages are supported for read-only non-shmem filesystems, but are only used for vmas with VM_DENYWRITE. This condition ensures that file THPs are protected from writes while an application is running (ETXTBSY). Any existing file THPs are then dropped from the page cache when a file is opened for write in do_dentry_open(). Since sys_mmap ignores MAP_DENYWRITE, this constrains the use of file THPs to vmas produced by execve(). Systems that make heavy use of shared libraries (e.g. Android) are unable to apply VM_DENYWRITE through the dynamic linker, preventing them from benefiting from the resultant reduced contention on the TLB. This patch reduces the constraint on file THPs allowing use with any executable mapping from a file not opened for write (see inode_is_open_for_write()). It also introduces additional conditions to ensure that files opened for write will never be backed by file THPs. Restricting the use of THPs to executable mappings eliminates the risk that a read-only file later opened for write would encounter significant latencies due to page cache truncation. The ld linker flag '-z max-page-size=(hugepage size)' can be used to produce executables with the necessary layout. The dynamic linker must map these file's segments at a hugepage size aligned vma for the mapping to be backed with THPs. Signed-off-by: Collin Fijalkovich --- fs/open.c | 13 +++++++++++-- mm/khugepaged.c | 16 +++++++++++++++- 2 files changed, 26 insertions(+), 3 deletions(-) diff --git a/fs/open.c b/fs/open.c index e53af13b5835..f76e960d10ea 100644 --- a/fs/open.c +++ b/fs/open.c @@ -852,8 +852,17 @@ static int do_dentry_open(struct file *f, * XXX: Huge page cache doesn't support writing yet. Drop all page * cache for this file before processing writes. */ - if ((f->f_mode & FMODE_WRITE) && filemap_nr_thps(inode->i_mapping)) - truncate_pagecache(inode, 0); + if (f->f_mode & FMODE_WRITE) { + /* + * Paired with smp_mb() in collapse_file() to ensure nr_thps + * is up to date and the update to i_writecount by + * get_write_access() is visible. Ensures subsequent insertion + * of THPs into the page cache will fail. + */ + smp_mb(); + if (filemap_nr_thps(inode->i_mapping)) + truncate_pagecache(inode, 0); + } return 0; diff --git a/mm/khugepaged.c b/mm/khugepaged.c index a7d6cb912b05..4c7cc877d5e3 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -459,7 +459,8 @@ static bool hugepage_vma_check(struct vm_area_struct *vma, /* Read-only file mappings need to be aligned for THP to work. */ if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && vma->vm_file && - (vm_flags & VM_DENYWRITE)) { + !inode_is_open_for_write(vma->vm_file->f_inode) && + (vm_flags & VM_EXEC)) { return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff, HPAGE_PMD_NR); } @@ -1872,6 +1873,19 @@ static void collapse_file(struct mm_struct *mm, else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, nr); filemap_nr_thps_inc(mapping); + /* + * Paired with smp_mb() in do_dentry_open() to ensure + * i_writecount is up to date and the update to nr_thps is + * visible. Ensures the page cache will be truncated if the + * file is opened writable. + */ + smp_mb(); + if (inode_is_open_for_write(mapping->host)) { + result = SCAN_FAIL; + __mod_lruvec_page_state(new_page, NR_FILE_THPS, -nr); + filemap_nr_thps_dec(mapping); + goto xa_locked; + } } if (nr_none) { -- 2.31.0.rc2.261.g7f71774620-goog