From: Hugh Dickins <firstname.lastname@example.org>
To: "Kirill A. Shutemov" <email@example.com>
Cc: Hugh Dickins <firstname.lastname@example.org>,
Andrew Morton <email@example.com>,
Shakeel Butt <firstname.lastname@example.org>,
"Kirill A. Shutemov" <email@example.com>,
Yang Shi <firstname.lastname@example.org>,
Miaohe Lin <email@example.com>,
Mike Kravetz <firstname.lastname@example.org>,
Michal Hocko <email@example.com>, Rik van Riel <firstname.lastname@example.org>,
Christoph Hellwig <email@example.com>,
Matthew Wilcox <firstname.lastname@example.org>,
"Eric W. Biederman" <email@example.com>,
Alexey Gladkov <firstname.lastname@example.org>,
Chris Wilson <email@example.com>,
Matthew Auld <firstname.lastname@example.org>,
Subject: Re: [PATCH 07/16] memfd: memfd_create(name, MFD_HUGEPAGE) for shmem huge pages
Date: Thu, 5 Aug 2021 20:33:21 -0700 (PDT) [thread overview]
Message-ID: <email@example.com> (raw)
On Wed, 4 Aug 2021, Kirill A. Shutemov wrote:
> On Fri, Jul 30, 2021 at 12:45:49AM -0700, Hugh Dickins wrote:
> > Commit 749df87bd7be ("mm/shmem: add hugetlbfs support to memfd_create()")
> > in 4.14 added the MFD_HUGETLB flag to memfd_create(), to use hugetlbfs
> > pages instead of tmpfs pages: now add the MFD_HUGEPAGE flag, to use tmpfs
> > Transparent Huge Pages when they can be allocated (flag named to follow
> > the precedent of madvise's MADV_HUGEPAGE for THPs).
> I don't like the interface. THP supposed to be transparent, not yet another
THP is transparent in the sense that it builds hugepages from the
normal page pool, when it can (or not when it cannot), rather than
promising hugepages from a separate pre-reserved hugetlbfs pool.
Not transparent in the sense that it cannot be limited or guided.
> > /sys/kernel/mm/transparent_hugepage/shmem_enabled "always" or "force"
> > already made this possible: but that is much too blunt an instrument,
> > affecting all the very different kinds of files on the internal shmem
> > mount, and was intended just for ease of testing hugepage loads.
> I wounder if your tried "always" in production? What breaks? Maybe we can
> make it work with a heuristic? This would speed up adoption.
We have not tried /sys/kernel/mm/transparent_hugepage/shmem_enabled
"always" in production. Is that an experiment I want to recommend for
production? No, I don't think so! Why should we?
I am not looking to "speed up adoption" of huge tmpfs everywhere:
let those who find it useful use it, there is no need for it to be
We have had this disagreement before: you were aiming for tmpfs on /tmp
huge=always, I didn't see the need for that; but we have always agreed
that it should not be broken there, and the better it works the better -
you did the unused_huge_shrink stuff in particular to meet such cases.
> If a tunable needed, I would rather go with fadvise(). It would operate on
> a couple of bits per struct file and they get translated into VM_HUGEPAGE
> and VM_NOHUGEPAGE on mmap().
> Later if needed fadvise() implementation may be extended to track
> requested ranges. But initially it can be simple.
Let me shift that to the 08/16 (fcntl) response, and here answer:
> Hm, But why is the MFD_* needed if the fcntl() can do the same.
You're right, MFD_HUGEPAGE (and MFD_MEM_LOCK) are not strictly
needed if there's an fcntl() or fadvise() which can do that too.
But MFD_HUGEPAGE is the option which was first asked for, and is
the most popular usage internally - I did the fcntl at the same time,
and it has been found useful, but MFD_HUGEPAGE was the priority
(largely because fiddling with shmem_enabled interferes with
everyone's different usages, whereas huge=always on a mount
can be deployed selectively).
And it makes good sense for memfd_create() to offer MFD_HUGEPAGE,
as it is already offering MFD_HUGETLB: when we document MFD_HUGEPAGE
next to MFD_HUGETLB in the memfd_create(2) man page, that will help
developers to make a good choice.
(You said MFD_*, so I take it that you're thinking of MFD_MEM_LOCK
too: MFD_MEM_LOCK is something I added when building this series,
when I realized that it became possible once size change permitted.
Nobody here is using it yet, I don't mind if it's dropped; but it's
natural to propose it as part of the series, and it can be justified
as offering the memlock option which MFD_HUGETLB already bundles in.)
next prev parent reply other threads:[~2021-08-06 3:34 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-30 7:22 [PATCH 00/16] tmpfs: HUGEPAGE and MEM_LOCK fcntls and memfds Hugh Dickins
2021-07-30 7:25 ` [PATCH 01/16] huge tmpfs: fix fallocate(vanilla) advance over huge pages Hugh Dickins
2021-07-30 21:36 ` Yang Shi
2021-08-01 3:38 ` Hugh Dickins
2021-08-02 20:36 ` Yang Shi
2021-07-30 7:28 ` [PATCH 02/16] huge tmpfs: fix split_huge_page() after FALLOC_FL_KEEP_SIZE Hugh Dickins
2021-07-30 23:48 ` Yang Shi
2021-07-30 7:30 ` [PATCH 03/16] huge tmpfs: remove shrinklist addition from shmem_setattr() Hugh Dickins
2021-07-30 21:50 ` Yang Shi
2021-07-30 7:36 ` [PATCH 04/16] huge tmpfs: revert shmem's use of transhuge_vma_enabled() Hugh Dickins
2021-07-30 21:56 ` Yang Shi
2021-08-01 4:01 ` Hugh Dickins
2021-08-02 20:39 ` Yang Shi
2021-07-30 7:39 ` [PATCH 05/16] huge tmpfs: move shmem_huge_enabled() upwards Hugh Dickins
2021-07-30 21:57 ` Yang Shi
2021-07-30 7:42 ` [PATCH 06/16] huge tmpfs: shmem_is_huge(vma, inode, index) Hugh Dickins
2021-07-30 23:34 ` Yang Shi
2021-08-01 5:22 ` Hugh Dickins
2021-08-01 5:37 ` Hugh Dickins
2021-08-02 21:14 ` Yang Shi
2021-08-04 8:28 ` Hugh Dickins
2021-08-04 19:01 ` Yang Shi
2021-08-06 5:21 ` Hugh Dickins
2021-08-06 17:41 ` Yang Shi
2021-08-05 23:04 ` Yang Shi
2021-08-06 5:43 ` Hugh Dickins
2021-08-06 17:57 ` Yang Shi
2021-08-12 18:19 ` Yang Shi
2021-07-30 7:45 ` [PATCH 07/16] memfd: memfd_create(name, MFD_HUGEPAGE) for shmem huge pages Hugh Dickins
2021-07-30 12:01 ` kernel test robot
2021-08-04 14:03 ` Kirill A. Shutemov
2021-08-06 3:33 ` Hugh Dickins [this message]
2021-07-30 7:48 ` [PATCH 08/16] huge tmpfs: fcntl(fd, F_HUGEPAGE) and fcntl(fd, F_NOHUGEPAGE) Hugh Dickins
2021-08-04 14:08 ` Kirill A. Shutemov
2021-08-06 4:34 ` Hugh Dickins
2021-07-30 7:51 ` [PATCH 09/16] huge tmpfs: decide stat.st_blksize by shmem_is_huge() Hugh Dickins
2021-07-30 23:40 ` Yang Shi
2021-07-30 7:55 ` [PATCH 10/16] tmpfs: fcntl(fd, F_MEM_LOCK) to memlock a tmpfs file Hugh Dickins
2021-08-03 1:38 ` Matthew Wilcox
2021-08-04 9:15 ` Hugh Dickins
2021-07-30 7:57 ` [PATCH 11/16] tmpfs: fcntl(fd, F_MEM_LOCKED) to test if memlocked Hugh Dickins
2021-07-30 8:00 ` [PATCH 12/16] tmpfs: refuse memlock when fallocated beyond i_size Hugh Dickins
2021-07-30 8:03 ` [PATCH 13/16] mm: bool user_shm_lock(loff_t size, struct ucounts *) Hugh Dickins
2021-07-30 8:06 ` [PATCH 14/16] mm: user_shm_lock(,,getuc) and user_shm_unlock(,,putuc) Hugh Dickins
2021-07-30 8:09 ` [PATCH 15/16] tmpfs: permit changing size of memlocked file Hugh Dickins
2021-07-30 8:13 ` [PATCH 16/16] memfd: memfd_create(name, MFD_MEM_LOCK) for memlocked shmem Hugh Dickins
2021-07-30 11:24 ` kernel test robot
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).