From: "Matthew Wilcox (Oracle)" <willy@infradead.org>
To: linux-mm@kvack.org
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org
Subject: [PATCH v4 00/25] Page folios
Date: Fri, 5 Mar 2021 04:18:36 +0000 [thread overview]
Message-ID: <20210305041901.2396498-1-willy@infradead.org> (raw)
Our type system does not currently distinguish between tail pages and
head or single pages. This is a problem because we call compound_head()
multiple times (and the compiler cannot optimise it out), bloating the
kernel. It also makes programming hard as it is often unclear whether
a function operates on an individual page, or an entire compound page.
This patch series introduces the struct folio, which is a type that
represents an entire compound page. This initial set reduces the kernel
size by approximately 6kB, although its real purpose is adding
infrastructure to enable further use of the folio.
The big correctness proof that exists in this patch series is that we
never lock or wait for writeback on a tail page. This is important as
we would miss wakeups due to being on the wrong page waitqueue if we
ever did.
I analysed the text size reduction using a config based on Oracle UEK
with all modules changed to built-in. That's obviously not a kernel
which makes sense to run, but it serves to compare the effects on (many
common) filesystems & drivers, not just the core.
add/remove: 33510/33499 grow/shrink: 1831/1898 up/down: 888762/-894719 (-5957)
For a Debian config, just comparing vmlinux.o gives a reduction of 3828
bytes of text and 72 bytes of data:
text data bss dec hex filename
16125879 4421122 1846344 22393345 155b201 linus/vmlinux.o
16122051 4421050 1846344 22389445 155a2c5 folio/vmlinux.o
For nfs (a module I happened to notice was particularly affected), it's
a reduction of 588 bytes of text and 16 bytes of data:
257142 59228 408 316778 4d56a linus/fs/nfs/nfs.o
256554 59212 408 316174 4d30e folio/fs/nfs/nfs.o
Current tree at:
https://git.infradead.org/users/willy/pagecache.git/shortlog/refs/heads/folio
(contains another ~70 patches on top of this batch)
v4:
- Rebase on current Linus tree (including swap fix)
- Analyse each patch in terms of its effects on kernel text size.
A few were modified to improve their effect. In particular, where
pushing calls to page_folio() into the callers resulted in unacceptable
size increases, the wrapper was placed in mm/folio-compat.c. This lets
us see all the places which are good targets for conversion to folios.
- Some of the patches were reordered, split or merged in order to make
more logical sense.
- Use nth_page() for folio_next() if we're using SPARSEMEM and not
VMEMMAP (Zi Yan)
- Increment and decrement page stats in units of pages instead of units
of folios (Zi Yan)
v3:
- Rebase on next-20210127. Two major sources of conflict, the
generic_file_buffered_read refactoring (in akpm tree) and the
fscache work (in dhowells tree).
v2:
- Pare patch series back to just infrastructure and the page waiting
parts.
Matthew Wilcox (Oracle) (25):
mm: Introduce struct folio
mm: Add folio_pgdat and folio_zone
mm/vmstat: Add functions to account folio statistics
mm/debug: Add VM_BUG_ON_FOLIO and VM_WARN_ON_ONCE_FOLIO
mm: Add put_folio
mm: Add get_folio
mm: Create FolioFlags
mm: Handle per-folio private data
mm: Add folio_index, folio_page and folio_contains
mm/util: Add folio_mapping and folio_file_mapping
mm/memcg: Add folio wrappers for various memcontrol functions
mm/filemap: Add unlock_folio
mm/filemap: Add lock_folio
mm/filemap: Add lock_folio_killable
mm/filemap: Convert lock_page_async to lock_folio_async
mm/filemap: Convert end_page_writeback to end_folio_writeback
mm/filemap: Add wait_on_folio_locked & wait_on_folio_locked_killable
mm/page-writeback: Add wait_on_folio_writeback
mm/page-writeback: Add wait_for_stable_folio
mm/filemap: Convert wait_on_page_bit to wait_on_folio_bit
mm/filemap: Add __lock_folio_or_retry
mm/filemap: Convert wake_up_page_bit to wake_up_folio_bit
mm/page-writeback: Convert test_clear_page_writeback to take a folio
mm/filemap: Convert page wait queues to be folios
cachefiles: Switch to wait_page_key
fs/afs/write.c | 21 ++--
fs/cachefiles/rdwr.c | 13 +--
fs/io_uring.c | 2 +-
include/linux/fscache.h | 5 +
include/linux/memcontrol.h | 30 +++++
include/linux/mm.h | 88 ++++++++++-----
include/linux/mm_types.h | 33 ++++++
include/linux/mmdebug.h | 20 ++++
include/linux/page-flags.h | 106 ++++++++++++++----
include/linux/pagemap.h | 197 +++++++++++++++++++++++----------
include/linux/vmstat.h | 83 ++++++++++++++
mm/Makefile | 2 +-
mm/filemap.c | 218 ++++++++++++++++++-------------------
mm/folio-compat.c | 37 +++++++
mm/memory.c | 8 +-
mm/page-writeback.c | 58 +++++-----
mm/swapfile.c | 6 +-
mm/util.c | 20 ++--
18 files changed, 666 insertions(+), 281 deletions(-)
create mode 100644 mm/folio-compat.c
--
2.30.0
next reply other threads:[~2021-03-05 4:19 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-05 4:18 Matthew Wilcox (Oracle) [this message]
2021-03-05 4:18 ` [PATCH v4 01/25] mm: Introduce struct folio Matthew Wilcox (Oracle)
2021-03-13 20:37 ` Andrew Morton
2021-03-14 4:07 ` Matthew Wilcox
2021-03-17 17:14 ` Christoph Hellwig
2021-03-18 23:56 ` Balbir Singh
2021-03-19 1:25 ` Matthew Wilcox
2021-03-20 2:09 ` Balbir Singh
2021-03-22 2:52 ` Nicholas Piggin
2021-03-22 3:54 ` Matthew Wilcox
2021-03-05 4:18 ` [PATCH v4 02/25] mm: Add folio_pgdat and folio_zone Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 03/25] mm/vmstat: Add functions to account folio statistics Matthew Wilcox (Oracle)
2021-03-13 20:37 ` Andrew Morton
2021-03-14 4:11 ` Matthew Wilcox
2021-03-14 4:51 ` Andrew Morton
2021-03-17 17:16 ` Christoph Hellwig
2021-03-05 4:18 ` [PATCH v4 04/25] mm/debug: Add VM_BUG_ON_FOLIO and VM_WARN_ON_ONCE_FOLIO Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 05/25] mm: Add put_folio Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 06/25] mm: Add get_folio Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 07/25] mm: Create FolioFlags Matthew Wilcox (Oracle)
2021-03-15 2:24 ` Matthew Wilcox
2021-03-05 4:18 ` [PATCH v4 08/25] mm: Handle per-folio private data Matthew Wilcox (Oracle)
2021-03-17 17:20 ` Christoph Hellwig
2021-03-18 17:57 ` Matthew Wilcox
2021-03-05 4:18 ` [PATCH v4 09/25] mm: Add folio_index, folio_page and folio_contains Matthew Wilcox (Oracle)
2021-03-13 20:37 ` Andrew Morton
2021-03-14 3:45 ` Matthew Wilcox
2021-03-17 17:22 ` Christoph Hellwig
2021-03-05 4:18 ` [PATCH v4 10/25] mm/util: Add folio_mapping and folio_file_mapping Matthew Wilcox (Oracle)
2021-03-17 17:26 ` Christoph Hellwig
2021-03-05 4:18 ` [PATCH v4 11/25] mm/memcg: Add folio wrappers for various memcontrol functions Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 12/25] mm/filemap: Add unlock_folio Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 13/25] mm/filemap: Add lock_folio Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 14/25] mm/filemap: Add lock_folio_killable Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 15/25] mm/filemap: Convert lock_page_async to lock_folio_async Matthew Wilcox (Oracle)
2021-03-17 17:29 ` Christoph Hellwig
2021-03-05 4:18 ` [PATCH v4 16/25] mm/filemap: Convert end_page_writeback to end_folio_writeback Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 17/25] mm/filemap: Add wait_on_folio_locked & wait_on_folio_locked_killable Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 18/25] mm/page-writeback: Add wait_on_folio_writeback Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 19/25] mm/page-writeback: Add wait_for_stable_folio Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 20/25] mm/filemap: Convert wait_on_page_bit to wait_on_folio_bit Matthew Wilcox (Oracle)
2021-03-17 17:39 ` Christoph Hellwig
2021-03-05 4:18 ` [PATCH v4 21/25] mm/filemap: Add __lock_folio_or_retry Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 22/25] mm/filemap: Convert wake_up_page_bit to wake_up_folio_bit Matthew Wilcox (Oracle)
2021-03-05 4:18 ` [PATCH v4 23/25] mm/page-writeback: Convert test_clear_page_writeback to take a folio Matthew Wilcox (Oracle)
2021-03-05 4:19 ` [PATCH v4 24/25] mm/filemap: Convert page wait queues to be folios Matthew Wilcox (Oracle)
2021-03-05 4:19 ` [PATCH v4 25/25] cachefiles: Switch to wait_page_key Matthew Wilcox (Oracle)
2021-03-17 17:45 ` Christoph Hellwig
2021-03-13 20:36 ` [PATCH v4 00/25] Page folios Andrew Morton
2021-03-14 2:30 ` Matthew Wilcox
2021-03-14 3:09 ` Hugh Dickins
2021-03-14 3:09 ` Hugh Dickins
2021-03-15 11:55 ` Kirill A. Shutemov
2021-03-15 12:38 ` Michal Hocko
2021-03-15 19:09 ` Christoph Hellwig
2021-03-15 19:40 ` Matthew Wilcox
2021-03-15 22:27 ` Dave Chinner
2021-03-17 17:48 ` Christoph Hellwig
2021-03-15 21:33 ` Andi Kleen
2021-03-15 21:33 ` Andi Kleen
2021-03-15 13:45 ` Matthew Wilcox
2021-03-19 4:01 ` Matthew Wilcox
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210305041901.2396498-1-willy@infradead.org \
--to=willy@infradead.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.