From: Matthew Wilcox <willy@infradead.org>
To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
linux-kernel@vger.kernel.org
Cc: Matthew Wilcox <willy@infradead.org>, Jan Kara <jack@suse.cz>,
Jeff Layton <jlayton@redhat.com>,
Lukas Czerner <lczerner@redhat.com>,
Ross Zwisler <ross.zwisler@linux.intel.com>,
Christoph Hellwig <hch@lst.de>,
Goldwyn Rodrigues <rgoldwyn@suse.com>,
Nicholas Piggin <npiggin@gmail.com>,
Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>,
linux-nilfs@vger.kernel.org, Jaegeuk Kim <jaegeuk@kernel.org>,
Chao Yu <yuchao0@huawei.com>,
linux-f2fs-devel@lists.sourceforge.net
Subject: [PATCH v14 00/74] Convert page cache to XArray
Date: Sat, 16 Jun 2018 18:59:38 -0700 [thread overview]
Message-ID: <20180617020052.4759-1-willy@infradead.org> (raw)
The XArray is a replacement for the radix tree. For the moment it uses
the same data structures, enabling a gradual replacement. This patch
set implements the XArray and converts the page cache to use it.
A version of these patches has been running under xfstests for over
48 hours, so I have some confidence in them. The DAX changes have now
also had a reasonable test outing. This is based on next-20180615 and
is available as a git tree at
git://git.infradead.org/users/willy/linux-dax.git xarray-20180615
I shall create a git branch from -rc1 and ask for that to be included in
-next. I'm a little concerned I still have no reviews on some of the
later patches.
Changes since v13:
- Actually fixed bug in workingset conversion that led to exceptional
entries not being deleted from the XArray. Not sure how I dropped
that patch for v13. Thanks to David Sterba for noticing.
- Fixed bug in DAX writeback conversion that failed to wake up waiters.
Thanks to Ross for testing, and to Dan & Jeff for helping me get a
setup working to reproduce the problem.
- Converted the new dax_lock_page / dax_unlock_page functions.
- Moved XArray test suite entirely into the test_xarray kernel module
to match other test suites. It can still be built in userspace as
part of the radix tree test suite.
- Changed email address.
- Moved a few functions into different patches to make the test-suite
additions more logical.
- Fixed a bug in XA_BUG_ON (oh the irony) where it evaluated the
condition twice.
- Constified xa_head() / xa_parent() / xa_entry() and their _locked
variants.
- Moved xa_parent() to xarray.h so it can be used from the workingset code.
- Call the xarray testsuite from the radix tree test suite to ensure
that I remember to run both test suites ;-)
- Added some more tests to the test suite.
Matthew Wilcox (74):
Update email address
radix tree test suite: Enable ubsan
dax: Fix use of zero page
xarray: Replace exceptional entries
xarray: Change definition of sibling entries
xarray: Add definition of struct xarray
xarray: Define struct xa_node
xarray: Add documentation
xarray: Add XArray load operation
xarray: Add XArray tags
xarray: Add XArray unconditional store operations
xarray: Add XArray conditional store operations
xarray: Add XArray iterators
xarray: Extract entries from an XArray
xarray: Destroy an XArray
xarray: Step through an XArray
xarray: Add xas_for_each_conflict
xarray: Add xas_create_range
xarray: Add MAINTAINERS entry
page cache: Rearrange address_space
page cache: Convert hole search to XArray
page cache: Add and replace pages using the XArray
page cache: Convert page deletion to XArray
page cache: Convert find_get_entry to XArray
page cache: Convert find_get_entries to XArray
page cache: Convert find_get_pages_range to XArray
page cache: Convert find_get_pages_contig to XArray
page cache; Convert find_get_pages_range_tag to XArray
page cache: Convert find_get_entries_tag to XArray
page cache: Convert filemap_map_pages to XArray
radix tree test suite: Convert regression1 to XArray
page cache: Convert delete_batch to XArray
page cache: Remove stray radix comment
page cache: Convert filemap_range_has_page to XArray
mm: Convert page-writeback to XArray
mm: Convert workingset to XArray
mm: Convert truncate to XArray
mm: Convert add_to_swap_cache to XArray
mm: Convert delete_from_swap_cache to XArray
mm: Convert __do_page_cache_readahead to XArray
mm: Convert page migration to XArray
mm: Convert huge_memory to XArray
mm: Convert collapse_shmem to XArray
mm: Convert khugepaged_scan_shmem to XArray
mm: Convert is_page_cache_freeable to XArray
pagevec: Use xa_tag_t
shmem: Convert shmem_radix_tree_replace to XArray
shmem: Convert shmem_confirm_swap to XArray
shmem: Convert find_swap_entry to XArray
shmem: Convert shmem_add_to_page_cache to XArray
shmem: Convert shmem_alloc_hugepage to XArray
shmem: Convert shmem_free_swap to XArray
shmem: Convert shmem_partial_swap_usage to XArray
memfd: Convert memfd_wait_for_pins to XArray
memfd: Convert memfd_tag_pins to XArray
shmem: Comment fixups
btrfs: Convert page cache to XArray
fs: Convert buffer to XArray
fs: Convert writeback to XArray
nilfs2: Convert to XArray
f2fs: Convert to XArray
dax: Rename some functions
dax: Hash on XArray instead of mapping
dax: Convert dax_insert_pfn_mkwrite to XArray
dax: Convert dax_layout_busy_page to XArray
dax: Convert __dax_invalidate_entry to XArray
dax: Convert dax writeback to XArray
dax: Convert dax_lock_page to XArray
dax: Convert page fault handlers to XArray
page cache: Finish XArray conversion
radix tree: Remove radix_tree_update_node_t
radix tree: Remove split/join code
radix tree: Remove radix_tree_maybe_preload_order
radix tree: Remove radix_tree_clear_tags
.clang-format | 1 -
.mailmap | 7 +
Documentation/core-api/index.rst | 1 +
Documentation/core-api/xarray.rst | 395 ++++
MAINTAINERS | 17 +-
arch/powerpc/include/asm/book3s/64/pgtable.h | 4 +-
arch/powerpc/include/asm/nohash/64/pgtable.h | 4 +-
drivers/gpu/drm/i915/i915_gem.c | 17 +-
fs/btrfs/compression.c | 6 +-
fs/btrfs/extent_io.c | 12 +-
fs/buffer.c | 14 +-
fs/dax.c | 878 ++++-----
fs/ext4/inode.c | 2 +-
fs/f2fs/data.c | 6 +-
fs/f2fs/dir.c | 2 +-
fs/f2fs/f2fs.h | 2 +-
fs/f2fs/inline.c | 2 +-
fs/f2fs/node.c | 6 +-
fs/fs-writeback.c | 25 +-
fs/gfs2/aops.c | 2 +-
fs/inode.c | 2 +-
fs/nfs/blocklayout/blocklayout.c | 2 +-
fs/nilfs2/btnode.c | 26 +-
fs/nilfs2/page.c | 29 +-
fs/proc/task_mmu.c | 2 +-
include/linux/fs.h | 63 +-
include/linux/pagemap.h | 10 +-
include/linux/pagevec.h | 8 +-
include/linux/radix-tree.h | 136 +-
include/linux/swap.h | 22 +-
include/linux/swapops.h | 19 +-
include/linux/xarray.h | 1047 +++++++++-
lib/Kconfig.debug | 3 +
lib/Makefile | 3 +-
lib/idr.c | 66 +-
lib/radix-tree.c | 575 +-----
lib/test_xarray.c | 676 +++++++
lib/xarray.c | 1753 +++++++++++++++++
mm/filemap.c | 723 +++----
mm/huge_memory.c | 17 +-
mm/khugepaged.c | 177 +-
mm/madvise.c | 2 +-
mm/memcontrol.c | 2 +-
mm/memfd.c | 105 +-
mm/migrate.c | 48 +-
mm/mincore.c | 2 +-
mm/page-writeback.c | 72 +-
mm/readahead.c | 10 +-
mm/shmem.c | 201 +-
mm/swap.c | 6 +-
mm/swap_state.c | 119 +-
mm/truncate.c | 27 +-
mm/vmscan.c | 10 +-
mm/workingset.c | 69 +-
tools/include/asm-generic/bitops.h | 1 +
tools/include/asm-generic/bitops/atomic.h | 9 -
tools/include/asm-generic/bitops/non-atomic.h | 109 +
tools/include/linux/bitmap.h | 1 +
tools/include/linux/kernel.h | 1 +
tools/include/linux/spinlock.h | 12 +-
tools/testing/radix-tree/.gitignore | 1 +
tools/testing/radix-tree/Makefile | 16 +-
tools/testing/radix-tree/benchmark.c | 91 -
tools/testing/radix-tree/bitmap.c | 23 +
tools/testing/radix-tree/idr-test.c | 6 +-
tools/testing/radix-tree/linux/bug.h | 1 +
tools/testing/radix-tree/linux/kconfig.h | 1 +
tools/testing/radix-tree/linux/kernel.h | 5 +
tools/testing/radix-tree/linux/lockdep.h | 11 +
tools/testing/radix-tree/linux/radix-tree.h | 1 -
tools/testing/radix-tree/linux/rcupdate.h | 2 +
tools/testing/radix-tree/linux/xarray.h | 2 +
tools/testing/radix-tree/main.c | 21 +-
tools/testing/radix-tree/multiorder.c | 272 +--
tools/testing/radix-tree/regression1.c | 58 +-
tools/testing/radix-tree/regression3.c | 23 -
tools/testing/radix-tree/tag_check.c | 29 -
tools/testing/radix-tree/test.c | 8 +-
tools/testing/radix-tree/test.h | 1 +
tools/testing/radix-tree/xarray.c | 33 +
80 files changed, 5462 insertions(+), 2711 deletions(-)
create mode 100644 Documentation/core-api/xarray.rst
create mode 100644 lib/test_xarray.c
create mode 100644 lib/xarray.c
create mode 100644 tools/include/asm-generic/bitops/non-atomic.h
create mode 100644 tools/testing/radix-tree/bitmap.c
create mode 100644 tools/testing/radix-tree/linux/kconfig.h
create mode 100644 tools/testing/radix-tree/linux/lockdep.h
create mode 100644 tools/testing/radix-tree/linux/xarray.h
create mode 100644 tools/testing/radix-tree/xarray.c
--
2.17.1
next reply other threads:[~2018-06-17 1:59 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-17 1:59 Matthew Wilcox [this message]
2018-06-17 1:59 ` [PATCH v14 01/74] Update email address Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 02/74] radix tree test suite: Enable ubsan Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 03/74] dax: Fix use of zero page Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 04/74] xarray: Replace exceptional entries Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 05/74] xarray: Change definition of sibling entries Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 06/74] xarray: Add definition of struct xarray Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 07/74] xarray: Define struct xa_node Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 08/74] xarray: Add documentation Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 09/74] xarray: Add XArray load operation Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 10/74] xarray: Add XArray tags Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 11/74] xarray: Add XArray unconditional store operations Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 12/74] xarray: Add XArray conditional " Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 13/74] xarray: Add XArray iterators Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 14/74] xarray: Extract entries from an XArray Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 15/74] xarray: Destroy " Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 16/74] xarray: Step through " Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 17/74] xarray: Add xas_for_each_conflict Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 18/74] xarray: Add xas_create_range Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 19/74] xarray: Add MAINTAINERS entry Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 20/74] page cache: Rearrange address_space Matthew Wilcox
2018-06-17 1:59 ` [PATCH v14 21/74] page cache: Convert hole search to XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 22/74] page cache: Add and replace pages using the XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 23/74] page cache: Convert page deletion to XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 24/74] page cache: Convert find_get_entry " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 25/74] page cache: Convert find_get_entries " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 26/74] page cache: Convert find_get_pages_range " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 27/74] page cache: Convert find_get_pages_contig " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 28/74] page cache; Convert find_get_pages_range_tag " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 29/74] page cache: Convert find_get_entries_tag " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 30/74] page cache: Convert filemap_map_pages " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 31/74] radix tree test suite: Convert regression1 " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 32/74] page cache: Convert delete_batch " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 33/74] page cache: Remove stray radix comment Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 34/74] page cache: Convert filemap_range_has_page to XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 35/74] mm: Convert page-writeback " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 36/74] mm: Convert workingset " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 37/74] mm: Convert truncate " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 38/74] mm: Convert add_to_swap_cache " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 39/74] mm: Convert delete_from_swap_cache " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 40/74] mm: Convert __do_page_cache_readahead " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 41/74] mm: Convert page migration " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 42/74] mm: Convert huge_memory " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 43/74] mm: Convert collapse_shmem " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 44/74] mm: Convert khugepaged_scan_shmem " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 45/74] mm: Convert is_page_cache_freeable " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 46/74] pagevec: Use xa_tag_t Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 47/74] shmem: Convert shmem_radix_tree_replace to XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 48/74] shmem: Convert shmem_confirm_swap " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 49/74] shmem: Convert find_swap_entry " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 50/74] shmem: Convert shmem_add_to_page_cache " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 51/74] shmem: Convert shmem_alloc_hugepage " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 52/74] shmem: Convert shmem_free_swap " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 53/74] shmem: Convert shmem_partial_swap_usage " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 54/74] memfd: Convert memfd_wait_for_pins " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 55/74] memfd: Convert memfd_tag_pins " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 56/74] shmem: Comment fixups Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 57/74] btrfs: Convert page cache to XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 58/74] fs: Convert buffer " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 59/74] fs: Convert writeback " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 60/74] nilfs2: Convert " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 61/74] f2fs: " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 62/74] dax: Rename some functions Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 63/74] dax: Hash on XArray instead of mapping Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 64/74] dax: Convert dax_insert_pfn_mkwrite to XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 65/74] dax: Convert dax_layout_busy_page " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 66/74] dax: Convert __dax_invalidate_entry " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 67/74] dax: Convert dax writeback " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 68/74] dax: Convert dax_lock_page " Matthew Wilcox
2018-06-29 17:30 ` Ross Zwisler
2018-07-06 11:54 ` [PATCH] fs/dax: remove unused function Anders Roxell
2018-07-06 13:04 ` [PATCH v14 68/74] dax: Convert dax_lock_page to XArray Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 69/74] dax: Convert page fault handlers " Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 70/74] page cache: Finish XArray conversion Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 71/74] radix tree: Remove radix_tree_update_node_t Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 72/74] radix tree: Remove split/join code Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 73/74] radix tree: Remove radix_tree_maybe_preload_order Matthew Wilcox
2018-06-17 2:00 ` [PATCH v14 74/74] radix tree: Remove radix_tree_clear_tags Matthew Wilcox
2018-06-19 3:12 ` [PATCH v14 00/74] Convert page cache to XArray Ross Zwisler
2018-06-19 9:22 ` Matthew Wilcox
2018-06-19 16:40 ` Ross Zwisler
2018-06-19 17:16 ` Matthew Wilcox
2018-06-27 11:05 ` Matthew Wilcox
2018-06-27 19:44 ` Ross Zwisler
2018-06-28 8:39 ` Matthew Wilcox
2018-06-28 16:30 ` Ross Zwisler
2018-07-25 21:03 ` Matthew Wilcox
2018-07-25 21:12 ` Ross Zwisler
2018-07-27 17:20 ` Ross Zwisler
2018-07-30 15:43 ` Matthew Wilcox
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180617020052.4759-1-willy@infradead.org \
--to=willy@infradead.org \
--cc=hch@lst.de \
--cc=jack@suse.cz \
--cc=jaegeuk@kernel.org \
--cc=jlayton@redhat.com \
--cc=konishi.ryusuke@lab.ntt.co.jp \
--cc=lczerner@redhat.com \
--cc=linux-f2fs-devel@lists.sourceforge.net \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nilfs@vger.kernel.org \
--cc=npiggin@gmail.com \
--cc=rgoldwyn@suse.com \
--cc=ross.zwisler@linux.intel.com \
--cc=yuchao0@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).