From: Matthew Wilcox <willy@infradead.org>
To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org
Cc: Matthew Wilcox <mawilcox@microsoft.com>, Jan Kara <jack@suse.cz>,
Jeff Layton <jlayton@redhat.com>,
Lukas Czerner <lczerner@redhat.com>,
Ross Zwisler <ross.zwisler@linux.intel.com>,
Christoph Hellwig <hch@lst.de>,
Goldwyn Rodrigues <rgoldwyn@suse.com>,
Nicholas Piggin <npiggin@gmail.com>,
Ryusuke Konishi <konishi.ryusuke@lab.ntt.co.jp>,
linux-nilfs@vger.kernel.org, Jaegeuk Kim <jaegeuk@kernel.org>,
Chao Yu <yuchao0@huawei.com>,
linux-f2fs-devel@lists.sourceforge.net,
Oleg Drokin <oleg.drokin@intel.com>,
Andreas Dilger <andreas.dilger@intel.com>,
James Simmons <jsimmons@infradead.org>,
Mike Kravetz <mike.kravetz@oracle.com>
Subject: [PATCH v11 24/63] mm: Convert page-writeback to XArray
Date: Sat, 14 Apr 2018 07:12:37 -0700 [thread overview]
Message-ID: <20180414141316.7167-25-willy@infradead.org> (raw)
In-Reply-To: <20180414141316.7167-1-willy@infradead.org>
From: Matthew Wilcox <mawilcox@microsoft.com>
Includes moving mapping_tagged() to fs.h as a static inline, and
changing it to return bool.
Signed-off-by: Matthew Wilcox <mawilcox@microsoft.com>
---
include/linux/fs.h | 17 ++++++-----
mm/page-writeback.c | 72 ++++++++++++++++-----------------------------
2 files changed, 36 insertions(+), 53 deletions(-)
diff --git a/include/linux/fs.h b/include/linux/fs.h
index 0a4e213f1069..620cce9a97f0 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -469,15 +469,18 @@ struct block_device {
struct mutex bd_fsfreeze_mutex;
} __randomize_layout;
+/* XArray tags, for tagging dirty and writeback pages in the pagecache. */
+#define PAGECACHE_TAG_DIRTY XA_TAG_0
+#define PAGECACHE_TAG_WRITEBACK XA_TAG_1
+#define PAGECACHE_TAG_TOWRITE XA_TAG_2
+
/*
- * Radix-tree tags, for tagging dirty and writeback pages within the pagecache
- * radix trees
+ * Returns true if any of the pages in the mapping are marked with the tag.
*/
-#define PAGECACHE_TAG_DIRTY 0
-#define PAGECACHE_TAG_WRITEBACK 1
-#define PAGECACHE_TAG_TOWRITE 2
-
-int mapping_tagged(struct address_space *mapping, int tag);
+static inline bool mapping_tagged(struct address_space *mapping, xa_tag_t tag)
+{
+ return xa_tagged(&mapping->i_pages, tag);
+}
static inline void i_mmap_lock_write(struct address_space *mapping)
{
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 337c6afb3345..93b44af5abe2 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -2098,34 +2098,25 @@ void __init page_writeback_init(void)
* dirty pages in the file (thus it is important for this function to be quick
* so that it can tag pages faster than a dirtying process can create them).
*/
-/*
- * We tag pages in batches of WRITEBACK_TAG_BATCH to reduce the i_pages lock
- * latency.
- */
void tag_pages_for_writeback(struct address_space *mapping,
pgoff_t start, pgoff_t end)
{
-#define WRITEBACK_TAG_BATCH 4096
- unsigned long tagged = 0;
- struct radix_tree_iter iter;
- void **slot;
+ XA_STATE(xas, &mapping->i_pages, start);
+ unsigned int tagged = 0;
+ void *page;
- xa_lock_irq(&mapping->i_pages);
- radix_tree_for_each_tagged(slot, &mapping->i_pages, &iter, start,
- PAGECACHE_TAG_DIRTY) {
- if (iter.index > end)
- break;
- radix_tree_iter_tag_set(&mapping->i_pages, &iter,
- PAGECACHE_TAG_TOWRITE);
- tagged++;
- if ((tagged % WRITEBACK_TAG_BATCH) != 0)
+ xas_lock_irq(&xas);
+ xas_for_each_tag(&xas, page, end, PAGECACHE_TAG_DIRTY) {
+ xas_set_tag(&xas, PAGECACHE_TAG_TOWRITE);
+ if (++tagged % XA_CHECK_SCHED)
continue;
- slot = radix_tree_iter_resume(slot, &iter);
- xa_unlock_irq(&mapping->i_pages);
+
+ xas_pause(&xas);
+ xas_unlock_irq(&xas);
cond_resched();
- xa_lock_irq(&mapping->i_pages);
+ xas_lock_irq(&xas);
}
- xa_unlock_irq(&mapping->i_pages);
+ xas_unlock_irq(&xas);
}
EXPORT_SYMBOL(tag_pages_for_writeback);
@@ -2165,7 +2156,7 @@ int write_cache_pages(struct address_space *mapping,
pgoff_t done_index;
int cycled;
int range_whole = 0;
- int tag;
+ xa_tag_t tag;
pagevec_init(&pvec);
if (wbc->range_cyclic) {
@@ -2446,7 +2437,7 @@ void account_page_cleaned(struct page *page, struct address_space *mapping,
/*
* For address_spaces which do not use buffers. Just tag the page as dirty in
- * its radix tree.
+ * the xarray.
*
* This is also used when a single buffer is being dirtied: we want to set the
* page dirty in that case, but not all the buffers. This is a "bottom-up"
@@ -2472,7 +2463,7 @@ int __set_page_dirty_nobuffers(struct page *page)
BUG_ON(page_mapping(page) != mapping);
WARN_ON_ONCE(!PagePrivate(page) && !PageUptodate(page));
account_page_dirtied(page, mapping);
- radix_tree_tag_set(&mapping->i_pages, page_index(page),
+ __xa_set_tag(&mapping->i_pages, page_index(page),
PAGECACHE_TAG_DIRTY);
xa_unlock_irqrestore(&mapping->i_pages, flags);
unlock_page_memcg(page);
@@ -2635,13 +2626,13 @@ EXPORT_SYMBOL(__cancel_dirty_page);
* Returns true if the page was previously dirty.
*
* This is for preparing to put the page under writeout. We leave the page
- * tagged as dirty in the radix tree so that a concurrent write-for-sync
+ * tagged as dirty in the xarray so that a concurrent write-for-sync
* can discover it via a PAGECACHE_TAG_DIRTY walk. The ->writepage
* implementation will run either set_page_writeback() or set_page_dirty(),
- * at which stage we bring the page's dirty flag and radix-tree dirty tag
+ * at which stage we bring the page's dirty flag and xarray dirty tag
* back into sync.
*
- * This incoherency between the page's dirty flag and radix-tree tag is
+ * This incoherency between the page's dirty flag and xarray tag is
* unfortunate, but it only exists while the page is locked.
*/
int clear_page_dirty_for_io(struct page *page)
@@ -2722,7 +2713,7 @@ int test_clear_page_writeback(struct page *page)
xa_lock_irqsave(&mapping->i_pages, flags);
ret = TestClearPageWriteback(page);
if (ret) {
- radix_tree_tag_clear(&mapping->i_pages, page_index(page),
+ __xa_clear_tag(&mapping->i_pages, page_index(page),
PAGECACHE_TAG_WRITEBACK);
if (bdi_cap_account_writeback(bdi)) {
struct bdi_writeback *wb = inode_to_wb(inode);
@@ -2762,11 +2753,13 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
lock_page_memcg(page);
if (mapping && mapping_use_writeback_tags(mapping)) {
+ XA_STATE(xas, &mapping->i_pages, page_index(page));
struct inode *inode = mapping->host;
struct backing_dev_info *bdi = inode_to_bdi(inode);
unsigned long flags;
- xa_lock_irqsave(&mapping->i_pages, flags);
+ xas_lock_irqsave(&xas, flags);
+ xas_load(&xas);
ret = TestSetPageWriteback(page);
if (!ret) {
bool on_wblist;
@@ -2774,8 +2767,7 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
on_wblist = mapping_tagged(mapping,
PAGECACHE_TAG_WRITEBACK);
- radix_tree_tag_set(&mapping->i_pages, page_index(page),
- PAGECACHE_TAG_WRITEBACK);
+ xas_set_tag(&xas, PAGECACHE_TAG_WRITEBACK);
if (bdi_cap_account_writeback(bdi))
inc_wb_stat(inode_to_wb(inode), WB_WRITEBACK);
@@ -2788,12 +2780,10 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
sb_mark_inode_writeback(mapping->host);
}
if (!PageDirty(page))
- radix_tree_tag_clear(&mapping->i_pages, page_index(page),
- PAGECACHE_TAG_DIRTY);
+ xas_clear_tag(&xas, PAGECACHE_TAG_DIRTY);
if (!keep_write)
- radix_tree_tag_clear(&mapping->i_pages, page_index(page),
- PAGECACHE_TAG_TOWRITE);
- xa_unlock_irqrestore(&mapping->i_pages, flags);
+ xas_clear_tag(&xas, PAGECACHE_TAG_TOWRITE);
+ xas_unlock_irqrestore(&xas, flags);
} else {
ret = TestSetPageWriteback(page);
}
@@ -2807,16 +2797,6 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
}
EXPORT_SYMBOL(__test_set_page_writeback);
-/*
- * Return true if any of the pages in the mapping are marked with the
- * passed tag.
- */
-int mapping_tagged(struct address_space *mapping, int tag)
-{
- return radix_tree_tagged(&mapping->i_pages, tag);
-}
-EXPORT_SYMBOL(mapping_tagged);
-
/**
* wait_for_stable_page() - wait for writeback to finish, if necessary.
* @page: The page to wait on.
--
2.17.0
next prev parent reply other threads:[~2018-04-14 14:12 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-14 14:12 [PATCH v11 00/63] Convert page cache to XArray Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 01/63] xarray: Replace exceptional entries Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 02/63] xarray: Change definition of sibling entries Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 03/63] xarray: Add definition of struct xarray Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 04/63] xarray: Define struct xa_node Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 05/63] xarray: Add documentation Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 06/63] xarray: Add xa_load Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 07/63] xarray: Add xa_get_tag, xa_set_tag and xa_clear_tag Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 08/63] xarray: Add xa_store Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 09/63] xarray: Add xa_cmpxchg and xa_insert Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 10/63] xarray: Add xa_for_each Matthew Wilcox
2018-04-20 12:00 ` Goldwyn Rodrigues
2018-04-21 1:34 ` Matthew Wilcox
2018-04-22 20:33 ` Goldwyn Rodrigues
2018-04-14 14:12 ` [PATCH v11 11/63] xarray: Add xa_extract Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 12/63] xarray: Add xa_destroy Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 13/63] xarray: Add xas_next and xas_prev Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 14/63] xarray: Add xas_create_range Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 15/63] xarray: Add MAINTAINERS entry Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 16/63] page cache: Rearrange address_space Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 17/63] page cache: Convert hole search to XArray Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 18/63] page cache: Add and replace pages using the XArray Matthew Wilcox
2018-04-27 3:24 ` Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 19/63] page cache: Convert page deletion to XArray Matthew Wilcox
2018-04-20 12:00 ` Goldwyn Rodrigues
2018-04-27 2:58 ` Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 20/63] page cache: Convert page cache lookups " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 21/63] page cache: Convert delete_batch " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 22/63] page cache: Remove stray radix comment Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 23/63] page cache: Convert filemap_range_has_page to XArray Matthew Wilcox
2018-04-14 14:12 ` Matthew Wilcox [this message]
2018-04-14 14:12 ` [PATCH v11 25/63] mm: Convert workingset " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 26/63] mm: Convert truncate " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 27/63] mm: Convert add_to_swap_cache " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 28/63] mm: Convert delete_from_swap_cache " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 29/63] mm: Convert __do_page_cache_readahead " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 30/63] mm: Convert page migration " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 31/63] mm: Convert huge_memory " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 32/63] mm: Convert collapse_shmem " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 33/63] mm: Convert khugepaged_scan_shmem " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 34/63] pagevec: Use xa_tag_t Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 35/63] shmem: Convert replace to XArray Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 36/63] shmem: Convert shmem_confirm_swap " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 37/63] shmem: Convert find_swap_entry " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 38/63] shmem: Convert shmem_add_to_page_cache " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 39/63] shmem: Convert shmem_alloc_hugepage " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 40/63] shmem: Convert shmem_free_swap " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 41/63] shmem: Convert shmem_partial_swap_usage " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 42/63] memfd: Convert shmem_wait_for_pins " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 43/63] memfd: Convert shmem_tag_pins " Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 44/63] shmem: Comment fixups Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 45/63] btrfs: Convert page cache to XArray Matthew Wilcox
2018-04-14 14:12 ` [PATCH v11 46/63] fs: Convert buffer " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 47/63] fs: Convert writeback " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 48/63] nilfs2: Convert " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 49/63] f2fs: " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 50/63] lustre: " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 51/63] dax: Fix use of zero page Matthew Wilcox
2018-05-21 4:25 ` Ross Zwisler
2018-04-14 14:13 ` [PATCH v11 52/63] dax: dax_insert_mapping_entry always succeeds Matthew Wilcox
2018-05-21 4:33 ` Ross Zwisler
2018-04-14 14:13 ` [PATCH v11 53/63] dax: Rename some functions Matthew Wilcox
2018-05-21 4:42 ` Ross Zwisler
2018-05-21 10:11 ` Matthew Wilcox
2018-05-22 21:39 ` Ross Zwisler
2018-04-14 14:13 ` [PATCH v11 54/63] dax: Hash on XArray instead of mapping Matthew Wilcox
2018-05-21 4:47 ` Ross Zwisler
2018-05-21 10:25 ` Matthew Wilcox
2018-05-22 21:38 ` Ross Zwisler
2018-04-14 14:13 ` [PATCH v11 55/63] dax: Convert dax_insert_pfn_mkwrite to XArray Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 56/63] dax: Convert __dax_invalidate_entry " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 57/63] dax: Convert dax writeback " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 58/63] dax: Convert page fault handlers " Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 59/63] dax: Return fault code from dax_load_hole Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 60/63] page cache: Finish XArray conversion Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 61/63] radix tree: Remove unused functions Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 62/63] radix tree: Remove radix_tree_update_node_t Matthew Wilcox
2018-04-14 14:13 ` [PATCH v11 63/63] radix tree: Remove radix_tree_clear_tags Matthew Wilcox
2018-04-16 16:01 ` [PATCH v11 00/63] Convert page cache to XArray Ross Zwisler
2018-05-31 21:36 ` Ross Zwisler
2018-05-31 21:37 ` Ross Zwisler
2018-05-31 21:46 ` Matthew Wilcox
2018-05-31 21:53 ` Ross Zwisler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180414141316.7167-25-willy@infradead.org \
--to=willy@infradead.org \
--cc=andreas.dilger@intel.com \
--cc=hch@lst.de \
--cc=jack@suse.cz \
--cc=jaegeuk@kernel.org \
--cc=jlayton@redhat.com \
--cc=jsimmons@infradead.org \
--cc=konishi.ryusuke@lab.ntt.co.jp \
--cc=lczerner@redhat.com \
--cc=linux-f2fs-devel@lists.sourceforge.net \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nilfs@vger.kernel.org \
--cc=mawilcox@microsoft.com \
--cc=mike.kravetz@oracle.com \
--cc=npiggin@gmail.com \
--cc=oleg.drokin@intel.com \
--cc=rgoldwyn@suse.com \
--cc=ross.zwisler@linux.intel.com \
--cc=yuchao0@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).