* + mm-remove-flush_kernel_dcache_page.patch added to -mm tree
@ 2021-07-15 22:34 akpm
0 siblings, 0 replies; only message in thread
From: akpm @ 2021-07-15 22:34 UTC (permalink / raw)
To: alexs, dalias, deanbo422, deller, geoff, green.hu, guoren, hch,
ira.weiny, James.Bottomley, linux, mm-commits, nickhu, paul,
torvalds, tsbogend, ulf.hansson, ysato
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=utf-8, Size: 20837 bytes --]
The patch titled
Subject: mm: remove flush_kernel_dcache_page
has been added to the -mm tree. Its filename is
mm-remove-flush_kernel_dcache_page.patch
This patch should soon appear at
https://ozlabs.org/~akpm/mmots/broken-out/mm-remove-flush_kernel_dcache_page.patch
and later at
https://ozlabs.org/~akpm/mmotm/broken-out/mm-remove-flush_kernel_dcache_page.patch
Before you just go and hit "reply", please:
a) Consider who else should be cc'ed
b) Prefer to cc a suitable mailing list as well
c) Ideally: find the original patch on the mailing list and do a
reply-to-all to that, adding suitable additional cc's
*** Remember to use Documentation/process/submit-checklist.rst when testing your code ***
The -mm tree is included into linux-next and is updated
there every 3-4 working days
------------------------------------------------------
From: Christoph Hellwig <hch@lst.de>
Subject: mm: remove flush_kernel_dcache_page
flush_kernel_dcache_page is a rather confusing interface that implements a
subset of flush_dcache_page by not being able to properly handle page
cache mapped pages.
The only callers left are in the exec code as all other previous
callers were incorrect as they could have dealt with page cache pages.
Replace the calls to flush_kernel_dcache_page with calls to
flush_dcache_page, which for all architectures does either exactly the
same thing, can contains one or more of the following:
1) an optimization to defer the cache flush for page cache pages not
mapped into userspace
2) additional flushing for mapped page cache pages if cache aliases
are possible
Link: https://lkml.kernel.org/r/20210712060928.4161649-7-hch@lst.de
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Cc: Alex Shi <alexs@kernel.org>
Cc: Geoff Levand <geoff@infradead.org>
Cc: Greentime Hu <green.hu@gmail.com>
Cc: Guo Ren <guoren@kernel.org>
Cc: Helge Deller <deller@gmx.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Nick Hu <nickhu@andestech.com>
Cc: Paul Cercueil <paul@crapouillou.net>
Cc: Rich Felker <dalias@libc.org>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: Ulf Hansson <ulf.hansson@linaro.org>
Cc: Vincent Chen <deanbo422@gmail.com>
Cc: Yoshinori Sato <ysato@users.osdn.me>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---
Documentation/core-api/cachetlb.rst | 86 ++++------
Documentation/translations/zh_CN/core-api/cachetlb.rst | 9 -
arch/arm/include/asm/cacheflush.h | 4
arch/arm/mm/flush.c | 33 ---
arch/arm/mm/nommu.c | 6
arch/csky/abiv1/cacheflush.c | 11 -
arch/csky/abiv1/inc/abi/cacheflush.h | 4
arch/mips/include/asm/cacheflush.h | 8
arch/nds32/include/asm/cacheflush.h | 3
arch/nds32/mm/cacheflush.c | 9 -
arch/parisc/include/asm/cacheflush.h | 8
arch/parisc/kernel/cache.c | 3
arch/sh/include/asm/cacheflush.h | 8
block/blk-map.c | 2
fs/exec.c | 6
include/linux/highmem.h | 5
tools/testing/scatterlist/linux/mm.h | 1
17 files changed, 51 insertions(+), 155 deletions(-)
--- a/arch/arm/include/asm/cacheflush.h~mm-remove-flush_kernel_dcache_page
+++ a/arch/arm/include/asm/cacheflush.h
@@ -291,6 +291,7 @@ extern void flush_cache_page(struct vm_a
#define ARCH_IMPLEMENTS_FLUSH_DCACHE_PAGE 1
extern void flush_dcache_page(struct page *);
+#define ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE 1
static inline void flush_kernel_vmap_range(void *addr, int size)
{
if ((cache_is_vivt() || cache_is_vipt_aliasing()))
@@ -312,9 +313,6 @@ static inline void flush_anon_page(struc
__flush_anon_page(vma, page, vmaddr);
}
-#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
-extern void flush_kernel_dcache_page(struct page *);
-
#define flush_dcache_mmap_lock(mapping) xa_lock_irq(&mapping->i_pages)
#define flush_dcache_mmap_unlock(mapping) xa_unlock_irq(&mapping->i_pages)
--- a/arch/arm/mm/flush.c~mm-remove-flush_kernel_dcache_page
+++ a/arch/arm/mm/flush.c
@@ -346,39 +346,6 @@ void flush_dcache_page(struct page *page
EXPORT_SYMBOL(flush_dcache_page);
/*
- * Ensure cache coherency for the kernel mapping of this page. We can
- * assume that the page is pinned via kmap.
- *
- * If the page only exists in the page cache and there are no user
- * space mappings, this is a no-op since the page was already marked
- * dirty at creation. Otherwise, we need to flush the dirty kernel
- * cache lines directly.
- */
-void flush_kernel_dcache_page(struct page *page)
-{
- if (cache_is_vivt() || cache_is_vipt_aliasing()) {
- struct address_space *mapping;
-
- mapping = page_mapping_file(page);
-
- if (!mapping || mapping_mapped(mapping)) {
- void *addr;
-
- addr = page_address(page);
- /*
- * kmap_atomic() doesn't set the page virtual
- * address for highmem pages, and
- * kunmap_atomic() takes care of cache
- * flushing already.
- */
- if (!IS_ENABLED(CONFIG_HIGHMEM) || addr)
- __cpuc_flush_dcache_area(addr, PAGE_SIZE);
- }
- }
-}
-EXPORT_SYMBOL(flush_kernel_dcache_page);
-
-/*
* Flush an anonymous page so that users of get_user_pages()
* can safely access the data. The expected sequence is:
*
--- a/arch/arm/mm/nommu.c~mm-remove-flush_kernel_dcache_page
+++ a/arch/arm/mm/nommu.c
@@ -166,12 +166,6 @@ void flush_dcache_page(struct page *page
}
EXPORT_SYMBOL(flush_dcache_page);
-void flush_kernel_dcache_page(struct page *page)
-{
- __cpuc_flush_dcache_area(page_address(page), PAGE_SIZE);
-}
-EXPORT_SYMBOL(flush_kernel_dcache_page);
-
void copy_to_user_page(struct vm_area_struct *vma, struct page *page,
unsigned long uaddr, void *dst, const void *src,
unsigned long len)
--- a/arch/csky/abiv1/cacheflush.c~mm-remove-flush_kernel_dcache_page
+++ a/arch/csky/abiv1/cacheflush.c
@@ -56,17 +56,6 @@ void update_mmu_cache(struct vm_area_str
}
}
-void flush_kernel_dcache_page(struct page *page)
-{
- struct address_space *mapping;
-
- mapping = page_mapping_file(page);
-
- if (!mapping || mapping_mapped(mapping))
- dcache_wbinv_all();
-}
-EXPORT_SYMBOL(flush_kernel_dcache_page);
-
void flush_cache_range(struct vm_area_struct *vma, unsigned long start,
unsigned long end)
{
--- a/arch/csky/abiv1/inc/abi/cacheflush.h~mm-remove-flush_kernel_dcache_page
+++ a/arch/csky/abiv1/inc/abi/cacheflush.h
@@ -14,12 +14,10 @@ extern void flush_dcache_page(struct pag
#define flush_cache_page(vma, page, pfn) cache_wbinv_all()
#define flush_cache_dup_mm(mm) cache_wbinv_all()
-#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
-extern void flush_kernel_dcache_page(struct page *);
-
#define flush_dcache_mmap_lock(mapping) xa_lock_irq(&mapping->i_pages)
#define flush_dcache_mmap_unlock(mapping) xa_unlock_irq(&mapping->i_pages)
+#define ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE 1
static inline void flush_kernel_vmap_range(void *addr, int size)
{
dcache_wbinv_all();
--- a/arch/mips/include/asm/cacheflush.h~mm-remove-flush_kernel_dcache_page
+++ a/arch/mips/include/asm/cacheflush.h
@@ -125,13 +125,7 @@ static inline void kunmap_noncoherent(vo
kunmap_coherent();
}
-#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
-static inline void flush_kernel_dcache_page(struct page *page)
-{
- BUG_ON(cpu_has_dc_aliases && PageHighMem(page));
- flush_dcache_page(page);
-}
-
+#define ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE 1
/*
* For now flush_kernel_vmap_range and invalidate_kernel_vmap_range both do a
* cache writeback and invalidate operation.
--- a/arch/nds32/include/asm/cacheflush.h~mm-remove-flush_kernel_dcache_page
+++ a/arch/nds32/include/asm/cacheflush.h
@@ -36,8 +36,7 @@ void copy_from_user_page(struct vm_area_
void flush_anon_page(struct vm_area_struct *vma,
struct page *page, unsigned long vaddr);
-#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
-void flush_kernel_dcache_page(struct page *page);
+#define ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE 1
void flush_kernel_vmap_range(void *addr, int size);
void invalidate_kernel_vmap_range(void *addr, int size);
#define flush_dcache_mmap_lock(mapping) xa_lock_irq(&(mapping)->i_pages)
--- a/arch/nds32/mm/cacheflush.c~mm-remove-flush_kernel_dcache_page
+++ a/arch/nds32/mm/cacheflush.c
@@ -318,15 +318,6 @@ void flush_anon_page(struct vm_area_stru
local_irq_restore(flags);
}
-void flush_kernel_dcache_page(struct page *page)
-{
- unsigned long flags;
- local_irq_save(flags);
- cpu_dcache_wbinval_page((unsigned long)page_address(page));
- local_irq_restore(flags);
-}
-EXPORT_SYMBOL(flush_kernel_dcache_page);
-
void flush_kernel_vmap_range(void *addr, int size)
{
unsigned long flags;
--- a/arch/parisc/include/asm/cacheflush.h~mm-remove-flush_kernel_dcache_page
+++ a/arch/parisc/include/asm/cacheflush.h
@@ -36,16 +36,12 @@ void flush_cache_all_local(void);
void flush_cache_all(void);
void flush_cache_mm(struct mm_struct *mm);
-#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
void flush_kernel_dcache_page_addr(void *addr);
-static inline void flush_kernel_dcache_page(struct page *page)
-{
- flush_kernel_dcache_page_addr(page_address(page));
-}
#define flush_kernel_dcache_range(start,size) \
flush_kernel_dcache_range_asm((start), (start)+(size));
+#define ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE 1
void flush_kernel_vmap_range(void *vaddr, int size);
void invalidate_kernel_vmap_range(void *vaddr, int size);
@@ -59,7 +55,7 @@ extern void flush_dcache_page(struct pag
#define flush_dcache_mmap_unlock(mapping) xa_unlock_irq(&mapping->i_pages)
#define flush_icache_page(vma,page) do { \
- flush_kernel_dcache_page(page); \
+ flush_kernel_dcache_page_addr(page_address(page)); \
flush_kernel_icache_page(page_address(page)); \
} while (0)
--- a/arch/parisc/kernel/cache.c~mm-remove-flush_kernel_dcache_page
+++ a/arch/parisc/kernel/cache.c
@@ -334,7 +334,7 @@ void flush_dcache_page(struct page *page
return;
}
- flush_kernel_dcache_page(page);
+ flush_kernel_dcache_page_addr(page_address(page));
if (!mapping)
return;
@@ -375,7 +375,6 @@ EXPORT_SYMBOL(flush_dcache_page);
/* Defined in arch/parisc/kernel/pacache.S */
EXPORT_SYMBOL(flush_kernel_dcache_range_asm);
-EXPORT_SYMBOL(flush_kernel_dcache_page_asm);
EXPORT_SYMBOL(flush_data_cache_local);
EXPORT_SYMBOL(flush_kernel_icache_range_asm);
--- a/arch/sh/include/asm/cacheflush.h~mm-remove-flush_kernel_dcache_page
+++ a/arch/sh/include/asm/cacheflush.h
@@ -63,6 +63,8 @@ static inline void flush_anon_page(struc
if (boot_cpu_data.dcache.n_aliases && PageAnon(page))
__flush_anon_page(page, vmaddr);
}
+
+#define ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE 1
static inline void flush_kernel_vmap_range(void *addr, int size)
{
__flush_wback_region(addr, size);
@@ -72,12 +74,6 @@ static inline void invalidate_kernel_vma
__flush_invalidate_region(addr, size);
}
-#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
-static inline void flush_kernel_dcache_page(struct page *page)
-{
- flush_dcache_page(page);
-}
-
extern void copy_to_user_page(struct vm_area_struct *vma,
struct page *page, unsigned long vaddr, void *dst, const void *src,
unsigned long len);
--- a/block/blk-map.c~mm-remove-flush_kernel_dcache_page
+++ a/block/blk-map.c
@@ -309,7 +309,7 @@ static int bio_map_user_iov(struct reque
static void bio_invalidate_vmalloc_pages(struct bio *bio)
{
-#ifdef ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
+#ifdef ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE
if (bio->bi_private && !op_is_write(bio_op(bio))) {
unsigned long i, len = 0;
--- a/Documentation/core-api/cachetlb.rst~mm-remove-flush_kernel_dcache_page
+++ a/Documentation/core-api/cachetlb.rst
@@ -271,10 +271,15 @@ maps this page at its virtual address.
``void flush_dcache_page(struct page *page)``
- Any time the kernel writes to a page cache page, _OR_
- the kernel is about to read from a page cache page and
- user space shared/writable mappings of this page potentially
- exist, this routine is called.
+ This routines must be called when:
+
+ a) the kernel did write to a page that is in the page cache page
+ and / or in high memory
+ b) the kernel is about to read from a page cache page and user space
+ shared/writable mappings of this page potentially exist. Note
+ that {get,pin}_user_pages{_fast} already call flush_dcache_page
+ on any page found in the user address space and thus driver
+ code rarely needs to take this into account.
.. note::
@@ -284,38 +289,34 @@ maps this page at its virtual address.
handling vfs symlinks in the page cache need not call
this interface at all.
- The phrase "kernel writes to a page cache page" means,
- specifically, that the kernel executes store instructions
- that dirty data in that page at the page->virtual mapping
- of that page. It is important to flush here to handle
- D-cache aliasing, to make sure these kernel stores are
- visible to user space mappings of that page.
-
- The corollary case is just as important, if there are users
- which have shared+writable mappings of this file, we must make
- sure that kernel reads of these pages will see the most recent
- stores done by the user.
-
- If D-cache aliasing is not an issue, this routine may
- simply be defined as a nop on that architecture.
-
- There is a bit set aside in page->flags (PG_arch_1) as
- "architecture private". The kernel guarantees that,
- for pagecache pages, it will clear this bit when such
- a page first enters the pagecache.
-
- This allows these interfaces to be implemented much more
- efficiently. It allows one to "defer" (perhaps indefinitely)
- the actual flush if there are currently no user processes
- mapping this page. See sparc64's flush_dcache_page and
- update_mmu_cache implementations for an example of how to go
- about doing this.
-
- The idea is, first at flush_dcache_page() time, if
- page->mapping->i_mmap is an empty tree, just mark the architecture
- private page flag bit. Later, in update_mmu_cache(), a check is
- made of this flag bit, and if set the flush is done and the flag
- bit is cleared.
+ The phrase "kernel writes to a page cache page" means, specifically,
+ that the kernel executes store instructions that dirty data in that
+ page at the page->virtual mapping of that page. It is important to
+ flush here to handle D-cache aliasing, to make sure these kernel stores
+ are visible to user space mappings of that page.
+
+ The corollary case is just as important, if there are users which have
+ shared+writable mappings of this file, we must make sure that kernel
+ reads of these pages will see the most recent stores done by the user.
+
+ If D-cache aliasing is not an issue, this routine may simply be defined
+ as a nop on that architecture.
+
+ There is a bit set aside in page->flags (PG_arch_1) as "architecture
+ private". The kernel guarantees that, for pagecache pages, it will
+ clear this bit when such a page first enters the pagecache.
+
+ This allows these interfaces to be implemented much more efficiently.
+ It allows one to "defer" (perhaps indefinitely) the actual flush if
+ there are currently no user processes mapping this page. See sparc64's
+ flush_dcache_page and update_mmu_cache implementations for an example
+ of how to go about doing this.
+
+ The idea is, first at flush_dcache_page() time, if page_file_mapping()
+ returns a mapping, and mapping_mapped on that mapping returns %false,
+ just mark the architecture private page flag bit. Later, in
+ update_mmu_cache(), a check is made of this flag bit, and if set the
+ flush is done and the flag bit is cleared.
.. important::
@@ -351,19 +352,6 @@ maps this page at its virtual address.
architectures). For incoherent architectures, it should flush
the cache of the page at vmaddr.
- ``void flush_kernel_dcache_page(struct page *page)``
-
- When the kernel needs to modify a user page is has obtained
- with kmap, it calls this function after all modifications are
- complete (but before kunmapping it) to bring the underlying
- page up to date. It is assumed here that the user has no
- incoherent cached copies (i.e. the original page was obtained
- from a mechanism like get_user_pages()). The default
- implementation is a nop and should remain so on all coherent
- architectures. On incoherent architectures, this should flush
- the kernel cache for page (using page_address(page)).
-
-
``void flush_icache_range(unsigned long start, unsigned long end)``
When the kernel stores into addresses that it will execute
--- a/Documentation/translations/zh_CN/core-api/cachetlb.rst~mm-remove-flush_kernel_dcache_page
+++ a/Documentation/translations/zh_CN/core-api/cachetlb.rst
@@ -298,15 +298,6 @@ é®é¢æå¯è½åå¨ï¼å 为å
æ ¸å·²ç
ç¨ãé»è®¤çå®ç°æ¯nopï¼å¯¹äºææç¸å¹²çæ¶æåºè¯¥ä¿æè¿æ ·ï¼ã对äºä¸ä¸è´æ§
çæ¶æï¼å®åºè¯¥å·æ°vmaddrå¤ç页é¢ç¼åã
- ``void flush_kernel_dcache_page(struct page *page)``
-
- å½å
æ ¸éè¦ä¿®æ¹ä¸ä¸ªç¨kmapè·å¾çç¨æ·é¡µæ¶ï¼å®ä¼å¨ææä¿®æ¹å®æåï¼ä½å¨
- kunmappingä¹åï¼è°ç¨è¿ä¸ªå½æ°ï¼ä»¥ä½¿åºå±é¡µé¢è¾¾å°ææ°ç¶æãè¿éåå®ç¨
- æ·æ²¡æä¸ä¸è´æ§çç¼åå¯æ¬ï¼å³åå§é¡µé¢æ¯ä»ç±»ä¼¼get_user_pages()çæºå¶
- ä¸è·å¾çï¼ãé»è®¤çå®ç°æ¯ä¸ä¸ªnopï¼å¨ææç¸å¹²çæ¶æä¸é½åºè¯¥å¦æ¤ãå¨ä¸
- ä¸è´æ§çæ¶æä¸ï¼è¿åºè¯¥å·æ°å
æ ¸ç¼åä¸ç页é¢ï¼ä½¿ç¨page_address(page)ï¼ã
-
-
``void flush_icache_range(unsigned long start, unsigned long end)``
å½å
æ ¸åå¨å°å®å°æ§è¡çå°åä¸æ¶ï¼ä¾å¦å¨å 载模åæ¶ï¼ï¼è¿ä¸ªå½æ°è¢«è°ç¨ã
--- a/fs/exec.c~mm-remove-flush_kernel_dcache_page
+++ a/fs/exec.c
@@ -574,7 +574,7 @@ static int copy_strings(int argc, struct
}
if (kmapped_page) {
- flush_kernel_dcache_page(kmapped_page);
+ flush_dcache_page(kmapped_page);
kunmap(kmapped_page);
put_arg_page(kmapped_page);
}
@@ -592,7 +592,7 @@ static int copy_strings(int argc, struct
ret = 0;
out:
if (kmapped_page) {
- flush_kernel_dcache_page(kmapped_page);
+ flush_dcache_page(kmapped_page);
kunmap(kmapped_page);
put_arg_page(kmapped_page);
}
@@ -634,7 +634,7 @@ int copy_string_kernel(const char *arg,
kaddr = kmap_atomic(page);
flush_arg_page(bprm, pos & PAGE_MASK, page);
memcpy(kaddr + offset_in_page(pos), arg, bytes_to_copy);
- flush_kernel_dcache_page(page);
+ flush_dcache_page(page);
kunmap_atomic(kaddr);
put_arg_page(page);
}
--- a/include/linux/highmem.h~mm-remove-flush_kernel_dcache_page
+++ a/include/linux/highmem.h
@@ -130,10 +130,7 @@ static inline void flush_anon_page(struc
}
#endif
-#ifndef ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
-static inline void flush_kernel_dcache_page(struct page *page)
-{
-}
+#ifndef ARCH_IMPLEMENTS_FLUSH_KERNEL_VMAP_RANGE
static inline void flush_kernel_vmap_range(void *vaddr, int size)
{
}
--- a/tools/testing/scatterlist/linux/mm.h~mm-remove-flush_kernel_dcache_page
+++ a/tools/testing/scatterlist/linux/mm.h
@@ -127,7 +127,6 @@ kmalloc_array(unsigned int n, unsigned i
#define kmemleak_free(a)
#define PageSlab(p) (0)
-#define flush_kernel_dcache_page(p)
#define MAX_ERRNO 4095
_
Patches currently in -mm which might be from hch@lst.de are
mm-fix-the-try_to_unmap-prototype-for-config_mmu.patch
mm-call-flush_dcache_page-in-memcpy_to_page-and-memzero_page.patch
mm-use-kmap_local_page-in-memzero_page.patch
mmc-jz4740-remove-the-flush_kernel_dcache_page-call-in-jz4740_mmc_read_data.patch
mmc-mmc_spi-replace-flush_kernel_dcache_page-with-flush_dcache_page.patch
ps3disk-replace-flush_kernel_dcache_page-with-flush_dcache_page.patch
scatterlist-replace-flush_kernel_dcache_page-with-flush_dcache_page.patch
aacraid-remove-an-unused-include.patch
mm-remove-flush_kernel_dcache_page.patch
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2021-07-15 22:34 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-15 22:34 + mm-remove-flush_kernel_dcache_page.patch added to -mm tree akpm
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).