* [PATCH V2 1/2] ARM: Handle user space mapped pages in flush_kernel_dcache_page
2012-09-26 21:40 [PATCH 0/2] fix and improvement of flush(_kernel)_dcache_page() Simon Baatz
@ 2012-09-26 21:40 ` Simon Baatz
2012-09-26 21:40 ` [PATCH 2/2] ARM: remove unnecessary flush of anon pages in flush(_kernel)_dcache_page() Simon Baatz
1 sibling, 0 replies; 5+ messages in thread
From: Simon Baatz @ 2012-09-26 21:40 UTC (permalink / raw)
To: linux-arm-kernel
Commit f8b63c1 made flush_kernel_dcache_page() a no-op assuming that
the pages it needs to handle are kernel mapped only. However, for
example when doing direct I/O, pages with user space mappings may
occur.
Thus, do lazy flushing like in flush_dcache_page() if there are no user
space mappings. Otherwise, flush the kernel cache lines directly.
Signed-off-by: Simon Baatz <gmbnomis@gmail.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Russell King <linux@arm.linux.org.uk>
---
Changes in v2:
- flush_kernel_dcache_page() follows flush_dcache_page() now, except that it
does not flush the user mappings
arch/arm/include/asm/cacheflush.h | 4 ++++
arch/arm/mm/flush.c | 39 +++++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/arch/arm/include/asm/cacheflush.h b/arch/arm/include/asm/cacheflush.h
index e4448e1..eca955f 100644
--- a/arch/arm/include/asm/cacheflush.h
+++ b/arch/arm/include/asm/cacheflush.h
@@ -307,6 +307,10 @@ static inline void flush_anon_page(struct vm_area_struct *vma,
#define ARCH_HAS_FLUSH_KERNEL_DCACHE_PAGE
static inline void flush_kernel_dcache_page(struct page *page)
{
+ extern void __flush_kernel_dcache_page(struct page *);
+ /* highmem pages are always flushed upon kunmap already */
+ if (!PageHighMem(page))
+ __flush_kernel_dcache_page(page);
}
#define flush_dcache_mmap_lock(mapping) \
diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c
index 40ca11e..982db2f 100644
--- a/arch/arm/mm/flush.c
+++ b/arch/arm/mm/flush.c
@@ -192,6 +192,45 @@ void __flush_dcache_page(struct address_space *mapping, struct page *page)
page->index << PAGE_CACHE_SHIFT);
}
+/*
+ * Ensure cache coherency for the kernel mapping of this page.
+ *
+ * If the page only exists in the page cache and there are no user
+ * space mappings, we can be lazy and remember that we may have dirty
+ * kernel cache lines for later. Otherwise, we need to flush the
+ * dirty kernel cache lines directly.
+ *
+ * Note that we disable the lazy flush for SMP configurations where
+ * the cache maintenance operations are not automatically broadcasted.
+ *
+ * We can assume that the page is no high mem page, see
+ * flush_kernel_dcache_page.
+ */
+void __flush_kernel_dcache_page(struct page *page)
+{
+ struct address_space *mapping;
+
+ /*
+ * The zero page is never written to, so never has any dirty
+ * cache lines, and therefore never needs to be flushed.
+ */
+ if (page == ZERO_PAGE(0))
+ return;
+
+ mapping = page_mapping(page);
+
+ if (!cache_ops_need_broadcast() &&
+ mapping && !mapping_mapped(mapping))
+ clear_bit(PG_dcache_clean, &page->flags);
+ else {
+ __cpuc_flush_dcache_area(page_address(page), PAGE_SIZE);
+ if (mapping && !cache_is_vivt())
+ __flush_icache_all();
+ set_bit(PG_dcache_clean, &page->flags);
+ }
+}
+EXPORT_SYMBOL(__flush_kernel_dcache_page);
+
static void __flush_dcache_aliases(struct address_space *mapping, struct page *page)
{
struct mm_struct *mm = current->active_mm;
--
1.7.9.5
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/2] ARM: remove unnecessary flush of anon pages in flush(_kernel)_dcache_page()
2012-09-26 21:40 [PATCH 0/2] fix and improvement of flush(_kernel)_dcache_page() Simon Baatz
2012-09-26 21:40 ` [PATCH V2 1/2] ARM: Handle user space mapped pages in flush_kernel_dcache_page Simon Baatz
@ 2012-09-26 21:40 ` Simon Baatz
2012-09-27 12:23 ` Catalin Marinas
1 sibling, 1 reply; 5+ messages in thread
From: Simon Baatz @ 2012-09-26 21:40 UTC (permalink / raw)
To: linux-arm-kernel
On non-aliasing VIPT D-caches, there is no need to flush the kernel
mapping of anon pages in flush_kernel_dcache_page() and
flush_dcache_page() directly. If the page is mapped as executable
later, the necessary D/I-cache flush will be done in
__sync_icache_dcache().
Signed-off-by: Simon Baatz <gmbnomis@gmail.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Russell King <linux@arm.linux.org.uk>
---
arch/arm/mm/flush.c | 60 ++++++++++++++++++++++++++++++---------------------
1 file changed, 35 insertions(+), 25 deletions(-)
diff --git a/arch/arm/mm/flush.c b/arch/arm/mm/flush.c
index 982db2f..077c8fd 100644
--- a/arch/arm/mm/flush.c
+++ b/arch/arm/mm/flush.c
@@ -219,15 +219,18 @@ void __flush_kernel_dcache_page(struct page *page)
mapping = page_mapping(page);
- if (!cache_ops_need_broadcast() &&
- mapping && !mapping_mapped(mapping))
- clear_bit(PG_dcache_clean, &page->flags);
- else {
- __cpuc_flush_dcache_area(page_address(page), PAGE_SIZE);
- if (mapping && !cache_is_vivt())
- __flush_icache_all();
- set_bit(PG_dcache_clean, &page->flags);
+ if (!cache_ops_need_broadcast()) {
+ if ((mapping && !mapping_mapped(mapping)) ||
+ (!mapping && cache_is_vipt_nonaliasing())) {
+ clear_bit(PG_dcache_clean, &page->flags);
+ return;
+ }
}
+
+ __cpuc_flush_dcache_area(page_address(page), PAGE_SIZE);
+ if (mapping && !cache_is_vivt())
+ __flush_icache_all();
+ set_bit(PG_dcache_clean, &page->flags);
}
EXPORT_SYMBOL(__flush_kernel_dcache_page);
@@ -296,16 +299,20 @@ void __sync_icache_dcache(pte_t pteval)
* of this page.
*
* We have three cases to consider:
- * - VIPT non-aliasing cache: fully coherent so nothing required.
+ * - VIPT non-aliasing cache:
+ * D-cache: fully coherent so nothing required.
+ * I-cache: Ensure I/D coherency in case of an already mapped page;
+ * __sync_icache_dcache() will handle the other cases.
+ * - VIPT aliasing:
+ * D-cache: need to handle one alias in our current VM view.
+ * I-cache: same as VIPT non-aliasing cache
* - VIVT: fully aliasing, so we need to handle every alias in our
* current VM view.
- * - VIPT aliasing: need to handle one alias in our current VM view.
*
- * If we need to handle aliasing:
- * If the page only exists in the page cache and there are no user
- * space mappings, we can be lazy and remember that we may have dirty
- * kernel cache lines for later. Otherwise, we assume we have
- * aliasing mappings.
+ * If the page only exists in the page cache and there are no user
+ * space mappings, we can be lazy and remember that we may have dirty
+ * kernel cache lines for later. Otherwise, we assume we have
+ * aliasing mappings.
*
* Note that we disable the lazy flush for SMP configurations where
* the cache maintenance operations are not automatically broadcasted.
@@ -323,17 +330,20 @@ void flush_dcache_page(struct page *page)
mapping = page_mapping(page);
- if (!cache_ops_need_broadcast() &&
- mapping && !mapping_mapped(mapping))
- clear_bit(PG_dcache_clean, &page->flags);
- else {
- __flush_dcache_page(mapping, page);
- if (mapping && cache_is_vivt())
- __flush_dcache_aliases(mapping, page);
- else if (mapping)
- __flush_icache_all();
- set_bit(PG_dcache_clean, &page->flags);
+ if (!cache_ops_need_broadcast()) {
+ if ((mapping && !mapping_mapped(mapping)) ||
+ (!mapping && cache_is_vipt_nonaliasing())) {
+ clear_bit(PG_dcache_clean, &page->flags);
+ return;
+ }
}
+
+ __flush_dcache_page(mapping, page);
+ if (mapping && cache_is_vivt())
+ __flush_dcache_aliases(mapping, page);
+ else if (mapping)
+ __flush_icache_all();
+ set_bit(PG_dcache_clean, &page->flags);
}
EXPORT_SYMBOL(flush_dcache_page);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 5+ messages in thread