From: Robin Murphy <robin.murphy@arm.com> To: will.deacon@arm.com, catalin.marinas@arm.com Cc: mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org, linux-nvdimm@lists.01.org Subject: [PATCH 2/6] arm64: Convert __inval_cache_range() to area-based Date: Tue, 25 Jul 2017 11:55:39 +0100 [thread overview] Message-ID: <d8ee26edfc114ba4d93b042a22b159ba90936b6c.1500397441.git.robin.murphy@arm.com> (raw) In-Reply-To: <cover.1500397441.git.robin.murphy@arm.com> __inval_cache_range() is already the odd one out among our data cache maintenance routines as the only remaining range-based one; as we're going to want an invalidation routine to call from C code for the pmem API, let's tweak the prototype and name to bring it in line with the clean operations, and to make its relationship with __dma_inv_area() neatly mirror that of __clean_dcache_area_poc() and __dma_clean_area(). The loop clearing the early page tables gets mildly massaged in the process for the sake of consistency. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- arch/arm64/include/asm/cacheflush.h | 1 + arch/arm64/kernel/head.S | 18 +++++++++--------- arch/arm64/mm/cache.S | 23 ++++++++++++++--------- 3 files changed, 24 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 4d4f650c290e..b4b43a94dffd 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -67,6 +67,7 @@ */ extern void flush_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); +extern void __inval_dcache_area(void *addr, size_t len); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 973df7de7bf8..73a0531e0187 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -143,8 +143,8 @@ preserve_boot_args: dmb sy // needed before dc ivac with // MMU off - add x1, x0, #0x20 // 4 x 8 bytes - b __inval_cache_range // tail call + mov x1, #0x20 // 4 x 8 bytes + b __inval_dcache_area // tail call ENDPROC(preserve_boot_args) /* @@ -221,20 +221,20 @@ __create_page_tables: * dirty cache lines being evicted. */ adrp x0, idmap_pg_dir - adrp x1, swapper_pg_dir + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE - bl __inval_cache_range + ldr x1, =(IDMAP_DIR_SIZE + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + bl __inval_dcache_area /* * Clear the idmap and swapper page tables. */ adrp x0, idmap_pg_dir - adrp x6, swapper_pg_dir + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE + ldr x1, =(IDMAP_DIR_SIZE + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) 1: stp xzr, xzr, [x0], #16 stp xzr, xzr, [x0], #16 stp xzr, xzr, [x0], #16 stp xzr, xzr, [x0], #16 - cmp x0, x6 - b.lo 1b + subs x1, x1, #64 + b.ne 1b mov x7, SWAPPER_MM_MMUFLAGS @@ -307,9 +307,9 @@ __create_page_tables: * tables again to remove any speculatively loaded cache lines. */ adrp x0, idmap_pg_dir - adrp x1, swapper_pg_dir + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE + ldr x1, =(IDMAP_DIR_SIZE + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) dmb sy - bl __inval_cache_range + bl __inval_dcache_area ret x28 ENDPROC(__create_page_tables) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 83c27b6e6dca..ed47fbbb4b05 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -109,20 +109,25 @@ ENTRY(__clean_dcache_area_pou) ENDPROC(__clean_dcache_area_pou) /* + * __inval_dcache_area(kaddr, size) + * + * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * are invalidated. Any partial lines at the ends of the interval are + * also cleaned to PoC to prevent data loss. + * + * - kaddr - kernel address + * - size - size in question + */ +ENTRY(__inval_dcache_area) + /* FALLTHROUGH */ + +/* * __dma_inv_area(start, size) * - start - virtual start address of region * - size - size in question */ __dma_inv_area: add x1, x1, x0 - /* FALLTHROUGH */ - -/* - * __inval_cache_range(start, end) - * - start - start address of region - * - end - end address of region - */ -ENTRY(__inval_cache_range) dcache_line_size x2, x3 sub x3, x2, #1 tst x1, x3 // end cache line aligned? @@ -140,7 +145,7 @@ ENTRY(__inval_cache_range) b.lo 2b dsb sy ret -ENDPIPROC(__inval_cache_range) +ENDPIPROC(__inval_dcache_area) ENDPROC(__dma_inv_area) /* -- 2.12.2.dirty _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm
WARNING: multiple messages have this Message-ID (diff)
From: robin.murphy@arm.com (Robin Murphy) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 2/6] arm64: Convert __inval_cache_range() to area-based Date: Tue, 25 Jul 2017 11:55:39 +0100 [thread overview] Message-ID: <d8ee26edfc114ba4d93b042a22b159ba90936b6c.1500397441.git.robin.murphy@arm.com> (raw) In-Reply-To: <cover.1500397441.git.robin.murphy@arm.com> __inval_cache_range() is already the odd one out among our data cache maintenance routines as the only remaining range-based one; as we're going to want an invalidation routine to call from C code for the pmem API, let's tweak the prototype and name to bring it in line with the clean operations, and to make its relationship with __dma_inv_area() neatly mirror that of __clean_dcache_area_poc() and __dma_clean_area(). The loop clearing the early page tables gets mildly massaged in the process for the sake of consistency. Signed-off-by: Robin Murphy <robin.murphy@arm.com> --- arch/arm64/include/asm/cacheflush.h | 1 + arch/arm64/kernel/head.S | 18 +++++++++--------- arch/arm64/mm/cache.S | 23 ++++++++++++++--------- 3 files changed, 24 insertions(+), 18 deletions(-) diff --git a/arch/arm64/include/asm/cacheflush.h b/arch/arm64/include/asm/cacheflush.h index 4d4f650c290e..b4b43a94dffd 100644 --- a/arch/arm64/include/asm/cacheflush.h +++ b/arch/arm64/include/asm/cacheflush.h @@ -67,6 +67,7 @@ */ extern void flush_icache_range(unsigned long start, unsigned long end); extern void __flush_dcache_area(void *addr, size_t len); +extern void __inval_dcache_area(void *addr, size_t len); extern void __clean_dcache_area_poc(void *addr, size_t len); extern void __clean_dcache_area_pou(void *addr, size_t len); extern long __flush_cache_user_range(unsigned long start, unsigned long end); diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 973df7de7bf8..73a0531e0187 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -143,8 +143,8 @@ preserve_boot_args: dmb sy // needed before dc ivac with // MMU off - add x1, x0, #0x20 // 4 x 8 bytes - b __inval_cache_range // tail call + mov x1, #0x20 // 4 x 8 bytes + b __inval_dcache_area // tail call ENDPROC(preserve_boot_args) /* @@ -221,20 +221,20 @@ __create_page_tables: * dirty cache lines being evicted. */ adrp x0, idmap_pg_dir - adrp x1, swapper_pg_dir + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE - bl __inval_cache_range + ldr x1, =(IDMAP_DIR_SIZE + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) + bl __inval_dcache_area /* * Clear the idmap and swapper page tables. */ adrp x0, idmap_pg_dir - adrp x6, swapper_pg_dir + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE + ldr x1, =(IDMAP_DIR_SIZE + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) 1: stp xzr, xzr, [x0], #16 stp xzr, xzr, [x0], #16 stp xzr, xzr, [x0], #16 stp xzr, xzr, [x0], #16 - cmp x0, x6 - b.lo 1b + subs x1, x1, #64 + b.ne 1b mov x7, SWAPPER_MM_MMUFLAGS @@ -307,9 +307,9 @@ __create_page_tables: * tables again to remove any speculatively loaded cache lines. */ adrp x0, idmap_pg_dir - adrp x1, swapper_pg_dir + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE + ldr x1, =(IDMAP_DIR_SIZE + SWAPPER_DIR_SIZE + RESERVED_TTBR0_SIZE) dmb sy - bl __inval_cache_range + bl __inval_dcache_area ret x28 ENDPROC(__create_page_tables) diff --git a/arch/arm64/mm/cache.S b/arch/arm64/mm/cache.S index 83c27b6e6dca..ed47fbbb4b05 100644 --- a/arch/arm64/mm/cache.S +++ b/arch/arm64/mm/cache.S @@ -109,20 +109,25 @@ ENTRY(__clean_dcache_area_pou) ENDPROC(__clean_dcache_area_pou) /* + * __inval_dcache_area(kaddr, size) + * + * Ensure that any D-cache lines for the interval [kaddr, kaddr+size) + * are invalidated. Any partial lines at the ends of the interval are + * also cleaned to PoC to prevent data loss. + * + * - kaddr - kernel address + * - size - size in question + */ +ENTRY(__inval_dcache_area) + /* FALLTHROUGH */ + +/* * __dma_inv_area(start, size) * - start - virtual start address of region * - size - size in question */ __dma_inv_area: add x1, x1, x0 - /* FALLTHROUGH */ - -/* - * __inval_cache_range(start, end) - * - start - start address of region - * - end - end address of region - */ -ENTRY(__inval_cache_range) dcache_line_size x2, x3 sub x3, x2, #1 tst x1, x3 // end cache line aligned? @@ -140,7 +145,7 @@ ENTRY(__inval_cache_range) b.lo 2b dsb sy ret -ENDPIPROC(__inval_cache_range) +ENDPIPROC(__inval_dcache_area) ENDPROC(__dma_inv_area) /* -- 2.12.2.dirty
next prev parent reply other threads:[~2017-07-25 10:54 UTC|newest] Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-07-25 10:55 [PATCH 0/6] arm64 pmem support Robin Murphy 2017-07-25 10:55 ` Robin Murphy 2017-07-25 10:55 ` [PATCH 1/6] arm64: mm: Fix set_memory_valid() declaration Robin Murphy 2017-07-25 10:55 ` Robin Murphy 2017-07-25 10:55 ` Robin Murphy [this message] 2017-07-25 10:55 ` [PATCH 2/6] arm64: Convert __inval_cache_range() to area-based Robin Murphy 2017-07-25 10:55 ` [PATCH 3/6] arm64: Expose DC CVAP to userspace Robin Murphy 2017-07-25 10:55 ` Robin Murphy 2017-07-25 10:55 ` [PATCH 4/6] arm64: Handle trapped DC CVAP Robin Murphy 2017-07-25 10:55 ` Robin Murphy 2017-07-25 10:55 ` [PATCH 5/6] arm64: Implement pmem API support Robin Murphy 2017-07-25 10:55 ` Robin Murphy 2017-08-04 15:25 ` Catalin Marinas 2017-08-04 15:25 ` Catalin Marinas 2017-08-04 17:43 ` Robin Murphy 2017-08-04 17:43 ` Robin Murphy 2017-08-04 18:09 ` Dan Williams 2017-08-04 18:09 ` Dan Williams 2017-08-04 18:35 ` Robin Murphy 2017-08-04 18:35 ` Robin Murphy 2017-08-04 19:36 ` Dan Williams 2017-08-04 19:36 ` Dan Williams 2017-08-07 18:33 ` Will Deacon 2017-08-07 18:33 ` Will Deacon 2017-07-25 10:55 ` [PATCH 6/6] arm64: uaccess: Implement *_flushcache variants Robin Murphy 2017-07-25 10:55 ` Robin Murphy 2017-08-07 18:32 ` Will Deacon 2017-08-07 18:32 ` Will Deacon 2017-08-10 10:58 ` Arnd Bergmann 2017-08-10 10:58 ` Arnd Bergmann 2017-08-10 14:12 ` Catalin Marinas 2017-08-10 14:12 ` Catalin Marinas 2017-08-07 18:34 ` [PATCH 0/6] arm64 pmem support Will Deacon 2017-08-07 18:34 ` Will Deacon
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=d8ee26edfc114ba4d93b042a22b159ba90936b6c.1500397441.git.robin.murphy@arm.com \ --to=robin.murphy@arm.com \ --cc=catalin.marinas@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-nvdimm@lists.01.org \ --cc=mark.rutland@arm.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.