* [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation @ 2019-08-20 12:34 guoren 2019-08-20 12:34 ` [PATCH 2/3] csky: Fixup defer cache flush for 610 guoren ` (2 more replies) 0 siblings, 3 replies; 7+ messages in thread From: guoren @ 2019-08-20 12:34 UTC (permalink / raw) To: arnd; +Cc: linux-kernel, linux-arch, linux-csky, douzhk, Guo Ren From: Guo Ren <ren_guo@c-sky.com> Current arch_get_unmapped_area() of abiv1 doesn't use standard kernel api. After referring to the implementation of arch/arm, we implement it with vm_unmapped_area() from linux/mm.h. Signed-off-by: Guo Ren <ren_guo@c-sky.com> Cc: Arnd Bergmann <arnd@arndb.de> --- arch/csky/abiv1/inc/abi/page.h | 5 +-- arch/csky/abiv1/mmap.c | 75 ++++++++++++++++++++++-------------------- 2 files changed, 43 insertions(+), 37 deletions(-) diff --git a/arch/csky/abiv1/inc/abi/page.h b/arch/csky/abiv1/inc/abi/page.h index 6336e92..c864519 100644 --- a/arch/csky/abiv1/inc/abi/page.h +++ b/arch/csky/abiv1/inc/abi/page.h @@ -1,13 +1,14 @@ /* SPDX-License-Identifier: GPL-2.0 */ // Copyright (C) 2018 Hangzhou C-SKY Microsystems co.,ltd. -extern unsigned long shm_align_mask; +#include <asm/shmparam.h> + extern void flush_dcache_page(struct page *page); static inline unsigned long pages_do_alias(unsigned long addr1, unsigned long addr2) { - return (addr1 ^ addr2) & shm_align_mask; + return (addr1 ^ addr2) & (SHMLBA-1); } static inline void clear_user_page(void *addr, unsigned long vaddr, diff --git a/arch/csky/abiv1/mmap.c b/arch/csky/abiv1/mmap.c index b462fd5..6792aca 100644 --- a/arch/csky/abiv1/mmap.c +++ b/arch/csky/abiv1/mmap.c @@ -9,58 +9,63 @@ #include <linux/random.h> #include <linux/io.h> -unsigned long shm_align_mask = (0x4000 >> 1) - 1; /* Sane caches */ +#define COLOUR_ALIGN(addr,pgoff) \ + ((((addr)+SHMLBA-1)&~(SHMLBA-1)) + \ + (((pgoff)<<PAGE_SHIFT) & (SHMLBA-1))) -#define COLOUR_ALIGN(addr, pgoff) \ - ((((addr) + shm_align_mask) & ~shm_align_mask) + \ - (((pgoff) << PAGE_SHIFT) & shm_align_mask)) - -unsigned long arch_get_unmapped_area(struct file *filp, unsigned long addr, +/* + * We need to ensure that shared mappings are correctly aligned to + * avoid aliasing issues with VIPT caches. We need to ensure that + * a specific page of an object is always mapped at a multiple of + * SHMLBA bytes. + * + * We unconditionally provide this function for all cases. + */ +unsigned long +arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { - struct vm_area_struct *vmm; - int do_color_align; + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma; + int do_align = 0; + struct vm_unmapped_area_info info; + + /* + * We only need to do colour alignment if either the I or D + * caches alias. + */ + do_align = filp || (flags & MAP_SHARED); + /* + * We enforce the MAP_FIXED case. + */ if (flags & MAP_FIXED) { - /* - * We do not accept a shared mapping if it would violate - * cache aliasing constraints. - */ - if ((flags & MAP_SHARED) && - ((addr - (pgoff << PAGE_SHIFT)) & shm_align_mask)) + if (flags & MAP_SHARED && + (addr - (pgoff << PAGE_SHIFT)) & (SHMLBA - 1)) return -EINVAL; return addr; } if (len > TASK_SIZE) return -ENOMEM; - do_color_align = 0; - if (filp || (flags & MAP_SHARED)) - do_color_align = 1; + if (addr) { - if (do_color_align) + if (do_align) addr = COLOUR_ALIGN(addr, pgoff); else addr = PAGE_ALIGN(addr); - vmm = find_vma(current->mm, addr); + + vma = find_vma(mm, addr); if (TASK_SIZE - len >= addr && - (!vmm || addr + len <= vmm->vm_start)) + (!vma || addr + len <= vm_start_gap(vma))) return addr; } - addr = TASK_UNMAPPED_BASE; - if (do_color_align) - addr = COLOUR_ALIGN(addr, pgoff); - else - addr = PAGE_ALIGN(addr); - for (vmm = find_vma(current->mm, addr); ; vmm = vmm->vm_next) { - /* At this point: (!vmm || addr < vmm->vm_end). */ - if (TASK_SIZE - len < addr) - return -ENOMEM; - if (!vmm || addr + len <= vmm->vm_start) - return addr; - addr = vmm->vm_end; - if (do_color_align) - addr = COLOUR_ALIGN(addr, pgoff); - } + info.flags = 0; + info.length = len; + info.low_limit = mm->mmap_base; + info.high_limit = TASK_SIZE; + info.align_mask = do_align ? (PAGE_MASK & (SHMLBA - 1)) : 0; + info.align_offset = pgoff << PAGE_SHIFT; + return vm_unmapped_area(&info); } -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/3] csky: Fixup defer cache flush for 610 2019-08-20 12:34 [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation guoren @ 2019-08-20 12:34 ` guoren 2019-08-20 12:34 ` [PATCH 3/3] csky: Support kernel non-aligned access guoren 2019-08-21 2:16 ` [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation Christoph Hellwig 2 siblings, 0 replies; 7+ messages in thread From: guoren @ 2019-08-20 12:34 UTC (permalink / raw) To: arnd; +Cc: linux-kernel, linux-arch, linux-csky, douzhk, Guo Ren From: Guo Ren <ren_guo@c-sky.com> We use defer cache flush mechanism to improve the performance of 610, but the implementation is wrong. We fix it up now and update the mechanism: - Zero page needn't be flushed. - If page is file mapping & non-touched in user space, defer flush. - If page is anon mapping or dirty file mapping, flush immediately. - In update_mmu_cache finish the defer flush by flush_dcache_page(). For 610 we need take care the dcache aliasing issue: - VIPT cache with 8K-bytes size per way in 4K page granularity. Signed-off-by: Guo Ren <ren_guo@c-sky.com> Cc: Arnd Bergmann <arnd@arndb.de> --- arch/csky/abiv1/cacheflush.c | 50 +++++++++++++++++++----------------- arch/csky/abiv1/inc/abi/cacheflush.h | 4 +-- 2 files changed, 29 insertions(+), 25 deletions(-) diff --git a/arch/csky/abiv1/cacheflush.c b/arch/csky/abiv1/cacheflush.c index 10af8b6..fee99fc 100644 --- a/arch/csky/abiv1/cacheflush.c +++ b/arch/csky/abiv1/cacheflush.c @@ -11,42 +11,46 @@ #include <asm/cacheflush.h> #include <asm/cachectl.h> +#define PG_dcache_clean PG_arch_1 + void flush_dcache_page(struct page *page) { - struct address_space *mapping = page_mapping(page); - unsigned long addr; + struct address_space *mapping; - if (mapping && !mapping_mapped(mapping)) { - set_bit(PG_arch_1, &(page)->flags); + if (page == ZERO_PAGE(0)) return; - } - /* - * We could delay the flush for the !page_mapping case too. But that - * case is for exec env/arg pages and those are %99 certainly going to - * get faulted into the tlb (and thus flushed) anyways. - */ - addr = (unsigned long) page_address(page); - dcache_wb_range(addr, addr + PAGE_SIZE); + mapping = page_mapping_file(page); + + if (mapping && !page_mapcount(page)) + clear_bit(PG_dcache_clean, &page->flags); + else { + dcache_wbinv_all(); + if (mapping) + icache_inv_all(); + set_bit(PG_dcache_clean, &page->flags); + } } +EXPORT_SYMBOL(flush_dcache_page); -void update_mmu_cache(struct vm_area_struct *vma, unsigned long address, - pte_t *pte) +void update_mmu_cache(struct vm_area_struct *vma, unsigned long addr, + pte_t *ptep) { - unsigned long addr; + unsigned long pfn = pte_pfn(*ptep); struct page *page; - unsigned long pfn; - pfn = pte_pfn(*pte); - if (unlikely(!pfn_valid(pfn))) + if (!pfn_valid(pfn)) return; page = pfn_to_page(pfn); - addr = (unsigned long) page_address(page); + if (page == ZERO_PAGE(0)) + return; - if (vma->vm_flags & VM_EXEC || - pages_do_alias(addr, address & PAGE_MASK)) - cache_wbinv_all(); + if (!test_and_set_bit(PG_dcache_clean, &page->flags)) + dcache_wbinv_all(); - clear_bit(PG_arch_1, &(page)->flags); + if (page_mapping_file(page)) { + if (vma->vm_flags & VM_EXEC) + icache_inv_all(); + } } diff --git a/arch/csky/abiv1/inc/abi/cacheflush.h b/arch/csky/abiv1/inc/abi/cacheflush.h index 5f663ae..fce5604 100644 --- a/arch/csky/abiv1/inc/abi/cacheflush.h +++ b/arch/csky/abiv1/inc/abi/cacheflush.h @@ -26,8 +26,8 @@ extern void flush_dcache_page(struct page *); #define flush_icache_page(vma, page) cache_wbinv_all() #define flush_icache_range(start, end) cache_wbinv_range(start, end) -#define flush_icache_user_range(vma, pg, adr, len) \ - cache_wbinv_range(adr, adr + len) +#define flush_icache_user_range(vma,page,addr,len) \ + flush_dcache_page(page) #define copy_from_user_page(vma, page, vaddr, dst, src, len) \ do { \ -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/3] csky: Support kernel non-aligned access 2019-08-20 12:34 [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation guoren 2019-08-20 12:34 ` [PATCH 2/3] csky: Fixup defer cache flush for 610 guoren @ 2019-08-20 12:34 ` guoren 2019-08-21 2:17 ` Christoph Hellwig 2019-08-21 2:16 ` [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation Christoph Hellwig 2 siblings, 1 reply; 7+ messages in thread From: guoren @ 2019-08-20 12:34 UTC (permalink / raw) To: arnd; +Cc: linux-kernel, linux-arch, linux-csky, douzhk, Guo Ren From: Guo Ren <ren_guo@c-sky.com> We prohibit non-aligned access in kernel mode, but some special NIC driver needs to support kernel-state unaligned access. For example, when the bus does not support unaligned access, IP header parsing will cause non-aligned access and driver does not recopy the skb buffer to dma for performance reasons. Added kernel_enable & user_enable to control unaligned access and added kernel_count & user_count for statistical unaligned access. Signed-off-by: Guo Ren <ren_guo@c-sky.com> Cc: Arnd Bergmann <arnd@arndb.de> --- arch/csky/abiv1/alignment.c | 62 ++++++++++++++++++++++++++++++++------------- 1 file changed, 45 insertions(+), 17 deletions(-) diff --git a/arch/csky/abiv1/alignment.c b/arch/csky/abiv1/alignment.c index 27ef5b2..cb2a0d9 100644 --- a/arch/csky/abiv1/alignment.c +++ b/arch/csky/abiv1/alignment.c @@ -5,8 +5,10 @@ #include <linux/uaccess.h> #include <linux/ptrace.h> -static int align_enable = 1; -static int align_count; +static int align_kern_enable = 1; +static int align_usr_enable = 1; +static int align_kern_count = 0; +static int align_usr_count = 0; static inline uint32_t get_ptreg(struct pt_regs *regs, uint32_t rx) { @@ -32,9 +34,6 @@ static int ldb_asm(uint32_t addr, uint32_t *valp) uint32_t val; int err; - if (!access_ok((void *)addr, 1)) - return 1; - asm volatile ( "movi %0, 0\n" "1:\n" @@ -67,9 +66,6 @@ static int stb_asm(uint32_t addr, uint32_t val) { int err; - if (!access_ok((void *)addr, 1)) - return 1; - asm volatile ( "movi %0, 0\n" "1:\n" @@ -203,8 +199,6 @@ static int stw_c(struct pt_regs *regs, uint32_t rz, uint32_t addr) if (stb_asm(addr, byte3)) return 1; - align_count++; - return 0; } @@ -226,7 +220,14 @@ void csky_alignment(struct pt_regs *regs) uint32_t addr = 0; if (!user_mode(regs)) + goto kernel_area; + + if (!align_usr_enable) { + pr_err("%s user disabled.\n", __func__); goto bad_area; + } + + align_usr_count++; ret = get_user(tmp, (uint16_t *)instruction_pointer(regs)); if (ret) { @@ -234,6 +235,19 @@ void csky_alignment(struct pt_regs *regs) goto bad_area; } + goto good_area; + +kernel_area: + if (!align_kern_enable) { + pr_err("%s kernel disabled.\n", __func__); + goto bad_area; + } + + align_kern_count++; + + tmp = *(uint16_t *)instruction_pointer(regs); + +good_area: opcode = (uint32_t)tmp; rx = opcode & 0xf; @@ -286,18 +300,32 @@ void csky_alignment(struct pt_regs *regs) force_sig_fault(SIGBUS, BUS_ADRALN, (void __user *)addr); } -static struct ctl_table alignment_tbl[4] = { +static struct ctl_table alignment_tbl[5] = { + { + .procname = "kernel_enable", + .data = &align_kern_enable, + .maxlen = sizeof(align_kern_enable), + .mode = 0666, + .proc_handler = &proc_dointvec + }, + { + .procname = "user_enable", + .data = &align_usr_enable, + .maxlen = sizeof(align_usr_enable), + .mode = 0666, + .proc_handler = &proc_dointvec + }, { - .procname = "enable", - .data = &align_enable, - .maxlen = sizeof(align_enable), + .procname = "kernel_count", + .data = &align_kern_count, + .maxlen = sizeof(align_kern_count), .mode = 0666, .proc_handler = &proc_dointvec }, { - .procname = "count", - .data = &align_count, - .maxlen = sizeof(align_count), + .procname = "user_count", + .data = &align_usr_count, + .maxlen = sizeof(align_usr_count), .mode = 0666, .proc_handler = &proc_dointvec }, -- 2.7.4 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] csky: Support kernel non-aligned access 2019-08-20 12:34 ` [PATCH 3/3] csky: Support kernel non-aligned access guoren @ 2019-08-21 2:17 ` Christoph Hellwig 2019-08-21 3:43 ` Guo Ren 0 siblings, 1 reply; 7+ messages in thread From: Christoph Hellwig @ 2019-08-21 2:17 UTC (permalink / raw) To: guoren; +Cc: arnd, linux-kernel, linux-arch, linux-csky, douzhk, Guo Ren On Tue, Aug 20, 2019 at 08:34:29PM +0800, guoren@kernel.org wrote: > From: Guo Ren <ren_guo@c-sky.com> > > We prohibit non-aligned access in kernel mode, but some special NIC > driver needs to support kernel-state unaligned access. For example, > when the bus does not support unaligned access, IP header parsing > will cause non-aligned access and driver does not recopy the skb > buffer to dma for performance reasons. > > Added kernel_enable & user_enable to control unaligned access and > added kernel_count & user_count for statistical unaligned access. If the NIC drivers requires this it is buggy. Kernel code must use the get_unaligned* / put_unaligned* helpers for that. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 3/3] csky: Support kernel non-aligned access 2019-08-21 2:17 ` Christoph Hellwig @ 2019-08-21 3:43 ` Guo Ren 0 siblings, 0 replies; 7+ messages in thread From: Guo Ren @ 2019-08-21 3:43 UTC (permalink / raw) To: Christoph Hellwig Cc: Arnd Bergmann, Linux Kernel Mailing List, linux-arch, linux-csky, douzhk, Guo Ren Thx Christoph On Wed, Aug 21, 2019 at 10:17 AM Christoph Hellwig <hch@infradead.org> wrote: > > On Tue, Aug 20, 2019 at 08:34:29PM +0800, guoren@kernel.org wrote: > > From: Guo Ren <ren_guo@c-sky.com> > > > > We prohibit non-aligned access in kernel mode, but some special NIC > > driver needs to support kernel-state unaligned access. For example, > > when the bus does not support unaligned access, IP header parsing > > will cause non-aligned access and driver does not recopy the skb > > buffer to dma for performance reasons. > > > > Added kernel_enable & user_enable to control unaligned access and > > added kernel_count & user_count for statistical unaligned access. > > If the NIC drivers requires this it is buggy. Yes, you are right, but I've no control on their non-upstreamed drivers. Every time kernel version updated I need to take care of that issue for them. So just give them a back door in arch/csky and they could disable it by manual. > Kernel code must > use the get_unaligned* / put_unaligned* helpers for that. Most of ethernet drivers use netdev_alloc_skb_ip_align() to let hardware deal with unaligned access, but some NICs couldn't and we may modify kernel's skb_ip_header parsing code with get_unaligned*/put_unaligned* ? -- Best Regards Guo Ren ML: https://lore.kernel.org/linux-csky/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation 2019-08-20 12:34 [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation guoren 2019-08-20 12:34 ` [PATCH 2/3] csky: Fixup defer cache flush for 610 guoren 2019-08-20 12:34 ` [PATCH 3/3] csky: Support kernel non-aligned access guoren @ 2019-08-21 2:16 ` Christoph Hellwig 2019-08-21 3:44 ` Guo Ren 2 siblings, 1 reply; 7+ messages in thread From: Christoph Hellwig @ 2019-08-21 2:16 UTC (permalink / raw) To: guoren; +Cc: arnd, linux-kernel, linux-arch, linux-csky, douzhk, Guo Ren > +/* > + * We need to ensure that shared mappings are correctly aligned to > + * avoid aliasing issues with VIPT caches. We need to ensure that > + * a specific page of an object is always mapped at a multiple of > + * SHMLBA bytes. > + * > + * We unconditionally provide this function for all cases. > + */ On something unrelated: If csky has virtually indexed caches you also need to implement the flush_kernel_vmap_range and invalidate_kernel_vmap_range functions to avoid data corruption when doing I/O on vmalloc/vmap ranges. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation 2019-08-21 2:16 ` [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation Christoph Hellwig @ 2019-08-21 3:44 ` Guo Ren 0 siblings, 0 replies; 7+ messages in thread From: Guo Ren @ 2019-08-21 3:44 UTC (permalink / raw) To: Christoph Hellwig Cc: Arnd Bergmann, Linux Kernel Mailing List, linux-arch, linux-csky, douzhk, Guo Ren Thx Christoph, On Wed, Aug 21, 2019 at 10:16 AM Christoph Hellwig <hch@infradead.org> wrote: > > > +/* > > + * We need to ensure that shared mappings are correctly aligned to > > + * avoid aliasing issues with VIPT caches. We need to ensure that > > + * a specific page of an object is always mapped at a multiple of > > + * SHMLBA bytes. > > + * > > + * We unconditionally provide this function for all cases. > > + */ > > On something unrelated: If csky has virtually indexed caches you also > need to implement the flush_kernel_vmap_range and > invalidate_kernel_vmap_range functions to avoid data corruption when > doing I/O on vmalloc/vmap ranges. I'll give another patch for this issue -- Best Regards Guo Ren ML: https://lore.kernel.org/linux-csky/ ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-08-21 3:44 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-08-20 12:34 [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation guoren 2019-08-20 12:34 ` [PATCH 2/3] csky: Fixup defer cache flush for 610 guoren 2019-08-20 12:34 ` [PATCH 3/3] csky: Support kernel non-aligned access guoren 2019-08-21 2:17 ` Christoph Hellwig 2019-08-21 3:43 ` Guo Ren 2019-08-21 2:16 ` [PATCH 1/3] csky: Fixup arch_get_unmapped_area() implementation Christoph Hellwig 2019-08-21 3:44 ` Guo Ren
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).