From: Marc Gauthier <consult-mg@gstardust.com>
To: linux-riscv@lists.infradead.org
Cc: Marc Gauthier <consult-mg@gstardust.com>,
Palmer Dabbelt <palmer@dabbelt.com>,
Albert Ou <aou@eecs.berkeley.edu>,
Paul Walmsley <paul.walmsley@sifive.com>
Subject: [PATCH v2 1/2] riscv: Align shared mappings to SHMLBA
Date: Tue, 26 Nov 2019 17:44:45 -0500 [thread overview]
Message-ID: <20191126224446.15145-2-consult-mg@gstardust.com> (raw)
In-Reply-To: <20191126224446.15145-1-consult-mg@gstardust.com>
Align shared mappings according to SHMLBA for VIPT cache performance.
arch_get_unmapped_area() and arch_get_unmapped_area_topdown() are
essentially copies of their default implementations in mm/mmap.c,
modified to align the address to SHMLBA for shared mappings, i.e.
where MAP_SHARED is specified or a file pointer is provided.
Allow MAP_FIXED to request unaligned shared mappings. Although this
may potentially reduce performance, very little software does this, as
it is not portable across architectures that enforce alignment.
Signed-off-by: Marc Gauthier <consult-mg@gstardust.com>
---
arch/riscv/include/asm/pgtable.h | 4 ++
arch/riscv/kernel/sys_riscv.c | 113 +++++++++++++++++++++++++++++++
2 files changed, 117 insertions(+)
diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h
index d3221017194d..7d1cc47ac5f9 100644
--- a/arch/riscv/include/asm/pgtable.h
+++ b/arch/riscv/include/asm/pgtable.h
@@ -436,6 +436,10 @@ extern void *dtb_early_va;
extern void setup_bootmem(void);
extern void paging_init(void);
+/* We provide arch_get_unmapped_area to handle VIPT caches efficiently. */
+#define HAVE_ARCH_UNMAPPED_AREA
+#define HAVE_ARCH_UNMAPPED_AREA_TOPDOWN
+
/*
* Task size is 0x4000000000 for RV64 or 0x9fc00000 for RV32.
* Note that PGDIR_SIZE must evenly divide TASK_SIZE.
diff --git a/arch/riscv/kernel/sys_riscv.c b/arch/riscv/kernel/sys_riscv.c
index f3619f59d85c..3e739b30b1f7 100644
--- a/arch/riscv/kernel/sys_riscv.c
+++ b/arch/riscv/kernel/sys_riscv.c
@@ -3,11 +3,15 @@
* Copyright (C) 2012 Regents of the University of California
* Copyright (C) 2014 Darius Rad <darius@bluespec.com>
* Copyright (C) 2017 SiFive
+ * Copyright (C) 2019 Aril Inc
*/
#include <linux/syscalls.h>
#include <asm/unistd.h>
#include <asm/cacheflush.h>
+#include <linux/shm.h>
+#include <linux/mman.h>
+#include <linux/security.h>
static long riscv_sys_mmap(unsigned long addr, unsigned long len,
unsigned long prot, unsigned long flags,
@@ -65,3 +69,112 @@ SYSCALL_DEFINE3(riscv_flush_icache, uintptr_t, start, uintptr_t, end,
return 0;
}
+
+/*
+ * RISC-V requires implementations to function correctly in the presence
+ * of cache aliasing, regardless of page alignment. It says nothing about
+ * performance. To ensure healthy performance with commonly implemented
+ * VIPT caches, the following code avoids most cases of cache aliasing by
+ * aligning shared mappings such that all mappings of a given physical
+ * page of an object are at a multiple of SHMLBA bytes from each other.
+ *
+ * It does not enforce alignment. Using MAP_FIXED to request unaligned
+ * shared mappings is not common, and may perform poorly with VIPT caches.
+ */
+unsigned long
+arch_get_unmapped_area(struct file *filp, unsigned long addr,
+ unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ struct mm_struct *mm = current->mm;
+ struct vm_area_struct *vma, *prev;
+ struct vm_unmapped_area_info info;
+ const unsigned long pgoffset = pgoff << PAGE_SHIFT;
+ int do_align = (filp || (flags & MAP_SHARED));
+
+ if (len > TASK_SIZE - mmap_min_addr)
+ return -ENOMEM;
+
+ if (flags & MAP_FIXED)
+ return addr;
+
+ if (addr) {
+ if (do_align)
+ addr = ALIGN(addr, SHMLBA) + (pgoffset & (SHMLBA - 1));
+ else
+ addr = PAGE_ALIGN(addr);
+ vma = find_vma_prev(mm, addr, &prev);
+ if (TASK_SIZE - len >= addr && addr >= mmap_min_addr &&
+ (!vma || addr + len <= vm_start_gap(vma)) &&
+ (!prev || addr >= vm_end_gap(prev)))
+ return addr;
+ }
+
+ info.flags = 0;
+ info.length = len;
+ info.low_limit = mm->mmap_base;
+ info.high_limit = TASK_SIZE;
+ info.align_mask = do_align ? SHMLBA - 1 : 0;
+ info.align_offset = pgoffset;
+ return vm_unmapped_area(&info);
+}
+
+/*
+ * Similar to arch_get_unmapped_area(), but allocating top-down from below the
+ * stack's low limit (the base).
+ */
+unsigned long
+arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr,
+ unsigned long len, unsigned long pgoff,
+ unsigned long flags)
+{
+ struct vm_area_struct *vma, *prev;
+ struct mm_struct *mm = current->mm;
+ struct vm_unmapped_area_info info;
+ const unsigned long pgoffset = pgoff << PAGE_SHIFT;
+ int do_align = (filp || (flags & MAP_SHARED));
+
+ /* requested length too big for entire address space */
+ if (len > TASK_SIZE - mmap_min_addr)
+ return -ENOMEM;
+
+ if (flags & MAP_FIXED)
+ return addr;
+
+ /* requesting a specific address */
+ if (addr) {
+ if (do_align)
+ addr = ALIGN(addr, SHMLBA) + (pgoffset & (SHMLBA - 1));
+ else
+ addr = PAGE_ALIGN(addr);
+ vma = find_vma_prev(mm, addr, &prev);
+ if (TASK_SIZE - len >= addr && addr >= mmap_min_addr &&
+ (!vma || addr + len <= vm_start_gap(vma)) &&
+ (!prev || addr >= vm_end_gap(prev)))
+ return addr;
+ }
+
+ info.flags = VM_UNMAPPED_AREA_TOPDOWN;
+ info.length = len;
+ info.low_limit = max(PAGE_SIZE, mmap_min_addr);
+ info.high_limit = mm->mmap_base;
+ info.align_mask = do_align ? SHMLBA - 1 : 0;
+ info.align_offset = pgoffset;
+ addr = vm_unmapped_area(&info);
+
+ /*
+ * A failed mmap() very likely causes application failure,
+ * so fall back to the bottom-up function here. This scenario
+ * can happen with large stack limits and large mmap()
+ * allocations.
+ */
+ if (offset_in_page(addr)) {
+ VM_BUG_ON(addr != -ENOMEM);
+ info.flags = 0;
+ info.low_limit = TASK_UNMAPPED_BASE;
+ info.high_limit = TASK_SIZE;
+ addr = vm_unmapped_area(&info);
+ }
+
+ return addr;
+}
--
2.17.1
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2019-11-26 22:45 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-26 22:44 [PATCH v2 0/2] riscv: Align shared mappings to avoid cache aliasing Marc Gauthier
2019-11-26 22:44 ` Marc Gauthier [this message]
2019-12-05 23:03 ` [PATCH v2 1/2] riscv: Align shared mappings to SHMLBA Palmer Dabbelt
2019-12-06 0:24 ` Marc Gauthier
2019-11-26 22:44 ` [PATCH v2 2/2] riscv: Set SHMLBA according to cache geometry Marc Gauthier
2019-12-05 23:03 ` Palmer Dabbelt
2019-12-05 23:58 ` Marc Gauthier
2019-12-06 0:07 ` Palmer Dabbelt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191126224446.15145-2-consult-mg@gstardust.com \
--to=consult-mg@gstardust.com \
--cc=aou@eecs.berkeley.edu \
--cc=linux-riscv@lists.infradead.org \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).