* [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 13:17 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
This patchset moves definition of mm_struct into separate header file.
It allows to get rid of nr_pmds if PMD page table level is folded.
We cannot do it with current mm_types.h because we need
__PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
dependencies.
I've done few build tests and looks like it works, but I expect breakage
on some configuration. Please test.
I hope one day we would rely on LTO for inlining instead of this header
mess :-/
Kirill A. Shutemov (4):
mm: move enum tlb_flush_reason into <trace/events/tlb.h>
mm: split up mm_struct to separate header file
mm: define __PAGETABLE_{PMD,PUD}_FOLDED to zero or one
mm: do not add nr_pmds into mm_struct if PMD is folded
arch/arm/include/asm/pgtable-2level.h | 2 +-
arch/arm64/include/asm/kvm_mmu.h | 4 +-
arch/arm64/kernel/efi.c | 1 +
arch/arm64/mm/hugetlbpage.c | 6 +-
arch/c6x/kernel/dma.c | 1 -
arch/microblaze/include/asm/pgtable.h | 2 +-
arch/mips/include/asm/pgalloc.h | 4 +-
arch/mips/kernel/asm-offsets.c | 4 +-
arch/mips/mm/init.c | 2 +-
arch/mips/mm/pgtable-64.c | 6 +-
arch/mips/mm/tlbex.c | 8 +-
arch/powerpc/mm/pgtable_64.c | 2 +-
arch/s390/include/asm/pgtable.h | 1 +
arch/sh/mm/init.c | 2 +-
arch/tile/mm/hugetlbpage.c | 4 +-
arch/tile/mm/pgtable.c | 4 +-
arch/x86/include/asm/mmu_context.h | 1 +
arch/x86/include/asm/pgtable.h | 15 +--
arch/x86/include/asm/xen/page.h | 2 +-
drivers/iommu/amd_iommu_v2.c | 1 +
drivers/staging/android/ion/ion.c | 1 -
include/asm-generic/4level-fixup.h | 2 +-
include/asm-generic/pgtable-nopmd.h | 2 +-
include/asm-generic/pgtable-nopud.h | 2 +-
include/asm-generic/pgtable.h | 8 ++
include/linux/mm.h | 5 +-
include/linux/mm_struct.h | 217 ++++++++++++++++++++++++++++++++
include/linux/mm_types.h | 226 ++--------------------------------
include/linux/mmu_notifier.h | 1 +
include/linux/sched.h | 1 +
include/trace/events/tlb.h | 15 ++-
kernel/fork.c | 2 +-
mm/init-mm.c | 1 +
mm/kmemcheck.c | 1 -
mm/memory.c | 4 +-
35 files changed, 289 insertions(+), 271 deletions(-)
create mode 100644 include/linux/mm_struct.h
--
2.1.4
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 13:17 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
This patchset moves definition of mm_struct into separate header file.
It allows to get rid of nr_pmds if PMD page table level is folded.
We cannot do it with current mm_types.h because we need
__PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
dependencies.
I've done few build tests and looks like it works, but I expect breakage
on some configuration. Please test.
I hope one day we would rely on LTO for inlining instead of this header
mess :-/
Kirill A. Shutemov (4):
mm: move enum tlb_flush_reason into <trace/events/tlb.h>
mm: split up mm_struct to separate header file
mm: define __PAGETABLE_{PMD,PUD}_FOLDED to zero or one
mm: do not add nr_pmds into mm_struct if PMD is folded
arch/arm/include/asm/pgtable-2level.h | 2 +-
arch/arm64/include/asm/kvm_mmu.h | 4 +-
arch/arm64/kernel/efi.c | 1 +
arch/arm64/mm/hugetlbpage.c | 6 +-
arch/c6x/kernel/dma.c | 1 -
arch/microblaze/include/asm/pgtable.h | 2 +-
arch/mips/include/asm/pgalloc.h | 4 +-
arch/mips/kernel/asm-offsets.c | 4 +-
arch/mips/mm/init.c | 2 +-
arch/mips/mm/pgtable-64.c | 6 +-
arch/mips/mm/tlbex.c | 8 +-
arch/powerpc/mm/pgtable_64.c | 2 +-
arch/s390/include/asm/pgtable.h | 1 +
arch/sh/mm/init.c | 2 +-
arch/tile/mm/hugetlbpage.c | 4 +-
arch/tile/mm/pgtable.c | 4 +-
arch/x86/include/asm/mmu_context.h | 1 +
arch/x86/include/asm/pgtable.h | 15 +--
arch/x86/include/asm/xen/page.h | 2 +-
drivers/iommu/amd_iommu_v2.c | 1 +
drivers/staging/android/ion/ion.c | 1 -
include/asm-generic/4level-fixup.h | 2 +-
include/asm-generic/pgtable-nopmd.h | 2 +-
include/asm-generic/pgtable-nopud.h | 2 +-
include/asm-generic/pgtable.h | 8 ++
include/linux/mm.h | 5 +-
include/linux/mm_struct.h | 217 ++++++++++++++++++++++++++++++++
include/linux/mm_types.h | 226 ++--------------------------------
include/linux/mmu_notifier.h | 1 +
include/linux/sched.h | 1 +
include/trace/events/tlb.h | 15 ++-
kernel/fork.c | 2 +-
mm/init-mm.c | 1 +
mm/kmemcheck.c | 1 -
mm/memory.c | 4 +-
35 files changed, 289 insertions(+), 271 deletions(-)
create mode 100644 include/linux/mm_struct.h
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 28+ messages in thread
* [PATCH 1/4] mm: move enum tlb_flush_reason into <trace/events/tlb.h>
2015-01-28 13:17 ` Kirill A. Shutemov
@ 2015-01-28 13:17 ` Kirill A. Shutemov
-1 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
The only user of tlb_flush_reason is trace_tlb_flush*(). There's no
reason to define it in mm_types.h
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
include/linux/mm_types.h | 8 --------
include/trace/events/tlb.h | 15 ++++++++++++---
2 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 199a03aab8dc..5dfdd5ed5254 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -527,14 +527,6 @@ struct vm_special_mapping
struct page **pages;
};
-enum tlb_flush_reason {
- TLB_FLUSH_ON_TASK_SWITCH,
- TLB_REMOTE_SHOOTDOWN,
- TLB_LOCAL_SHOOTDOWN,
- TLB_LOCAL_MM_SHOOTDOWN,
- NR_TLB_FLUSH_REASONS,
-};
-
/*
* A swap entry has to fit into a "unsigned long", as the entry is hidden
* in the "index" field of the swapper address space.
diff --git a/include/trace/events/tlb.h b/include/trace/events/tlb.h
index 13391d288107..1f764ff60cf6 100644
--- a/include/trace/events/tlb.h
+++ b/include/trace/events/tlb.h
@@ -4,9 +4,18 @@
#if !defined(_TRACE_TLB_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_TLB_H
-#include <linux/mm_types.h>
#include <linux/tracepoint.h>
+#ifndef TRACE_HEADER_MULTI_READ
+enum tlb_flush_reason {
+ TLB_FLUSH_ON_TASK_SWITCH,
+ TLB_REMOTE_SHOOTDOWN,
+ TLB_LOCAL_SHOOTDOWN,
+ TLB_LOCAL_MM_SHOOTDOWN,
+ NR_TLB_FLUSH_REASONS,
+};
+#endif
+
#define TLB_FLUSH_REASON \
{ TLB_FLUSH_ON_TASK_SWITCH, "flush on task switch" }, \
{ TLB_REMOTE_SHOOTDOWN, "remote shootdown" }, \
@@ -15,11 +24,11 @@
TRACE_EVENT(tlb_flush,
- TP_PROTO(int reason, unsigned long pages),
+ TP_PROTO(enum tlb_flush_reason reason, unsigned long pages),
TP_ARGS(reason, pages),
TP_STRUCT__entry(
- __field( int, reason)
+ __field(enum tlb_flush_reason, reason)
__field(unsigned long, pages)
),
--
2.1.4
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 1/4] mm: move enum tlb_flush_reason into <trace/events/tlb.h>
@ 2015-01-28 13:17 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
The only user of tlb_flush_reason is trace_tlb_flush*(). There's no
reason to define it in mm_types.h
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
include/linux/mm_types.h | 8 --------
include/trace/events/tlb.h | 15 ++++++++++++---
2 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 199a03aab8dc..5dfdd5ed5254 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -527,14 +527,6 @@ struct vm_special_mapping
struct page **pages;
};
-enum tlb_flush_reason {
- TLB_FLUSH_ON_TASK_SWITCH,
- TLB_REMOTE_SHOOTDOWN,
- TLB_LOCAL_SHOOTDOWN,
- TLB_LOCAL_MM_SHOOTDOWN,
- NR_TLB_FLUSH_REASONS,
-};
-
/*
* A swap entry has to fit into a "unsigned long", as the entry is hidden
* in the "index" field of the swapper address space.
diff --git a/include/trace/events/tlb.h b/include/trace/events/tlb.h
index 13391d288107..1f764ff60cf6 100644
--- a/include/trace/events/tlb.h
+++ b/include/trace/events/tlb.h
@@ -4,9 +4,18 @@
#if !defined(_TRACE_TLB_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_TLB_H
-#include <linux/mm_types.h>
#include <linux/tracepoint.h>
+#ifndef TRACE_HEADER_MULTI_READ
+enum tlb_flush_reason {
+ TLB_FLUSH_ON_TASK_SWITCH,
+ TLB_REMOTE_SHOOTDOWN,
+ TLB_LOCAL_SHOOTDOWN,
+ TLB_LOCAL_MM_SHOOTDOWN,
+ NR_TLB_FLUSH_REASONS,
+};
+#endif
+
#define TLB_FLUSH_REASON \
{ TLB_FLUSH_ON_TASK_SWITCH, "flush on task switch" }, \
{ TLB_REMOTE_SHOOTDOWN, "remote shootdown" }, \
@@ -15,11 +24,11 @@
TRACE_EVENT(tlb_flush,
- TP_PROTO(int reason, unsigned long pages),
+ TP_PROTO(enum tlb_flush_reason reason, unsigned long pages),
TP_ARGS(reason, pages),
TP_STRUCT__entry(
- __field( int, reason)
+ __field(enum tlb_flush_reason, reason)
__field(unsigned long, pages)
),
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 2/4] mm: split up mm_struct to separate header file
2015-01-28 13:17 ` Kirill A. Shutemov
@ 2015-01-28 13:17 ` Kirill A. Shutemov
-1 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
<asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
definition.
This patch move mm_struct definition into separate header file in order
to fix circular header dependencies.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/arm64/kernel/efi.c | 1 +
arch/c6x/kernel/dma.c | 1 -
arch/s390/include/asm/pgtable.h | 1 +
arch/x86/include/asm/mmu_context.h | 1 +
arch/x86/include/asm/pgtable.h | 15 +--
drivers/iommu/amd_iommu_v2.c | 1 +
drivers/staging/android/ion/ion.c | 1 -
include/linux/mm.h | 1 +
include/linux/mm_struct.h | 214 ++++++++++++++++++++++++++++++++++++
include/linux/mm_types.h | 218 ++-----------------------------------
include/linux/mmu_notifier.h | 1 +
include/linux/sched.h | 1 +
mm/init-mm.c | 1 +
mm/kmemcheck.c | 1 -
14 files changed, 232 insertions(+), 226 deletions(-)
create mode 100644 include/linux/mm_struct.h
diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index b42c7b480e1e..fbf0a6d6f691 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -17,6 +17,7 @@
#include <linux/export.h>
#include <linux/memblock.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/bootmem.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
diff --git a/arch/c6x/kernel/dma.c b/arch/c6x/kernel/dma.c
index ab7b12de144d..5a489f1eabbd 100644
--- a/arch/c6x/kernel/dma.c
+++ b/arch/c6x/kernel/dma.c
@@ -9,7 +9,6 @@
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/mm.h>
-#include <linux/mm_types.h>
#include <linux/scatterlist.h>
#include <asm/cacheflush.h>
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index fbb5ee3ae57c..578eb098b1be 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -29,6 +29,7 @@
#ifndef __ASSEMBLY__
#include <linux/sched.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/page-flags.h>
#include <linux/radix-tree.h>
#include <asm/bug.h>
diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
index 4b75d591eb5e..78a87a30ec50 100644
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -4,6 +4,7 @@
#include <asm/desc.h>
#include <linux/atomic.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <trace/events/tlb.h>
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 9d0ade00923e..594d09d07bb4 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -445,18 +445,9 @@ static inline int pte_present(pte_t a)
return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE);
}
-#define pte_accessible pte_accessible
-static inline bool pte_accessible(struct mm_struct *mm, pte_t a)
-{
- if (pte_flags(a) & _PAGE_PRESENT)
- return true;
-
- if ((pte_flags(a) & _PAGE_PROTNONE) &&
- mm_tlb_flush_pending(mm))
- return true;
-
- return false;
-}
+#define pte_accessible(mm, pte) \
+ (pte_flags(pte) & _PAGE_PRESENT) || \
+ ((pte_flags(pte) & _PAGE_PROTNONE) && mm_tlb_flush_pending(mm))
static inline int pte_hidden(pte_t pte)
{
diff --git a/drivers/iommu/amd_iommu_v2.c b/drivers/iommu/amd_iommu_v2.c
index 90f70d0e1141..d49cd95eaf7b 100644
--- a/drivers/iommu/amd_iommu_v2.c
+++ b/drivers/iommu/amd_iommu_v2.c
@@ -19,6 +19,7 @@
#include <linux/mmu_notifier.h>
#include <linux/amd-iommu.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/profile.h>
#include <linux/module.h>
#include <linux/sched.h>
diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c
index b8f1c491553e..1cb3c88f6423 100644
--- a/drivers/staging/android/ion/ion.c
+++ b/drivers/staging/android/ion/ion.c
@@ -27,7 +27,6 @@
#include <linux/miscdevice.h>
#include <linux/export.h>
#include <linux/mm.h>
-#include <linux/mm_types.h>
#include <linux/rbtree.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
diff --git a/include/linux/mm.h b/include/linux/mm.h
index b976d9ffbcd6..543e9723d441 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -14,6 +14,7 @@
#include <linux/atomic.h>
#include <linux/debug_locks.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/range.h>
#include <linux/pfn.h>
#include <linux/bit_spinlock.h>
diff --git a/include/linux/mm_struct.h b/include/linux/mm_struct.h
new file mode 100644
index 000000000000..0a233c232a39
--- /dev/null
+++ b/include/linux/mm_struct.h
@@ -0,0 +1,214 @@
+#ifndef _LINUX_MM_STRUCT_H
+#define _LINUX_MM_STRUCT_H
+
+#include <linux/auxvec.h>
+#include <linux/rbtree.h>
+#include <linux/rwsem.h>
+#include <linux/types.h>
+#include <linux/uprobes.h>
+
+#include <asm/mmu.h>
+
+struct kioctx_table;
+struct vm_area_struct;
+
+#ifndef AT_VECTOR_SIZE_ARCH
+#define AT_VECTOR_SIZE_ARCH 0
+#endif
+#define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1))
+
+enum {
+ MM_FILEPAGES,
+ MM_ANONPAGES,
+ MM_SWAPENTS,
+ NR_MM_COUNTERS
+};
+
+#if NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS && defined(CONFIG_MMU)
+#define SPLIT_RSS_COUNTING
+/* per-thread cached information, */
+struct task_rss_stat {
+ int events; /* for synchronization threshold */
+ int count[NR_MM_COUNTERS];
+};
+#endif /* USE_SPLIT_PTE_PTLOCKS */
+
+struct mm_rss_stat {
+ atomic_long_t count[NR_MM_COUNTERS];
+};
+
+struct mm_struct {
+ struct vm_area_struct *mmap; /* list of VMAs */
+ struct rb_root mm_rb;
+ u32 vmacache_seqnum; /* per-thread vmacache */
+#ifdef CONFIG_MMU
+ unsigned long (*get_unmapped_area) (struct file *filp,
+ unsigned long addr, unsigned long len,
+ unsigned long pgoff, unsigned long flags);
+#endif
+ unsigned long mmap_base; /* base of mmap area */
+ unsigned long mmap_legacy_base; /* base of mmap area in bottom-up allocations */
+ unsigned long task_size; /* size of task vm space */
+ unsigned long highest_vm_end; /* highest vma end address */
+ pgd_t * pgd;
+ atomic_t mm_users; /* How many users with user space? */
+ atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
+ atomic_long_t nr_ptes; /* PTE page table pages */
+ atomic_long_t nr_pmds; /* PMD page table pages */
+ int map_count; /* number of VMAs */
+
+ spinlock_t page_table_lock; /* Protects page tables and some counters */
+ struct rw_semaphore mmap_sem;
+
+ struct list_head mmlist; /* List of maybe swapped mm's. These are globally strung
+ * together off init_mm.mmlist, and are protected
+ * by mmlist_lock
+ */
+
+
+ unsigned long hiwater_rss; /* High-watermark of RSS usage */
+ unsigned long hiwater_vm; /* High-water virtual memory usage */
+
+ unsigned long total_vm; /* Total pages mapped */
+ unsigned long locked_vm; /* Pages that have PG_mlocked set */
+ unsigned long pinned_vm; /* Refcount permanently increased */
+ unsigned long shared_vm; /* Shared pages (files) */
+ unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE */
+ unsigned long stack_vm; /* VM_GROWSUP/DOWN */
+ unsigned long def_flags;
+ unsigned long start_code, end_code, start_data, end_data;
+ unsigned long start_brk, brk, start_stack;
+ unsigned long arg_start, arg_end, env_start, env_end;
+
+ unsigned long saved_auxv[AT_VECTOR_SIZE]; /* for /proc/PID/auxv */
+
+ /*
+ * Special counters, in some configurations protected by the
+ * page_table_lock, in other configurations by being atomic.
+ */
+ struct mm_rss_stat rss_stat;
+
+ struct linux_binfmt *binfmt;
+
+ cpumask_var_t cpu_vm_mask_var;
+
+ /* Architecture-specific MM context */
+ mm_context_t context;
+
+ unsigned long flags; /* Must use atomic bitops to access the bits */
+
+ struct core_state *core_state; /* coredumping support */
+#ifdef CONFIG_AIO
+ spinlock_t ioctx_lock;
+ struct kioctx_table __rcu *ioctx_table;
+#endif
+#ifdef CONFIG_MEMCG
+ /*
+ * "owner" points to a task that is regarded as the canonical
+ * user/owner of this mm. All of the following must be true in
+ * order for it to be changed:
+ *
+ * current == mm->owner
+ * current->mm != mm
+ * new_owner->mm == mm
+ * new_owner->alloc_lock is held
+ */
+ struct task_struct __rcu *owner;
+#endif
+
+ /* store ref to file /proc/<pid>/exe symlink points to */
+ struct file *exe_file;
+#ifdef CONFIG_MMU_NOTIFIER
+ struct mmu_notifier_mm *mmu_notifier_mm;
+#endif
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
+ pgtable_t pmd_huge_pte; /* protected by page_table_lock */
+#endif
+#ifdef CONFIG_CPUMASK_OFFSTACK
+ struct cpumask cpumask_allocation;
+#endif
+#ifdef CONFIG_NUMA_BALANCING
+ /*
+ * numa_next_scan is the next time that the PTEs will be marked
+ * pte_numa. NUMA hinting faults will gather statistics and migrate
+ * pages to new nodes if necessary.
+ */
+ unsigned long numa_next_scan;
+
+ /* Restart point for scanning and setting pte_numa */
+ unsigned long numa_scan_offset;
+
+ /* numa_scan_seq prevents two threads setting pte_numa */
+ int numa_scan_seq;
+#endif
+#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
+ /*
+ * An operation with batched TLB flushing is going on. Anything that
+ * can move process memory needs to flush the TLB when moving a
+ * PROT_NONE or PROT_NUMA mapped page.
+ */
+ bool tlb_flush_pending;
+#endif
+ struct uprobes_state uprobes_state;
+#ifdef CONFIG_X86_INTEL_MPX
+ /* address of the bounds directory */
+ void __user *bd_addr;
+#endif
+};
+
+static inline void mm_init_cpumask(struct mm_struct *mm)
+{
+#ifdef CONFIG_CPUMASK_OFFSTACK
+ mm->cpu_vm_mask_var = &mm->cpumask_allocation;
+#endif
+ cpumask_clear(mm->cpu_vm_mask_var);
+}
+
+/* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
+static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
+{
+ return mm->cpu_vm_mask_var;
+}
+
+#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
+/*
+ * Memory barriers to keep this state in sync are graciously provided by
+ * the page table locks, outside of which no page table modifications happen.
+ * The barriers below prevent the compiler from re-ordering the instructions
+ * around the memory barriers that are already present in the code.
+ */
+static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
+{
+ barrier();
+ return mm->tlb_flush_pending;
+}
+static inline void set_tlb_flush_pending(struct mm_struct *mm)
+{
+ mm->tlb_flush_pending = true;
+
+ /*
+ * Guarantee that the tlb_flush_pending store does not leak into the
+ * critical section updating the page tables
+ */
+ smp_mb__before_spinlock();
+}
+/* Clearing is done after a TLB flush, which also provides a barrier. */
+static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+{
+ barrier();
+ mm->tlb_flush_pending = false;
+}
+#else
+static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
+{
+ return false;
+}
+static inline void set_tlb_flush_pending(struct mm_struct *mm)
+{
+}
+static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+{
+}
+#endif
+
+#endif
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5dfdd5ed5254..80797b1c1e26 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1,24 +1,16 @@
#ifndef _LINUX_MM_TYPES_H
#define _LINUX_MM_TYPES_H
-#include <linux/auxvec.h>
-#include <linux/types.h>
-#include <linux/threads.h>
-#include <linux/list.h>
-#include <linux/spinlock.h>
-#include <linux/rbtree.h>
-#include <linux/rwsem.h>
#include <linux/completion.h>
#include <linux/cpumask.h>
-#include <linux/uprobes.h>
+#include <linux/list.h>
#include <linux/page-flags-layout.h>
-#include <asm/page.h>
-#include <asm/mmu.h>
+#include <linux/rbtree.h>
+#include <linux/spinlock.h>
+#include <linux/threads.h>
+#include <linux/types.h>
-#ifndef AT_VECTOR_SIZE_ARCH
-#define AT_VECTOR_SIZE_ARCH 0
-#endif
-#define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1))
+#include <asm/page.h>
struct address_space;
struct mem_cgroup;
@@ -326,203 +318,7 @@ struct core_state {
struct completion startup;
};
-enum {
- MM_FILEPAGES,
- MM_ANONPAGES,
- MM_SWAPENTS,
- NR_MM_COUNTERS
-};
-
-#if USE_SPLIT_PTE_PTLOCKS && defined(CONFIG_MMU)
-#define SPLIT_RSS_COUNTING
-/* per-thread cached information, */
-struct task_rss_stat {
- int events; /* for synchronization threshold */
- int count[NR_MM_COUNTERS];
-};
-#endif /* USE_SPLIT_PTE_PTLOCKS */
-
-struct mm_rss_stat {
- atomic_long_t count[NR_MM_COUNTERS];
-};
-
-struct kioctx_table;
-struct mm_struct {
- struct vm_area_struct *mmap; /* list of VMAs */
- struct rb_root mm_rb;
- u32 vmacache_seqnum; /* per-thread vmacache */
-#ifdef CONFIG_MMU
- unsigned long (*get_unmapped_area) (struct file *filp,
- unsigned long addr, unsigned long len,
- unsigned long pgoff, unsigned long flags);
-#endif
- unsigned long mmap_base; /* base of mmap area */
- unsigned long mmap_legacy_base; /* base of mmap area in bottom-up allocations */
- unsigned long task_size; /* size of task vm space */
- unsigned long highest_vm_end; /* highest vma end address */
- pgd_t * pgd;
- atomic_t mm_users; /* How many users with user space? */
- atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
- atomic_long_t nr_ptes; /* PTE page table pages */
- atomic_long_t nr_pmds; /* PMD page table pages */
- int map_count; /* number of VMAs */
-
- spinlock_t page_table_lock; /* Protects page tables and some counters */
- struct rw_semaphore mmap_sem;
-
- struct list_head mmlist; /* List of maybe swapped mm's. These are globally strung
- * together off init_mm.mmlist, and are protected
- * by mmlist_lock
- */
-
-
- unsigned long hiwater_rss; /* High-watermark of RSS usage */
- unsigned long hiwater_vm; /* High-water virtual memory usage */
-
- unsigned long total_vm; /* Total pages mapped */
- unsigned long locked_vm; /* Pages that have PG_mlocked set */
- unsigned long pinned_vm; /* Refcount permanently increased */
- unsigned long shared_vm; /* Shared pages (files) */
- unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE */
- unsigned long stack_vm; /* VM_GROWSUP/DOWN */
- unsigned long def_flags;
- unsigned long start_code, end_code, start_data, end_data;
- unsigned long start_brk, brk, start_stack;
- unsigned long arg_start, arg_end, env_start, env_end;
-
- unsigned long saved_auxv[AT_VECTOR_SIZE]; /* for /proc/PID/auxv */
-
- /*
- * Special counters, in some configurations protected by the
- * page_table_lock, in other configurations by being atomic.
- */
- struct mm_rss_stat rss_stat;
-
- struct linux_binfmt *binfmt;
-
- cpumask_var_t cpu_vm_mask_var;
-
- /* Architecture-specific MM context */
- mm_context_t context;
-
- unsigned long flags; /* Must use atomic bitops to access the bits */
-
- struct core_state *core_state; /* coredumping support */
-#ifdef CONFIG_AIO
- spinlock_t ioctx_lock;
- struct kioctx_table __rcu *ioctx_table;
-#endif
-#ifdef CONFIG_MEMCG
- /*
- * "owner" points to a task that is regarded as the canonical
- * user/owner of this mm. All of the following must be true in
- * order for it to be changed:
- *
- * current == mm->owner
- * current->mm != mm
- * new_owner->mm == mm
- * new_owner->alloc_lock is held
- */
- struct task_struct __rcu *owner;
-#endif
-
- /* store ref to file /proc/<pid>/exe symlink points to */
- struct file *exe_file;
-#ifdef CONFIG_MMU_NOTIFIER
- struct mmu_notifier_mm *mmu_notifier_mm;
-#endif
-#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
- pgtable_t pmd_huge_pte; /* protected by page_table_lock */
-#endif
-#ifdef CONFIG_CPUMASK_OFFSTACK
- struct cpumask cpumask_allocation;
-#endif
-#ifdef CONFIG_NUMA_BALANCING
- /*
- * numa_next_scan is the next time that the PTEs will be marked
- * pte_numa. NUMA hinting faults will gather statistics and migrate
- * pages to new nodes if necessary.
- */
- unsigned long numa_next_scan;
-
- /* Restart point for scanning and setting pte_numa */
- unsigned long numa_scan_offset;
-
- /* numa_scan_seq prevents two threads setting pte_numa */
- int numa_scan_seq;
-#endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
- /*
- * An operation with batched TLB flushing is going on. Anything that
- * can move process memory needs to flush the TLB when moving a
- * PROT_NONE or PROT_NUMA mapped page.
- */
- bool tlb_flush_pending;
-#endif
- struct uprobes_state uprobes_state;
-#ifdef CONFIG_X86_INTEL_MPX
- /* address of the bounds directory */
- void __user *bd_addr;
-#endif
-};
-
-static inline void mm_init_cpumask(struct mm_struct *mm)
-{
-#ifdef CONFIG_CPUMASK_OFFSTACK
- mm->cpu_vm_mask_var = &mm->cpumask_allocation;
-#endif
- cpumask_clear(mm->cpu_vm_mask_var);
-}
-
-/* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
-static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
-{
- return mm->cpu_vm_mask_var;
-}
-
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
-/*
- * Memory barriers to keep this state in sync are graciously provided by
- * the page table locks, outside of which no page table modifications happen.
- * The barriers below prevent the compiler from re-ordering the instructions
- * around the memory barriers that are already present in the code.
- */
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
- barrier();
- return mm->tlb_flush_pending;
-}
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
-{
- mm->tlb_flush_pending = true;
-
- /*
- * Guarantee that the tlb_flush_pending store does not leak into the
- * critical section updating the page tables
- */
- smp_mb__before_spinlock();
-}
-/* Clearing is done after a TLB flush, which also provides a barrier. */
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
-{
- barrier();
- mm->tlb_flush_pending = false;
-}
-#else
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
- return false;
-}
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-#endif
-
-struct vm_special_mapping
-{
+struct vm_special_mapping {
const char *name;
struct page **pages;
};
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index 95243d28a0ee..779e32567e6d 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -4,6 +4,7 @@
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/srcu.h>
struct mmu_notifier;
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 22ee0d5d7f8c..99001775ffa0 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -25,6 +25,7 @@ struct sched_param {
#include <linux/errno.h>
#include <linux/nodemask.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/preempt_mask.h>
#include <asm/page.h>
diff --git a/mm/init-mm.c b/mm/init-mm.c
index a56a851908d2..310ae8c7d9c6 100644
--- a/mm/init-mm.c
+++ b/mm/init-mm.c
@@ -1,4 +1,5 @@
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/rbtree.h>
#include <linux/rwsem.h>
#include <linux/spinlock.h>
diff --git a/mm/kmemcheck.c b/mm/kmemcheck.c
index cab58bb592d8..c40d065013d2 100644
--- a/mm/kmemcheck.c
+++ b/mm/kmemcheck.c
@@ -1,5 +1,4 @@
#include <linux/gfp.h>
-#include <linux/mm_types.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include "slab.h"
--
2.1.4
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 2/4] mm: split up mm_struct to separate header file
@ 2015-01-28 13:17 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
<asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
definition.
This patch move mm_struct definition into separate header file in order
to fix circular header dependencies.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/arm64/kernel/efi.c | 1 +
arch/c6x/kernel/dma.c | 1 -
arch/s390/include/asm/pgtable.h | 1 +
arch/x86/include/asm/mmu_context.h | 1 +
arch/x86/include/asm/pgtable.h | 15 +--
drivers/iommu/amd_iommu_v2.c | 1 +
drivers/staging/android/ion/ion.c | 1 -
include/linux/mm.h | 1 +
include/linux/mm_struct.h | 214 ++++++++++++++++++++++++++++++++++++
include/linux/mm_types.h | 218 ++-----------------------------------
include/linux/mmu_notifier.h | 1 +
include/linux/sched.h | 1 +
mm/init-mm.c | 1 +
mm/kmemcheck.c | 1 -
14 files changed, 232 insertions(+), 226 deletions(-)
create mode 100644 include/linux/mm_struct.h
diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index b42c7b480e1e..fbf0a6d6f691 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -17,6 +17,7 @@
#include <linux/export.h>
#include <linux/memblock.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/bootmem.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
diff --git a/arch/c6x/kernel/dma.c b/arch/c6x/kernel/dma.c
index ab7b12de144d..5a489f1eabbd 100644
--- a/arch/c6x/kernel/dma.c
+++ b/arch/c6x/kernel/dma.c
@@ -9,7 +9,6 @@
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/mm.h>
-#include <linux/mm_types.h>
#include <linux/scatterlist.h>
#include <asm/cacheflush.h>
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index fbb5ee3ae57c..578eb098b1be 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -29,6 +29,7 @@
#ifndef __ASSEMBLY__
#include <linux/sched.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/page-flags.h>
#include <linux/radix-tree.h>
#include <asm/bug.h>
diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
index 4b75d591eb5e..78a87a30ec50 100644
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -4,6 +4,7 @@
#include <asm/desc.h>
#include <linux/atomic.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <trace/events/tlb.h>
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 9d0ade00923e..594d09d07bb4 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -445,18 +445,9 @@ static inline int pte_present(pte_t a)
return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE);
}
-#define pte_accessible pte_accessible
-static inline bool pte_accessible(struct mm_struct *mm, pte_t a)
-{
- if (pte_flags(a) & _PAGE_PRESENT)
- return true;
-
- if ((pte_flags(a) & _PAGE_PROTNONE) &&
- mm_tlb_flush_pending(mm))
- return true;
-
- return false;
-}
+#define pte_accessible(mm, pte) \
+ (pte_flags(pte) & _PAGE_PRESENT) || \
+ ((pte_flags(pte) & _PAGE_PROTNONE) && mm_tlb_flush_pending(mm))
static inline int pte_hidden(pte_t pte)
{
diff --git a/drivers/iommu/amd_iommu_v2.c b/drivers/iommu/amd_iommu_v2.c
index 90f70d0e1141..d49cd95eaf7b 100644
--- a/drivers/iommu/amd_iommu_v2.c
+++ b/drivers/iommu/amd_iommu_v2.c
@@ -19,6 +19,7 @@
#include <linux/mmu_notifier.h>
#include <linux/amd-iommu.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/profile.h>
#include <linux/module.h>
#include <linux/sched.h>
diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c
index b8f1c491553e..1cb3c88f6423 100644
--- a/drivers/staging/android/ion/ion.c
+++ b/drivers/staging/android/ion/ion.c
@@ -27,7 +27,6 @@
#include <linux/miscdevice.h>
#include <linux/export.h>
#include <linux/mm.h>
-#include <linux/mm_types.h>
#include <linux/rbtree.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
diff --git a/include/linux/mm.h b/include/linux/mm.h
index b976d9ffbcd6..543e9723d441 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -14,6 +14,7 @@
#include <linux/atomic.h>
#include <linux/debug_locks.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/range.h>
#include <linux/pfn.h>
#include <linux/bit_spinlock.h>
diff --git a/include/linux/mm_struct.h b/include/linux/mm_struct.h
new file mode 100644
index 000000000000..0a233c232a39
--- /dev/null
+++ b/include/linux/mm_struct.h
@@ -0,0 +1,214 @@
+#ifndef _LINUX_MM_STRUCT_H
+#define _LINUX_MM_STRUCT_H
+
+#include <linux/auxvec.h>
+#include <linux/rbtree.h>
+#include <linux/rwsem.h>
+#include <linux/types.h>
+#include <linux/uprobes.h>
+
+#include <asm/mmu.h>
+
+struct kioctx_table;
+struct vm_area_struct;
+
+#ifndef AT_VECTOR_SIZE_ARCH
+#define AT_VECTOR_SIZE_ARCH 0
+#endif
+#define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1))
+
+enum {
+ MM_FILEPAGES,
+ MM_ANONPAGES,
+ MM_SWAPENTS,
+ NR_MM_COUNTERS
+};
+
+#if NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS && defined(CONFIG_MMU)
+#define SPLIT_RSS_COUNTING
+/* per-thread cached information, */
+struct task_rss_stat {
+ int events; /* for synchronization threshold */
+ int count[NR_MM_COUNTERS];
+};
+#endif /* USE_SPLIT_PTE_PTLOCKS */
+
+struct mm_rss_stat {
+ atomic_long_t count[NR_MM_COUNTERS];
+};
+
+struct mm_struct {
+ struct vm_area_struct *mmap; /* list of VMAs */
+ struct rb_root mm_rb;
+ u32 vmacache_seqnum; /* per-thread vmacache */
+#ifdef CONFIG_MMU
+ unsigned long (*get_unmapped_area) (struct file *filp,
+ unsigned long addr, unsigned long len,
+ unsigned long pgoff, unsigned long flags);
+#endif
+ unsigned long mmap_base; /* base of mmap area */
+ unsigned long mmap_legacy_base; /* base of mmap area in bottom-up allocations */
+ unsigned long task_size; /* size of task vm space */
+ unsigned long highest_vm_end; /* highest vma end address */
+ pgd_t * pgd;
+ atomic_t mm_users; /* How many users with user space? */
+ atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
+ atomic_long_t nr_ptes; /* PTE page table pages */
+ atomic_long_t nr_pmds; /* PMD page table pages */
+ int map_count; /* number of VMAs */
+
+ spinlock_t page_table_lock; /* Protects page tables and some counters */
+ struct rw_semaphore mmap_sem;
+
+ struct list_head mmlist; /* List of maybe swapped mm's. These are globally strung
+ * together off init_mm.mmlist, and are protected
+ * by mmlist_lock
+ */
+
+
+ unsigned long hiwater_rss; /* High-watermark of RSS usage */
+ unsigned long hiwater_vm; /* High-water virtual memory usage */
+
+ unsigned long total_vm; /* Total pages mapped */
+ unsigned long locked_vm; /* Pages that have PG_mlocked set */
+ unsigned long pinned_vm; /* Refcount permanently increased */
+ unsigned long shared_vm; /* Shared pages (files) */
+ unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE */
+ unsigned long stack_vm; /* VM_GROWSUP/DOWN */
+ unsigned long def_flags;
+ unsigned long start_code, end_code, start_data, end_data;
+ unsigned long start_brk, brk, start_stack;
+ unsigned long arg_start, arg_end, env_start, env_end;
+
+ unsigned long saved_auxv[AT_VECTOR_SIZE]; /* for /proc/PID/auxv */
+
+ /*
+ * Special counters, in some configurations protected by the
+ * page_table_lock, in other configurations by being atomic.
+ */
+ struct mm_rss_stat rss_stat;
+
+ struct linux_binfmt *binfmt;
+
+ cpumask_var_t cpu_vm_mask_var;
+
+ /* Architecture-specific MM context */
+ mm_context_t context;
+
+ unsigned long flags; /* Must use atomic bitops to access the bits */
+
+ struct core_state *core_state; /* coredumping support */
+#ifdef CONFIG_AIO
+ spinlock_t ioctx_lock;
+ struct kioctx_table __rcu *ioctx_table;
+#endif
+#ifdef CONFIG_MEMCG
+ /*
+ * "owner" points to a task that is regarded as the canonical
+ * user/owner of this mm. All of the following must be true in
+ * order for it to be changed:
+ *
+ * current == mm->owner
+ * current->mm != mm
+ * new_owner->mm == mm
+ * new_owner->alloc_lock is held
+ */
+ struct task_struct __rcu *owner;
+#endif
+
+ /* store ref to file /proc/<pid>/exe symlink points to */
+ struct file *exe_file;
+#ifdef CONFIG_MMU_NOTIFIER
+ struct mmu_notifier_mm *mmu_notifier_mm;
+#endif
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
+ pgtable_t pmd_huge_pte; /* protected by page_table_lock */
+#endif
+#ifdef CONFIG_CPUMASK_OFFSTACK
+ struct cpumask cpumask_allocation;
+#endif
+#ifdef CONFIG_NUMA_BALANCING
+ /*
+ * numa_next_scan is the next time that the PTEs will be marked
+ * pte_numa. NUMA hinting faults will gather statistics and migrate
+ * pages to new nodes if necessary.
+ */
+ unsigned long numa_next_scan;
+
+ /* Restart point for scanning and setting pte_numa */
+ unsigned long numa_scan_offset;
+
+ /* numa_scan_seq prevents two threads setting pte_numa */
+ int numa_scan_seq;
+#endif
+#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
+ /*
+ * An operation with batched TLB flushing is going on. Anything that
+ * can move process memory needs to flush the TLB when moving a
+ * PROT_NONE or PROT_NUMA mapped page.
+ */
+ bool tlb_flush_pending;
+#endif
+ struct uprobes_state uprobes_state;
+#ifdef CONFIG_X86_INTEL_MPX
+ /* address of the bounds directory */
+ void __user *bd_addr;
+#endif
+};
+
+static inline void mm_init_cpumask(struct mm_struct *mm)
+{
+#ifdef CONFIG_CPUMASK_OFFSTACK
+ mm->cpu_vm_mask_var = &mm->cpumask_allocation;
+#endif
+ cpumask_clear(mm->cpu_vm_mask_var);
+}
+
+/* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
+static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
+{
+ return mm->cpu_vm_mask_var;
+}
+
+#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
+/*
+ * Memory barriers to keep this state in sync are graciously provided by
+ * the page table locks, outside of which no page table modifications happen.
+ * The barriers below prevent the compiler from re-ordering the instructions
+ * around the memory barriers that are already present in the code.
+ */
+static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
+{
+ barrier();
+ return mm->tlb_flush_pending;
+}
+static inline void set_tlb_flush_pending(struct mm_struct *mm)
+{
+ mm->tlb_flush_pending = true;
+
+ /*
+ * Guarantee that the tlb_flush_pending store does not leak into the
+ * critical section updating the page tables
+ */
+ smp_mb__before_spinlock();
+}
+/* Clearing is done after a TLB flush, which also provides a barrier. */
+static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+{
+ barrier();
+ mm->tlb_flush_pending = false;
+}
+#else
+static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
+{
+ return false;
+}
+static inline void set_tlb_flush_pending(struct mm_struct *mm)
+{
+}
+static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+{
+}
+#endif
+
+#endif
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5dfdd5ed5254..80797b1c1e26 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1,24 +1,16 @@
#ifndef _LINUX_MM_TYPES_H
#define _LINUX_MM_TYPES_H
-#include <linux/auxvec.h>
-#include <linux/types.h>
-#include <linux/threads.h>
-#include <linux/list.h>
-#include <linux/spinlock.h>
-#include <linux/rbtree.h>
-#include <linux/rwsem.h>
#include <linux/completion.h>
#include <linux/cpumask.h>
-#include <linux/uprobes.h>
+#include <linux/list.h>
#include <linux/page-flags-layout.h>
-#include <asm/page.h>
-#include <asm/mmu.h>
+#include <linux/rbtree.h>
+#include <linux/spinlock.h>
+#include <linux/threads.h>
+#include <linux/types.h>
-#ifndef AT_VECTOR_SIZE_ARCH
-#define AT_VECTOR_SIZE_ARCH 0
-#endif
-#define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1))
+#include <asm/page.h>
struct address_space;
struct mem_cgroup;
@@ -326,203 +318,7 @@ struct core_state {
struct completion startup;
};
-enum {
- MM_FILEPAGES,
- MM_ANONPAGES,
- MM_SWAPENTS,
- NR_MM_COUNTERS
-};
-
-#if USE_SPLIT_PTE_PTLOCKS && defined(CONFIG_MMU)
-#define SPLIT_RSS_COUNTING
-/* per-thread cached information, */
-struct task_rss_stat {
- int events; /* for synchronization threshold */
- int count[NR_MM_COUNTERS];
-};
-#endif /* USE_SPLIT_PTE_PTLOCKS */
-
-struct mm_rss_stat {
- atomic_long_t count[NR_MM_COUNTERS];
-};
-
-struct kioctx_table;
-struct mm_struct {
- struct vm_area_struct *mmap; /* list of VMAs */
- struct rb_root mm_rb;
- u32 vmacache_seqnum; /* per-thread vmacache */
-#ifdef CONFIG_MMU
- unsigned long (*get_unmapped_area) (struct file *filp,
- unsigned long addr, unsigned long len,
- unsigned long pgoff, unsigned long flags);
-#endif
- unsigned long mmap_base; /* base of mmap area */
- unsigned long mmap_legacy_base; /* base of mmap area in bottom-up allocations */
- unsigned long task_size; /* size of task vm space */
- unsigned long highest_vm_end; /* highest vma end address */
- pgd_t * pgd;
- atomic_t mm_users; /* How many users with user space? */
- atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
- atomic_long_t nr_ptes; /* PTE page table pages */
- atomic_long_t nr_pmds; /* PMD page table pages */
- int map_count; /* number of VMAs */
-
- spinlock_t page_table_lock; /* Protects page tables and some counters */
- struct rw_semaphore mmap_sem;
-
- struct list_head mmlist; /* List of maybe swapped mm's. These are globally strung
- * together off init_mm.mmlist, and are protected
- * by mmlist_lock
- */
-
-
- unsigned long hiwater_rss; /* High-watermark of RSS usage */
- unsigned long hiwater_vm; /* High-water virtual memory usage */
-
- unsigned long total_vm; /* Total pages mapped */
- unsigned long locked_vm; /* Pages that have PG_mlocked set */
- unsigned long pinned_vm; /* Refcount permanently increased */
- unsigned long shared_vm; /* Shared pages (files) */
- unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE */
- unsigned long stack_vm; /* VM_GROWSUP/DOWN */
- unsigned long def_flags;
- unsigned long start_code, end_code, start_data, end_data;
- unsigned long start_brk, brk, start_stack;
- unsigned long arg_start, arg_end, env_start, env_end;
-
- unsigned long saved_auxv[AT_VECTOR_SIZE]; /* for /proc/PID/auxv */
-
- /*
- * Special counters, in some configurations protected by the
- * page_table_lock, in other configurations by being atomic.
- */
- struct mm_rss_stat rss_stat;
-
- struct linux_binfmt *binfmt;
-
- cpumask_var_t cpu_vm_mask_var;
-
- /* Architecture-specific MM context */
- mm_context_t context;
-
- unsigned long flags; /* Must use atomic bitops to access the bits */
-
- struct core_state *core_state; /* coredumping support */
-#ifdef CONFIG_AIO
- spinlock_t ioctx_lock;
- struct kioctx_table __rcu *ioctx_table;
-#endif
-#ifdef CONFIG_MEMCG
- /*
- * "owner" points to a task that is regarded as the canonical
- * user/owner of this mm. All of the following must be true in
- * order for it to be changed:
- *
- * current == mm->owner
- * current->mm != mm
- * new_owner->mm == mm
- * new_owner->alloc_lock is held
- */
- struct task_struct __rcu *owner;
-#endif
-
- /* store ref to file /proc/<pid>/exe symlink points to */
- struct file *exe_file;
-#ifdef CONFIG_MMU_NOTIFIER
- struct mmu_notifier_mm *mmu_notifier_mm;
-#endif
-#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
- pgtable_t pmd_huge_pte; /* protected by page_table_lock */
-#endif
-#ifdef CONFIG_CPUMASK_OFFSTACK
- struct cpumask cpumask_allocation;
-#endif
-#ifdef CONFIG_NUMA_BALANCING
- /*
- * numa_next_scan is the next time that the PTEs will be marked
- * pte_numa. NUMA hinting faults will gather statistics and migrate
- * pages to new nodes if necessary.
- */
- unsigned long numa_next_scan;
-
- /* Restart point for scanning and setting pte_numa */
- unsigned long numa_scan_offset;
-
- /* numa_scan_seq prevents two threads setting pte_numa */
- int numa_scan_seq;
-#endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
- /*
- * An operation with batched TLB flushing is going on. Anything that
- * can move process memory needs to flush the TLB when moving a
- * PROT_NONE or PROT_NUMA mapped page.
- */
- bool tlb_flush_pending;
-#endif
- struct uprobes_state uprobes_state;
-#ifdef CONFIG_X86_INTEL_MPX
- /* address of the bounds directory */
- void __user *bd_addr;
-#endif
-};
-
-static inline void mm_init_cpumask(struct mm_struct *mm)
-{
-#ifdef CONFIG_CPUMASK_OFFSTACK
- mm->cpu_vm_mask_var = &mm->cpumask_allocation;
-#endif
- cpumask_clear(mm->cpu_vm_mask_var);
-}
-
-/* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
-static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
-{
- return mm->cpu_vm_mask_var;
-}
-
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
-/*
- * Memory barriers to keep this state in sync are graciously provided by
- * the page table locks, outside of which no page table modifications happen.
- * The barriers below prevent the compiler from re-ordering the instructions
- * around the memory barriers that are already present in the code.
- */
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
- barrier();
- return mm->tlb_flush_pending;
-}
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
-{
- mm->tlb_flush_pending = true;
-
- /*
- * Guarantee that the tlb_flush_pending store does not leak into the
- * critical section updating the page tables
- */
- smp_mb__before_spinlock();
-}
-/* Clearing is done after a TLB flush, which also provides a barrier. */
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
-{
- barrier();
- mm->tlb_flush_pending = false;
-}
-#else
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
- return false;
-}
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-#endif
-
-struct vm_special_mapping
-{
+struct vm_special_mapping {
const char *name;
struct page **pages;
};
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index 95243d28a0ee..779e32567e6d 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -4,6 +4,7 @@
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/srcu.h>
struct mmu_notifier;
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 22ee0d5d7f8c..99001775ffa0 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -25,6 +25,7 @@ struct sched_param {
#include <linux/errno.h>
#include <linux/nodemask.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/preempt_mask.h>
#include <asm/page.h>
diff --git a/mm/init-mm.c b/mm/init-mm.c
index a56a851908d2..310ae8c7d9c6 100644
--- a/mm/init-mm.c
+++ b/mm/init-mm.c
@@ -1,4 +1,5 @@
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/rbtree.h>
#include <linux/rwsem.h>
#include <linux/spinlock.h>
diff --git a/mm/kmemcheck.c b/mm/kmemcheck.c
index cab58bb592d8..c40d065013d2 100644
--- a/mm/kmemcheck.c
+++ b/mm/kmemcheck.c
@@ -1,5 +1,4 @@
#include <linux/gfp.h>
-#include <linux/mm_types.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include "slab.h"
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 3/4] mm: define __PAGETABLE_{PMD,PUD}_FOLDED to zero or one
2015-01-28 13:17 ` Kirill A. Shutemov
@ 2015-01-28 13:17 ` Kirill A. Shutemov
-1 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
Currently, we define __PAGETABLE_PMD_FOLDED and __PAGETABLE_PUD_FOLDED
to empty value if the page table level is folded. The patch changes it
to define the macros to one if the page level is folded and zero
otherwise.
This would help to detect situation when include <asm/pgtable.h> is
missed or we have circular like <asm/pgtable.h> -> <linux/mm_struct.h>
-> <asm/pgtable.h>.
We still use #ifdef inside <asm/pgtables.h> since we only define macros
to zero at the end of header in <asm-generic/pgtable.h>.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/arm/include/asm/pgtable-2level.h | 2 +-
arch/arm64/include/asm/kvm_mmu.h | 4 ++--
arch/arm64/mm/hugetlbpage.c | 6 +++---
arch/microblaze/include/asm/pgtable.h | 2 +-
arch/mips/include/asm/pgalloc.h | 4 ++--
arch/mips/kernel/asm-offsets.c | 4 ++--
arch/mips/mm/init.c | 2 +-
arch/mips/mm/pgtable-64.c | 6 +++---
arch/mips/mm/tlbex.c | 8 ++++----
arch/powerpc/mm/pgtable_64.c | 2 +-
arch/sh/mm/init.c | 2 +-
arch/tile/mm/hugetlbpage.c | 4 ++--
arch/tile/mm/pgtable.c | 4 ++--
arch/x86/include/asm/xen/page.h | 2 +-
include/asm-generic/4level-fixup.h | 2 +-
include/asm-generic/pgtable-nopmd.h | 2 +-
include/asm-generic/pgtable-nopud.h | 2 +-
include/asm-generic/pgtable.h | 8 ++++++++
include/linux/mm.h | 4 ++--
kernel/fork.c | 2 +-
mm/memory.c | 4 ++--
21 files changed, 42 insertions(+), 34 deletions(-)
diff --git a/arch/arm/include/asm/pgtable-2level.h b/arch/arm/include/asm/pgtable-2level.h
index bfd662e49a25..7d77adce17ae 100644
--- a/arch/arm/include/asm/pgtable-2level.h
+++ b/arch/arm/include/asm/pgtable-2level.h
@@ -10,7 +10,7 @@
#ifndef _ASM_PGTABLE_2LEVEL_H
#define _ASM_PGTABLE_2LEVEL_H
-#define __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 1
/*
* Hardware-wise, we have a two level page table structure, where the first
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 66577581ce68..a14a14323cb6 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -240,14 +240,14 @@ static inline bool kvm_page_empty(void *ptr)
#define kvm_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
#define kvm_pmd_table_empty(kvm, pmdp) (0)
#else
#define kvm_pmd_table_empty(kvm, pmdp) \
(kvm_page_empty(pmdp) && (!(kvm) || KVM_PREALLOC_LEVEL < 2))
#endif
-#ifdef __PAGETABLE_PUD_FOLDED
+#if __PAGETABLE_PUD_FOLDED
#define kvm_pud_table_empty(kvm, pudp) (0)
#else
#define kvm_pud_table_empty(kvm, pudp) \
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index 2de9d2e59d96..f37dadacc4cd 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -45,10 +45,10 @@ int pmd_huge(pmd_t pmd)
int pud_huge(pud_t pud)
{
-#ifndef __PAGETABLE_PMD_FOLDED
- return !(pud_val(pud) & PUD_TABLE_BIT);
-#else
+#if __PAGETABLE_PMD_FOLDED
return 0;
+#else
+ return !(pud_val(pud) & PUD_TABLE_BIT);
#endif
}
diff --git a/arch/microblaze/include/asm/pgtable.h b/arch/microblaze/include/asm/pgtable.h
index e53b8532353c..2789da060229 100644
--- a/arch/microblaze/include/asm/pgtable.h
+++ b/arch/microblaze/include/asm/pgtable.h
@@ -61,7 +61,7 @@ extern int mem_init_done;
#include <asm-generic/4level-fixup.h>
-#define __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 1
#ifdef __KERNEL__
#ifndef __ASSEMBLY__
diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h
index b336037e8768..5b0164ba93d9 100644
--- a/arch/mips/include/asm/pgalloc.h
+++ b/arch/mips/include/asm/pgalloc.h
@@ -31,7 +31,7 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
*/
extern void pmd_init(unsigned long page, unsigned long pagetable);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
{
@@ -107,7 +107,7 @@ do { \
tlb_remove_page((tlb), pte); \
} while (0)
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
{
diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c
index 3b2dfdb4865f..1b2c5f8653aa 100644
--- a/arch/mips/kernel/asm-offsets.c
+++ b/arch/mips/kernel/asm-offsets.c
@@ -252,13 +252,13 @@ void output_mm_defines(void)
DEFINE(_PTE_T_SIZE, sizeof(pte_t));
BLANK();
DEFINE(_PGD_T_LOG2, PGD_T_LOG2);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
DEFINE(_PMD_T_LOG2, PMD_T_LOG2);
#endif
DEFINE(_PTE_T_LOG2, PTE_T_LOG2);
BLANK();
DEFINE(_PGD_ORDER, PGD_ORDER);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
DEFINE(_PMD_ORDER, PMD_ORDER);
#endif
DEFINE(_PTE_ORDER, PTE_ORDER);
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 448cde372af0..838838b05d1c 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -440,7 +440,7 @@ unsigned long pgd_current[NR_CPUS];
* it in the linker script.
*/
pgd_t swapper_pg_dir[_PTRS_PER_PGD] __section(.bss..swapper_pg_dir);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
pmd_t invalid_pmd_table[PTRS_PER_PMD] __page_aligned_bss;
#endif
pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss;
diff --git a/arch/mips/mm/pgtable-64.c b/arch/mips/mm/pgtable-64.c
index e8adc0069d66..379b8390f059 100644
--- a/arch/mips/mm/pgtable-64.c
+++ b/arch/mips/mm/pgtable-64.c
@@ -18,7 +18,7 @@ void pgd_init(unsigned long page)
unsigned long *p, *end;
unsigned long entry;
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
entry = (unsigned long)invalid_pte_table;
#else
entry = (unsigned long)invalid_pmd_table;
@@ -40,7 +40,7 @@ void pgd_init(unsigned long page)
} while (p != end);
}
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
void pmd_init(unsigned long addr, unsigned long pagetable)
{
unsigned long *p, *end;
@@ -99,7 +99,7 @@ void __init pagetable_init(void)
/* Initialize the entire pgd. */
pgd_init((unsigned long)swapper_pg_dir);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
pmd_init((unsigned long)invalid_pmd_table, (unsigned long)invalid_pte_table);
#endif
pgd_base = swapper_pg_dir;
diff --git a/arch/mips/mm/tlbex.c b/arch/mips/mm/tlbex.c
index 3978a3d81366..afbadec7d595 100644
--- a/arch/mips/mm/tlbex.c
+++ b/arch/mips/mm/tlbex.c
@@ -840,7 +840,7 @@ build_get_pmde64(u32 **p, struct uasm_label **l, struct uasm_reloc **r,
uasm_i_andi(p, tmp, tmp, (PTRS_PER_PGD - 1)<<3);
uasm_i_daddu(p, ptr, ptr, tmp); /* add in pgd offset */
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
uasm_i_dmfc0(p, tmp, C0_BADVADDR); /* get faulting address */
uasm_i_ld(p, ptr, 0, ptr); /* get pmd pointer */
uasm_i_dsrl_safe(p, tmp, tmp, PMD_SHIFT-3); /* get pmd offset in bytes */
@@ -1128,7 +1128,7 @@ build_fast_tlb_refill_handler (u32 **p, struct uasm_label **l,
uasm_i_drotr(p, ptr, ptr, 11);
}
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
#define LOC_PTEP scratch
#else
#define LOC_PTEP ptr
@@ -1150,7 +1150,7 @@ build_fast_tlb_refill_handler (u32 **p, struct uasm_label **l,
/* get pgd offset in bytes */
uasm_i_dsrl_safe(p, scratch, tmp, PGDIR_SHIFT - 3);
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
GET_CONTEXT(p, tmp); /* get context reg */
#endif
uasm_i_andi(p, scratch, scratch, (PTRS_PER_PGD - 1) << 3);
@@ -1162,7 +1162,7 @@ build_fast_tlb_refill_handler (u32 **p, struct uasm_label **l,
uasm_i_ld(p, LOC_PTEP, 0, ptr); /* get pmd pointer */
}
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
/* get pmd offset in bytes */
uasm_i_dsrl_safe(p, scratch, tmp, PMD_SHIFT - 3);
uasm_i_andi(p, scratch, scratch, (PTRS_PER_PMD - 1) << 3);
diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
index 6957cc1ca0a7..727832cad640 100644
--- a/arch/powerpc/mm/pgtable_64.c
+++ b/arch/powerpc/mm/pgtable_64.c
@@ -345,7 +345,7 @@ EXPORT_SYMBOL(iounmap);
EXPORT_SYMBOL(__iounmap);
EXPORT_SYMBOL(__iounmap_at);
-#ifndef __PAGETABLE_PUD_FOLDED
+#if !__PAGETABLE_PUD_FOLDED
/* 4 level page table */
struct page *pgd_page(pgd_t pgd)
{
diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c
index 2790b6a64157..1e611d71301c 100644
--- a/arch/sh/mm/init.c
+++ b/arch/sh/mm/init.c
@@ -175,7 +175,7 @@ void __init page_table_range_init(unsigned long start, unsigned long end,
pud = (pud_t *)pgd;
for ( ; (j < PTRS_PER_PUD) && (vaddr != end); pud++, j++) {
pmd = one_md_table_init(pud);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
pmd += k;
#endif
for (; (k < PTRS_PER_PMD) && (vaddr != end); pmd++, k++) {
diff --git a/arch/tile/mm/hugetlbpage.c b/arch/tile/mm/hugetlbpage.c
index 8416240c322c..97c4ccecd0ed 100644
--- a/arch/tile/mm/hugetlbpage.c
+++ b/arch/tile/mm/hugetlbpage.c
@@ -115,14 +115,14 @@ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr)
/* We don't have four levels. */
pud = pud_offset(pgd, addr);
-#ifndef __PAGETABLE_PUD_FOLDED
+#if !__PAGETABLE_PUD_FOLDED
# error support fourth page table level
#endif
if (!pud_present(*pud))
return NULL;
/* Check for an L0 huge PTE, if we have three levels. */
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
if (pud_huge(*pud))
return (pte_t *)pud;
diff --git a/arch/tile/mm/pgtable.c b/arch/tile/mm/pgtable.c
index 7bf2491a9c1f..e37081d62ca9 100644
--- a/arch/tile/mm/pgtable.c
+++ b/arch/tile/mm/pgtable.c
@@ -101,7 +101,7 @@ void shatter_huge_page(unsigned long addr)
pud_t *pud;
pmd_t *pmd;
unsigned long flags = 0; /* happy compiler */
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
struct list_head *pos;
#endif
@@ -127,7 +127,7 @@ void shatter_huge_page(unsigned long addr)
/* Shatter the huge page into the preallocated L2 page table. */
pmd_populate_kernel(&init_mm, pmd, get_prealloc_pte(pmd_pfn(*pmd)));
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
/* Walk every pgd on the system and update the pmd there. */
spin_lock(&pgd_lock);
list_for_each(pos, &pgd_list) {
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index 5eea09915a15..49fc452d4899 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -253,7 +253,7 @@ static inline pte_t __pte_ma(pteval_t x)
}
#define pmd_val_ma(v) ((v).pmd)
-#ifdef __PAGETABLE_PUD_FOLDED
+#if __PAGETABLE_PUD_FOLDED
#define pud_val_ma(v) ((v).pgd.pgd)
#else
#define pud_val_ma(v) ((v).pud)
diff --git a/include/asm-generic/4level-fixup.h b/include/asm-generic/4level-fixup.h
index 5bdab6bffd23..c6a349d459e5 100644
--- a/include/asm-generic/4level-fixup.h
+++ b/include/asm-generic/4level-fixup.h
@@ -2,7 +2,7 @@
#define _4LEVEL_FIXUP_H
#define __ARCH_HAS_4LEVEL_HACK
-#define __PAGETABLE_PUD_FOLDED
+#define __PAGETABLE_PUD_FOLDED 1
#define PUD_SHIFT PGDIR_SHIFT
#define PUD_SIZE PGDIR_SIZE
diff --git a/include/asm-generic/pgtable-nopmd.h b/include/asm-generic/pgtable-nopmd.h
index 725612b793ce..6373f61824a0 100644
--- a/include/asm-generic/pgtable-nopmd.h
+++ b/include/asm-generic/pgtable-nopmd.h
@@ -7,7 +7,7 @@
struct mm_struct;
-#define __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 1
/*
* Having the pmd type consist of a pud gets the size right, and allows
diff --git a/include/asm-generic/pgtable-nopud.h b/include/asm-generic/pgtable-nopud.h
index 810431d8351b..8bc0a92c0764 100644
--- a/include/asm-generic/pgtable-nopud.h
+++ b/include/asm-generic/pgtable-nopud.h
@@ -3,7 +3,7 @@
#ifndef __ASSEMBLY__
-#define __PAGETABLE_PUD_FOLDED
+#define __PAGETABLE_PUD_FOLDED 1
/*
* Having the pud type consist of a pgd gets the size right, and allows
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 4d46085c1b90..22bffcb2e96b 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -17,6 +17,14 @@
#define USER_PGTABLES_CEILING 0UL
#endif
+#ifndef __PAGETABLE_PUD_FOLDED
+#define __PAGETABLE_PUD_FOLDED 0
+#endif
+
+#ifndef __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 0
+#endif
+
#ifndef __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
extern int ptep_set_access_flags(struct vm_area_struct *vma,
unsigned long address, pte_t *ptep,
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 543e9723d441..98bde6d48640 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1435,7 +1435,7 @@ static inline pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
return ptep;
}
-#ifdef __PAGETABLE_PUD_FOLDED
+#if __PAGETABLE_PUD_FOLDED
static inline int __pud_alloc(struct mm_struct *mm, pgd_t *pgd,
unsigned long address)
{
@@ -1445,7 +1445,7 @@ static inline int __pud_alloc(struct mm_struct *mm, pgd_t *pgd,
int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address);
#endif
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
static inline int __pmd_alloc(struct mm_struct *mm, pud_t *pud,
unsigned long address)
{
diff --git a/kernel/fork.c b/kernel/fork.c
index 76d6f292274c..da94fa59b96a 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -555,7 +555,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p)
INIT_LIST_HEAD(&mm->mmlist);
mm->core_state = NULL;
atomic_long_set(&mm->nr_ptes, 0);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
atomic_long_set(&mm->nr_pmds, 0);
#endif
mm->map_count = 0;
diff --git a/mm/memory.c b/mm/memory.c
index ed697e9a5e5b..27eb05ddadb9 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3311,7 +3311,7 @@ int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
}
EXPORT_SYMBOL_GPL(handle_mm_fault);
-#ifndef __PAGETABLE_PUD_FOLDED
+#if !__PAGETABLE_PUD_FOLDED
/*
* Allocate page upper directory.
* We've already handled the fast-path in-line.
@@ -3334,7 +3334,7 @@ int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address)
}
#endif /* __PAGETABLE_PUD_FOLDED */
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
/*
* Allocate page middle directory.
* We've already handled the fast-path in-line.
--
2.1.4
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 3/4] mm: define __PAGETABLE_{PMD,PUD}_FOLDED to zero or one
@ 2015-01-28 13:17 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
Currently, we define __PAGETABLE_PMD_FOLDED and __PAGETABLE_PUD_FOLDED
to empty value if the page table level is folded. The patch changes it
to define the macros to one if the page level is folded and zero
otherwise.
This would help to detect situation when include <asm/pgtable.h> is
missed or we have circular like <asm/pgtable.h> -> <linux/mm_struct.h>
-> <asm/pgtable.h>.
We still use #ifdef inside <asm/pgtables.h> since we only define macros
to zero at the end of header in <asm-generic/pgtable.h>.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/arm/include/asm/pgtable-2level.h | 2 +-
arch/arm64/include/asm/kvm_mmu.h | 4 ++--
arch/arm64/mm/hugetlbpage.c | 6 +++---
arch/microblaze/include/asm/pgtable.h | 2 +-
arch/mips/include/asm/pgalloc.h | 4 ++--
arch/mips/kernel/asm-offsets.c | 4 ++--
arch/mips/mm/init.c | 2 +-
arch/mips/mm/pgtable-64.c | 6 +++---
arch/mips/mm/tlbex.c | 8 ++++----
arch/powerpc/mm/pgtable_64.c | 2 +-
arch/sh/mm/init.c | 2 +-
arch/tile/mm/hugetlbpage.c | 4 ++--
arch/tile/mm/pgtable.c | 4 ++--
arch/x86/include/asm/xen/page.h | 2 +-
include/asm-generic/4level-fixup.h | 2 +-
include/asm-generic/pgtable-nopmd.h | 2 +-
include/asm-generic/pgtable-nopud.h | 2 +-
include/asm-generic/pgtable.h | 8 ++++++++
include/linux/mm.h | 4 ++--
kernel/fork.c | 2 +-
mm/memory.c | 4 ++--
21 files changed, 42 insertions(+), 34 deletions(-)
diff --git a/arch/arm/include/asm/pgtable-2level.h b/arch/arm/include/asm/pgtable-2level.h
index bfd662e49a25..7d77adce17ae 100644
--- a/arch/arm/include/asm/pgtable-2level.h
+++ b/arch/arm/include/asm/pgtable-2level.h
@@ -10,7 +10,7 @@
#ifndef _ASM_PGTABLE_2LEVEL_H
#define _ASM_PGTABLE_2LEVEL_H
-#define __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 1
/*
* Hardware-wise, we have a two level page table structure, where the first
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 66577581ce68..a14a14323cb6 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -240,14 +240,14 @@ static inline bool kvm_page_empty(void *ptr)
#define kvm_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
#define kvm_pmd_table_empty(kvm, pmdp) (0)
#else
#define kvm_pmd_table_empty(kvm, pmdp) \
(kvm_page_empty(pmdp) && (!(kvm) || KVM_PREALLOC_LEVEL < 2))
#endif
-#ifdef __PAGETABLE_PUD_FOLDED
+#if __PAGETABLE_PUD_FOLDED
#define kvm_pud_table_empty(kvm, pudp) (0)
#else
#define kvm_pud_table_empty(kvm, pudp) \
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index 2de9d2e59d96..f37dadacc4cd 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -45,10 +45,10 @@ int pmd_huge(pmd_t pmd)
int pud_huge(pud_t pud)
{
-#ifndef __PAGETABLE_PMD_FOLDED
- return !(pud_val(pud) & PUD_TABLE_BIT);
-#else
+#if __PAGETABLE_PMD_FOLDED
return 0;
+#else
+ return !(pud_val(pud) & PUD_TABLE_BIT);
#endif
}
diff --git a/arch/microblaze/include/asm/pgtable.h b/arch/microblaze/include/asm/pgtable.h
index e53b8532353c..2789da060229 100644
--- a/arch/microblaze/include/asm/pgtable.h
+++ b/arch/microblaze/include/asm/pgtable.h
@@ -61,7 +61,7 @@ extern int mem_init_done;
#include <asm-generic/4level-fixup.h>
-#define __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 1
#ifdef __KERNEL__
#ifndef __ASSEMBLY__
diff --git a/arch/mips/include/asm/pgalloc.h b/arch/mips/include/asm/pgalloc.h
index b336037e8768..5b0164ba93d9 100644
--- a/arch/mips/include/asm/pgalloc.h
+++ b/arch/mips/include/asm/pgalloc.h
@@ -31,7 +31,7 @@ static inline void pmd_populate(struct mm_struct *mm, pmd_t *pmd,
*/
extern void pmd_init(unsigned long page, unsigned long pagetable);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
static inline void pud_populate(struct mm_struct *mm, pud_t *pud, pmd_t *pmd)
{
@@ -107,7 +107,7 @@ do { \
tlb_remove_page((tlb), pte); \
} while (0)
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
static inline pmd_t *pmd_alloc_one(struct mm_struct *mm, unsigned long address)
{
diff --git a/arch/mips/kernel/asm-offsets.c b/arch/mips/kernel/asm-offsets.c
index 3b2dfdb4865f..1b2c5f8653aa 100644
--- a/arch/mips/kernel/asm-offsets.c
+++ b/arch/mips/kernel/asm-offsets.c
@@ -252,13 +252,13 @@ void output_mm_defines(void)
DEFINE(_PTE_T_SIZE, sizeof(pte_t));
BLANK();
DEFINE(_PGD_T_LOG2, PGD_T_LOG2);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
DEFINE(_PMD_T_LOG2, PMD_T_LOG2);
#endif
DEFINE(_PTE_T_LOG2, PTE_T_LOG2);
BLANK();
DEFINE(_PGD_ORDER, PGD_ORDER);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
DEFINE(_PMD_ORDER, PMD_ORDER);
#endif
DEFINE(_PTE_ORDER, PTE_ORDER);
diff --git a/arch/mips/mm/init.c b/arch/mips/mm/init.c
index 448cde372af0..838838b05d1c 100644
--- a/arch/mips/mm/init.c
+++ b/arch/mips/mm/init.c
@@ -440,7 +440,7 @@ unsigned long pgd_current[NR_CPUS];
* it in the linker script.
*/
pgd_t swapper_pg_dir[_PTRS_PER_PGD] __section(.bss..swapper_pg_dir);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
pmd_t invalid_pmd_table[PTRS_PER_PMD] __page_aligned_bss;
#endif
pte_t invalid_pte_table[PTRS_PER_PTE] __page_aligned_bss;
diff --git a/arch/mips/mm/pgtable-64.c b/arch/mips/mm/pgtable-64.c
index e8adc0069d66..379b8390f059 100644
--- a/arch/mips/mm/pgtable-64.c
+++ b/arch/mips/mm/pgtable-64.c
@@ -18,7 +18,7 @@ void pgd_init(unsigned long page)
unsigned long *p, *end;
unsigned long entry;
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
entry = (unsigned long)invalid_pte_table;
#else
entry = (unsigned long)invalid_pmd_table;
@@ -40,7 +40,7 @@ void pgd_init(unsigned long page)
} while (p != end);
}
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
void pmd_init(unsigned long addr, unsigned long pagetable)
{
unsigned long *p, *end;
@@ -99,7 +99,7 @@ void __init pagetable_init(void)
/* Initialize the entire pgd. */
pgd_init((unsigned long)swapper_pg_dir);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
pmd_init((unsigned long)invalid_pmd_table, (unsigned long)invalid_pte_table);
#endif
pgd_base = swapper_pg_dir;
diff --git a/arch/mips/mm/tlbex.c b/arch/mips/mm/tlbex.c
index 3978a3d81366..afbadec7d595 100644
--- a/arch/mips/mm/tlbex.c
+++ b/arch/mips/mm/tlbex.c
@@ -840,7 +840,7 @@ build_get_pmde64(u32 **p, struct uasm_label **l, struct uasm_reloc **r,
uasm_i_andi(p, tmp, tmp, (PTRS_PER_PGD - 1)<<3);
uasm_i_daddu(p, ptr, ptr, tmp); /* add in pgd offset */
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
uasm_i_dmfc0(p, tmp, C0_BADVADDR); /* get faulting address */
uasm_i_ld(p, ptr, 0, ptr); /* get pmd pointer */
uasm_i_dsrl_safe(p, tmp, tmp, PMD_SHIFT-3); /* get pmd offset in bytes */
@@ -1128,7 +1128,7 @@ build_fast_tlb_refill_handler (u32 **p, struct uasm_label **l,
uasm_i_drotr(p, ptr, ptr, 11);
}
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
#define LOC_PTEP scratch
#else
#define LOC_PTEP ptr
@@ -1150,7 +1150,7 @@ build_fast_tlb_refill_handler (u32 **p, struct uasm_label **l,
/* get pgd offset in bytes */
uasm_i_dsrl_safe(p, scratch, tmp, PGDIR_SHIFT - 3);
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
GET_CONTEXT(p, tmp); /* get context reg */
#endif
uasm_i_andi(p, scratch, scratch, (PTRS_PER_PGD - 1) << 3);
@@ -1162,7 +1162,7 @@ build_fast_tlb_refill_handler (u32 **p, struct uasm_label **l,
uasm_i_ld(p, LOC_PTEP, 0, ptr); /* get pmd pointer */
}
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
/* get pmd offset in bytes */
uasm_i_dsrl_safe(p, scratch, tmp, PMD_SHIFT - 3);
uasm_i_andi(p, scratch, scratch, (PTRS_PER_PMD - 1) << 3);
diff --git a/arch/powerpc/mm/pgtable_64.c b/arch/powerpc/mm/pgtable_64.c
index 6957cc1ca0a7..727832cad640 100644
--- a/arch/powerpc/mm/pgtable_64.c
+++ b/arch/powerpc/mm/pgtable_64.c
@@ -345,7 +345,7 @@ EXPORT_SYMBOL(iounmap);
EXPORT_SYMBOL(__iounmap);
EXPORT_SYMBOL(__iounmap_at);
-#ifndef __PAGETABLE_PUD_FOLDED
+#if !__PAGETABLE_PUD_FOLDED
/* 4 level page table */
struct page *pgd_page(pgd_t pgd)
{
diff --git a/arch/sh/mm/init.c b/arch/sh/mm/init.c
index 2790b6a64157..1e611d71301c 100644
--- a/arch/sh/mm/init.c
+++ b/arch/sh/mm/init.c
@@ -175,7 +175,7 @@ void __init page_table_range_init(unsigned long start, unsigned long end,
pud = (pud_t *)pgd;
for ( ; (j < PTRS_PER_PUD) && (vaddr != end); pud++, j++) {
pmd = one_md_table_init(pud);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
pmd += k;
#endif
for (; (k < PTRS_PER_PMD) && (vaddr != end); pmd++, k++) {
diff --git a/arch/tile/mm/hugetlbpage.c b/arch/tile/mm/hugetlbpage.c
index 8416240c322c..97c4ccecd0ed 100644
--- a/arch/tile/mm/hugetlbpage.c
+++ b/arch/tile/mm/hugetlbpage.c
@@ -115,14 +115,14 @@ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr)
/* We don't have four levels. */
pud = pud_offset(pgd, addr);
-#ifndef __PAGETABLE_PUD_FOLDED
+#if !__PAGETABLE_PUD_FOLDED
# error support fourth page table level
#endif
if (!pud_present(*pud))
return NULL;
/* Check for an L0 huge PTE, if we have three levels. */
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
if (pud_huge(*pud))
return (pte_t *)pud;
diff --git a/arch/tile/mm/pgtable.c b/arch/tile/mm/pgtable.c
index 7bf2491a9c1f..e37081d62ca9 100644
--- a/arch/tile/mm/pgtable.c
+++ b/arch/tile/mm/pgtable.c
@@ -101,7 +101,7 @@ void shatter_huge_page(unsigned long addr)
pud_t *pud;
pmd_t *pmd;
unsigned long flags = 0; /* happy compiler */
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
struct list_head *pos;
#endif
@@ -127,7 +127,7 @@ void shatter_huge_page(unsigned long addr)
/* Shatter the huge page into the preallocated L2 page table. */
pmd_populate_kernel(&init_mm, pmd, get_prealloc_pte(pmd_pfn(*pmd)));
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
/* Walk every pgd on the system and update the pmd there. */
spin_lock(&pgd_lock);
list_for_each(pos, &pgd_list) {
diff --git a/arch/x86/include/asm/xen/page.h b/arch/x86/include/asm/xen/page.h
index 5eea09915a15..49fc452d4899 100644
--- a/arch/x86/include/asm/xen/page.h
+++ b/arch/x86/include/asm/xen/page.h
@@ -253,7 +253,7 @@ static inline pte_t __pte_ma(pteval_t x)
}
#define pmd_val_ma(v) ((v).pmd)
-#ifdef __PAGETABLE_PUD_FOLDED
+#if __PAGETABLE_PUD_FOLDED
#define pud_val_ma(v) ((v).pgd.pgd)
#else
#define pud_val_ma(v) ((v).pud)
diff --git a/include/asm-generic/4level-fixup.h b/include/asm-generic/4level-fixup.h
index 5bdab6bffd23..c6a349d459e5 100644
--- a/include/asm-generic/4level-fixup.h
+++ b/include/asm-generic/4level-fixup.h
@@ -2,7 +2,7 @@
#define _4LEVEL_FIXUP_H
#define __ARCH_HAS_4LEVEL_HACK
-#define __PAGETABLE_PUD_FOLDED
+#define __PAGETABLE_PUD_FOLDED 1
#define PUD_SHIFT PGDIR_SHIFT
#define PUD_SIZE PGDIR_SIZE
diff --git a/include/asm-generic/pgtable-nopmd.h b/include/asm-generic/pgtable-nopmd.h
index 725612b793ce..6373f61824a0 100644
--- a/include/asm-generic/pgtable-nopmd.h
+++ b/include/asm-generic/pgtable-nopmd.h
@@ -7,7 +7,7 @@
struct mm_struct;
-#define __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 1
/*
* Having the pmd type consist of a pud gets the size right, and allows
diff --git a/include/asm-generic/pgtable-nopud.h b/include/asm-generic/pgtable-nopud.h
index 810431d8351b..8bc0a92c0764 100644
--- a/include/asm-generic/pgtable-nopud.h
+++ b/include/asm-generic/pgtable-nopud.h
@@ -3,7 +3,7 @@
#ifndef __ASSEMBLY__
-#define __PAGETABLE_PUD_FOLDED
+#define __PAGETABLE_PUD_FOLDED 1
/*
* Having the pud type consist of a pgd gets the size right, and allows
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 4d46085c1b90..22bffcb2e96b 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -17,6 +17,14 @@
#define USER_PGTABLES_CEILING 0UL
#endif
+#ifndef __PAGETABLE_PUD_FOLDED
+#define __PAGETABLE_PUD_FOLDED 0
+#endif
+
+#ifndef __PAGETABLE_PMD_FOLDED
+#define __PAGETABLE_PMD_FOLDED 0
+#endif
+
#ifndef __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS
extern int ptep_set_access_flags(struct vm_area_struct *vma,
unsigned long address, pte_t *ptep,
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 543e9723d441..98bde6d48640 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1435,7 +1435,7 @@ static inline pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr,
return ptep;
}
-#ifdef __PAGETABLE_PUD_FOLDED
+#if __PAGETABLE_PUD_FOLDED
static inline int __pud_alloc(struct mm_struct *mm, pgd_t *pgd,
unsigned long address)
{
@@ -1445,7 +1445,7 @@ static inline int __pud_alloc(struct mm_struct *mm, pgd_t *pgd,
int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address);
#endif
-#ifdef __PAGETABLE_PMD_FOLDED
+#if __PAGETABLE_PMD_FOLDED
static inline int __pmd_alloc(struct mm_struct *mm, pud_t *pud,
unsigned long address)
{
diff --git a/kernel/fork.c b/kernel/fork.c
index 76d6f292274c..da94fa59b96a 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -555,7 +555,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p)
INIT_LIST_HEAD(&mm->mmlist);
mm->core_state = NULL;
atomic_long_set(&mm->nr_ptes, 0);
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
atomic_long_set(&mm->nr_pmds, 0);
#endif
mm->map_count = 0;
diff --git a/mm/memory.c b/mm/memory.c
index ed697e9a5e5b..27eb05ddadb9 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3311,7 +3311,7 @@ int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
}
EXPORT_SYMBOL_GPL(handle_mm_fault);
-#ifndef __PAGETABLE_PUD_FOLDED
+#if !__PAGETABLE_PUD_FOLDED
/*
* Allocate page upper directory.
* We've already handled the fast-path in-line.
@@ -3334,7 +3334,7 @@ int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address)
}
#endif /* __PAGETABLE_PUD_FOLDED */
-#ifndef __PAGETABLE_PMD_FOLDED
+#if !__PAGETABLE_PMD_FOLDED
/*
* Allocate page middle directory.
* We've already handled the fast-path in-line.
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 4/4] mm: do not add nr_pmds into mm_struct if PMD is folded
2015-01-28 13:17 ` Kirill A. Shutemov
@ 2015-01-28 13:17 ` Kirill A. Shutemov
-1 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
Everything seems in place. We can now include <asm/pgtable.h> in
<linux/mm_struct.h>.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
include/linux/mm_struct.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/linux/mm_struct.h b/include/linux/mm_struct.h
index 0a233c232a39..1759bed3dc61 100644
--- a/include/linux/mm_struct.h
+++ b/include/linux/mm_struct.h
@@ -8,6 +8,7 @@
#include <linux/uprobes.h>
#include <asm/mmu.h>
+#include <asm/pgtable.h>
struct kioctx_table;
struct vm_area_struct;
@@ -54,7 +55,9 @@ struct mm_struct {
atomic_t mm_users; /* How many users with user space? */
atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
atomic_long_t nr_ptes; /* PTE page table pages */
+#if !__PAGETABLE_PMD_FOLDED
atomic_long_t nr_pmds; /* PMD page table pages */
+#endif
int map_count; /* number of VMAs */
spinlock_t page_table_lock; /* Protects page tables and some counters */
--
2.1.4
^ permalink raw reply related [flat|nested] 28+ messages in thread
* [PATCH 4/4] mm: do not add nr_pmds into mm_struct if PMD is folded
@ 2015-01-28 13:17 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 13:17 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-mm, linux-kernel, Guenter Roeck, Kirill A. Shutemov
Everything seems in place. We can now include <asm/pgtable.h> in
<linux/mm_struct.h>.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
include/linux/mm_struct.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/linux/mm_struct.h b/include/linux/mm_struct.h
index 0a233c232a39..1759bed3dc61 100644
--- a/include/linux/mm_struct.h
+++ b/include/linux/mm_struct.h
@@ -8,6 +8,7 @@
#include <linux/uprobes.h>
#include <asm/mmu.h>
+#include <asm/pgtable.h>
struct kioctx_table;
struct vm_area_struct;
@@ -54,7 +55,9 @@ struct mm_struct {
atomic_t mm_users; /* How many users with user space? */
atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
atomic_long_t nr_ptes; /* PTE page table pages */
+#if !__PAGETABLE_PMD_FOLDED
atomic_long_t nr_pmds; /* PMD page table pages */
+#endif
int map_count; /* number of VMAs */
spinlock_t page_table_lock; /* Protects page tables and some counters */
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
2015-01-28 13:17 ` Kirill A. Shutemov
@ 2015-01-28 17:06 ` Guenter Roeck
-1 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 17:06 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> This patchset moves definition of mm_struct into separate header file.
> It allows to get rid of nr_pmds if PMD page table level is folded.
> We cannot do it with current mm_types.h because we need
> __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> dependencies.
>
> I've done few build tests and looks like it works, but I expect breakage
> on some configuration. Please test.
>
I applied your patches on top of the current mmotm and pushed into
my 'testing' branch. I'll send out test results after the test cycle
is complete.
Guenter
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 17:06 ` Guenter Roeck
0 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 17:06 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> This patchset moves definition of mm_struct into separate header file.
> It allows to get rid of nr_pmds if PMD page table level is folded.
> We cannot do it with current mm_types.h because we need
> __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> dependencies.
>
> I've done few build tests and looks like it works, but I expect breakage
> on some configuration. Please test.
>
I applied your patches on top of the current mmotm and pushed into
my 'testing' branch. I'll send out test results after the test cycle
is complete.
Guenter
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
2015-01-28 13:17 ` Kirill A. Shutemov
@ 2015-01-28 18:50 ` Guenter Roeck
-1 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 18:50 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> This patchset moves definition of mm_struct into separate header file.
> It allows to get rid of nr_pmds if PMD page table level is folded.
> We cannot do it with current mm_types.h because we need
> __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> dependencies.
>
> I've done few build tests and looks like it works, but I expect breakage
> on some configuration. Please test.
>
Doesn't look good.
Build results:
total: 134 pass: 63 fail: 71
Failed builds:
arm:s3c2410_defconfig
arm:omap2plus_defconfig
arm:imx_v6_v7_defconfig
arm:ixp4xx_defconfig
arm:u8500_defconfig
arm:multi_v5_defconfig
arm:multi_v7_defconfig
arm:omap1_defconfig
arm:footbridge_defconfig
arm:davinci_all_defconfig
arm:mini2440_defconfig
arm:rpc_defconfig
arm:axm55xx_defconfig
arm:mxs_defconfig
arm:keystone_defconfig
arm:vexpress_defconfig
arm:imx_v4_v5_defconfig
arm:at91_dt_defconfig
arm:s3c6400_defconfig
arm:lpc32xx_defconfig
arm:shmobile_defconfig
arm:nhk8815_defconfig
arm:bcm2835_defconfig
arm:sama5_defconfig
arm:orion5x_defconfig
arm:exynos_defconfig
arm:cm_x2xx_defconfig
arm:s5pv210_defconfig
arm:integrator_defconfig
arm:msm_defconfig
arm:pxa910_defconfig
arm:clps711x_defconfig
avr32:defconfig
avr32:merisc_defconfig
avr32:atngw100mkii_evklcd101_defconfig
cris:defconfig
cris:etrax-100lx_defconfig
cris:allnoconfig
cris:artpec_3_defconfig
cris:etraxfs_defconfig
frv:defconfig
hexagon:defconfig
ia64:defconfig
m68k:defconfig
m68k:allmodconfig
m68k:sun3_defconfig
m68k:m5475evb_defconfig
microblaze:mmu_defconfig
mips:allmodconfig
powerpc:ppc6xx_defconfig
powerpc:mpc83xx_defconfig
powerpc:mpc85xx_defconfig
powerpc:mpc85xx_smp_defconfig
powerpc:tqm8xx_defconfig
powerpc:85xx/sbc8548_defconfig
powerpc:83xx/mpc834x_mds_defconfig
powerpc:86xx/sbc8641d_defconfig
powerpc:ppc6xx_defconfig
powerpc:mpc83xx_defconfig
powerpc:mpc85xx_defconfig
powerpc:mpc85xx_smp_defconfig
powerpc:tqm8xx_defconfig
powerpc:85xx/sbc8548_defconfig
powerpc:83xx/mpc834x_mds_defconfig
powerpc:86xx/sbc8641d_defconfig
s390:defconfig
sparc32:defconfig
sparc64:defconfig
sparc64:allmodconfig
unicore32:defconfig
xtensa:allmodconfig
Qemu tests:
total: 30 pass: 20 fail: 10
Failed tests:
arm:arm_versatile_defconfig
arm:arm_vexpress_defconfig
microblaze:microblaze_defconfig
microblaze:microblazeel_defconfig
powerpc:ppc_book3s_defconfig
powerpc:ppc_book3s_smp_defconfig
sparc32:sparc_defconfig
sparc32:sparc_smp_defconfig
sparc64:sparc_smp_defconfig
sparc64:sparc_nosmp_defconfig
A few of those are other problems, but the majority is due to your patches.
Details are available at http://server.roeck-us.net:8010/builders;
look for the 'testing' column.
Guenter
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 18:50 ` Guenter Roeck
0 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 18:50 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> This patchset moves definition of mm_struct into separate header file.
> It allows to get rid of nr_pmds if PMD page table level is folded.
> We cannot do it with current mm_types.h because we need
> __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> dependencies.
>
> I've done few build tests and looks like it works, but I expect breakage
> on some configuration. Please test.
>
Doesn't look good.
Build results:
total: 134 pass: 63 fail: 71
Failed builds:
arm:s3c2410_defconfig
arm:omap2plus_defconfig
arm:imx_v6_v7_defconfig
arm:ixp4xx_defconfig
arm:u8500_defconfig
arm:multi_v5_defconfig
arm:multi_v7_defconfig
arm:omap1_defconfig
arm:footbridge_defconfig
arm:davinci_all_defconfig
arm:mini2440_defconfig
arm:rpc_defconfig
arm:axm55xx_defconfig
arm:mxs_defconfig
arm:keystone_defconfig
arm:vexpress_defconfig
arm:imx_v4_v5_defconfig
arm:at91_dt_defconfig
arm:s3c6400_defconfig
arm:lpc32xx_defconfig
arm:shmobile_defconfig
arm:nhk8815_defconfig
arm:bcm2835_defconfig
arm:sama5_defconfig
arm:orion5x_defconfig
arm:exynos_defconfig
arm:cm_x2xx_defconfig
arm:s5pv210_defconfig
arm:integrator_defconfig
arm:msm_defconfig
arm:pxa910_defconfig
arm:clps711x_defconfig
avr32:defconfig
avr32:merisc_defconfig
avr32:atngw100mkii_evklcd101_defconfig
cris:defconfig
cris:etrax-100lx_defconfig
cris:allnoconfig
cris:artpec_3_defconfig
cris:etraxfs_defconfig
frv:defconfig
hexagon:defconfig
ia64:defconfig
m68k:defconfig
m68k:allmodconfig
m68k:sun3_defconfig
m68k:m5475evb_defconfig
microblaze:mmu_defconfig
mips:allmodconfig
powerpc:ppc6xx_defconfig
powerpc:mpc83xx_defconfig
powerpc:mpc85xx_defconfig
powerpc:mpc85xx_smp_defconfig
powerpc:tqm8xx_defconfig
powerpc:85xx/sbc8548_defconfig
powerpc:83xx/mpc834x_mds_defconfig
powerpc:86xx/sbc8641d_defconfig
powerpc:ppc6xx_defconfig
powerpc:mpc83xx_defconfig
powerpc:mpc85xx_defconfig
powerpc:mpc85xx_smp_defconfig
powerpc:tqm8xx_defconfig
powerpc:85xx/sbc8548_defconfig
powerpc:83xx/mpc834x_mds_defconfig
powerpc:86xx/sbc8641d_defconfig
s390:defconfig
sparc32:defconfig
sparc64:defconfig
sparc64:allmodconfig
unicore32:defconfig
xtensa:allmodconfig
Qemu tests:
total: 30 pass: 20 fail: 10
Failed tests:
arm:arm_versatile_defconfig
arm:arm_vexpress_defconfig
microblaze:microblaze_defconfig
microblaze:microblazeel_defconfig
powerpc:ppc_book3s_defconfig
powerpc:ppc_book3s_smp_defconfig
sparc32:sparc_defconfig
sparc32:sparc_smp_defconfig
sparc64:sparc_smp_defconfig
sparc64:sparc_nosmp_defconfig
A few of those are other problems, but the majority is due to your patches.
Details are available at http://server.roeck-us.net:8010/builders;
look for the 'testing' column.
Guenter
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
2015-01-28 18:50 ` Guenter Roeck
@ 2015-01-28 20:45 ` Kirill A. Shutemov
-1 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 20:45 UTC (permalink / raw)
To: Guenter Roeck; +Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 10:50:52AM -0800, Guenter Roeck wrote:
> On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> > This patchset moves definition of mm_struct into separate header file.
> > It allows to get rid of nr_pmds if PMD page table level is folded.
> > We cannot do it with current mm_types.h because we need
> > __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> > dependencies.
> >
> > I've done few build tests and looks like it works, but I expect breakage
> > on some configuration. Please test.
> >
> Doesn't look good.
>
> Build results:
> total: 134 pass: 63 fail: 71
> Failed builds:
> arm:s3c2410_defconfig
> arm:omap2plus_defconfig
> arm:imx_v6_v7_defconfig
> arm:ixp4xx_defconfig
> arm:u8500_defconfig
> arm:multi_v5_defconfig
> arm:multi_v7_defconfig
> arm:omap1_defconfig
> arm:footbridge_defconfig
> arm:davinci_all_defconfig
> arm:mini2440_defconfig
> arm:rpc_defconfig
> arm:axm55xx_defconfig
> arm:mxs_defconfig
> arm:keystone_defconfig
> arm:vexpress_defconfig
> arm:imx_v4_v5_defconfig
> arm:at91_dt_defconfig
> arm:s3c6400_defconfig
> arm:lpc32xx_defconfig
> arm:shmobile_defconfig
> arm:nhk8815_defconfig
> arm:bcm2835_defconfig
> arm:sama5_defconfig
> arm:orion5x_defconfig
> arm:exynos_defconfig
> arm:cm_x2xx_defconfig
> arm:s5pv210_defconfig
> arm:integrator_defconfig
> arm:msm_defconfig
> arm:pxa910_defconfig
> arm:clps711x_defconfig
Could you try this for arm?
diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
index f40354198bad..bb4ae035e5e3 100644
--- a/arch/arm/include/asm/pgtable.h
+++ b/arch/arm/include/asm/pgtable.h
@@ -24,9 +24,6 @@
#include <asm/memory.h>
#include <asm/pgtable-hwdef.h>
-
-#include <asm/tlbflush.h>
-
#ifdef CONFIG_ARM_LPAE
#include <asm/pgtable-3level.h>
#else
diff --git a/arch/arm/mm/mm.h b/arch/arm/mm/mm.h
index ce727d47275c..b5e764e0d5a8 100644
--- a/arch/arm/mm/mm.h
+++ b/arch/arm/mm/mm.h
@@ -3,6 +3,7 @@
#include <linux/vmalloc.h>
#include <asm/pgtable.h>
+#include <asm/tlbflush.h>
/* the upper-most page table pointer */
extern pmd_t *top_pmd;
--
Kirill A. Shutemov
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 20:45 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 20:45 UTC (permalink / raw)
To: Guenter Roeck; +Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 10:50:52AM -0800, Guenter Roeck wrote:
> On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> > This patchset moves definition of mm_struct into separate header file.
> > It allows to get rid of nr_pmds if PMD page table level is folded.
> > We cannot do it with current mm_types.h because we need
> > __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> > dependencies.
> >
> > I've done few build tests and looks like it works, but I expect breakage
> > on some configuration. Please test.
> >
> Doesn't look good.
>
> Build results:
> total: 134 pass: 63 fail: 71
> Failed builds:
> arm:s3c2410_defconfig
> arm:omap2plus_defconfig
> arm:imx_v6_v7_defconfig
> arm:ixp4xx_defconfig
> arm:u8500_defconfig
> arm:multi_v5_defconfig
> arm:multi_v7_defconfig
> arm:omap1_defconfig
> arm:footbridge_defconfig
> arm:davinci_all_defconfig
> arm:mini2440_defconfig
> arm:rpc_defconfig
> arm:axm55xx_defconfig
> arm:mxs_defconfig
> arm:keystone_defconfig
> arm:vexpress_defconfig
> arm:imx_v4_v5_defconfig
> arm:at91_dt_defconfig
> arm:s3c6400_defconfig
> arm:lpc32xx_defconfig
> arm:shmobile_defconfig
> arm:nhk8815_defconfig
> arm:bcm2835_defconfig
> arm:sama5_defconfig
> arm:orion5x_defconfig
> arm:exynos_defconfig
> arm:cm_x2xx_defconfig
> arm:s5pv210_defconfig
> arm:integrator_defconfig
> arm:msm_defconfig
> arm:pxa910_defconfig
> arm:clps711x_defconfig
Could you try this for arm?
diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
index f40354198bad..bb4ae035e5e3 100644
--- a/arch/arm/include/asm/pgtable.h
+++ b/arch/arm/include/asm/pgtable.h
@@ -24,9 +24,6 @@
#include <asm/memory.h>
#include <asm/pgtable-hwdef.h>
-
-#include <asm/tlbflush.h>
-
#ifdef CONFIG_ARM_LPAE
#include <asm/pgtable-3level.h>
#else
diff --git a/arch/arm/mm/mm.h b/arch/arm/mm/mm.h
index ce727d47275c..b5e764e0d5a8 100644
--- a/arch/arm/mm/mm.h
+++ b/arch/arm/mm/mm.h
@@ -3,6 +3,7 @@
#include <linux/vmalloc.h>
#include <asm/pgtable.h>
+#include <asm/tlbflush.h>
/* the upper-most page table pointer */
extern pmd_t *top_pmd;
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
2015-01-28 18:50 ` Guenter Roeck
@ 2015-01-28 21:16 ` Kirill A. Shutemov
-1 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 21:16 UTC (permalink / raw)
To: Guenter Roeck; +Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 10:50:52AM -0800, Guenter Roeck wrote:
> On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> > This patchset moves definition of mm_struct into separate header file.
> > It allows to get rid of nr_pmds if PMD page table level is folded.
> > We cannot do it with current mm_types.h because we need
> > __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> > dependencies.
> >
> > I've done few build tests and looks like it works, but I expect breakage
> > on some configuration. Please test.
> >
> Doesn't look good.
>
> Build results:
> total: 134 pass: 63 fail: 71
> Failed builds:
> avr32:defconfig
> avr32:merisc_defconfig
> avr32:atngw100mkii_evklcd101_defconfig
Fixlet for AVR32:
diff --git a/arch/avr32/include/asm/pgtable.h b/arch/avr32/include/asm/pgtable.h
index 35800664076e..3af39532b25b 100644
--- a/arch/avr32/include/asm/pgtable.h
+++ b/arch/avr32/include/asm/pgtable.h
@@ -10,11 +10,6 @@
#include <asm/addrspace.h>
-#ifndef __ASSEMBLY__
-#include <linux/sched.h>
-
-#endif /* !__ASSEMBLY__ */
-
/*
* Use two-level page tables just as the i386 (without PAE)
*/
diff --git a/arch/avr32/mm/tlb.c b/arch/avr32/mm/tlb.c
index 0da23109f817..964130f8f89d 100644
--- a/arch/avr32/mm/tlb.c
+++ b/arch/avr32/mm/tlb.c
@@ -8,6 +8,7 @@
* published by the Free Software Foundation.
*/
#include <linux/mm.h>
+#include <linux/sched.h>
#include <asm/mmu_context.h>
--
Kirill A. Shutemov
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 21:16 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-28 21:16 UTC (permalink / raw)
To: Guenter Roeck; +Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 10:50:52AM -0800, Guenter Roeck wrote:
> On Wed, Jan 28, 2015 at 03:17:40PM +0200, Kirill A. Shutemov wrote:
> > This patchset moves definition of mm_struct into separate header file.
> > It allows to get rid of nr_pmds if PMD page table level is folded.
> > We cannot do it with current mm_types.h because we need
> > __PAGETABLE_PMD_FOLDED from <asm/pgtable.h> which creates circular
> > dependencies.
> >
> > I've done few build tests and looks like it works, but I expect breakage
> > on some configuration. Please test.
> >
> Doesn't look good.
>
> Build results:
> total: 134 pass: 63 fail: 71
> Failed builds:
> avr32:defconfig
> avr32:merisc_defconfig
> avr32:atngw100mkii_evklcd101_defconfig
Fixlet for AVR32:
diff --git a/arch/avr32/include/asm/pgtable.h b/arch/avr32/include/asm/pgtable.h
index 35800664076e..3af39532b25b 100644
--- a/arch/avr32/include/asm/pgtable.h
+++ b/arch/avr32/include/asm/pgtable.h
@@ -10,11 +10,6 @@
#include <asm/addrspace.h>
-#ifndef __ASSEMBLY__
-#include <linux/sched.h>
-
-#endif /* !__ASSEMBLY__ */
-
/*
* Use two-level page tables just as the i386 (without PAE)
*/
diff --git a/arch/avr32/mm/tlb.c b/arch/avr32/mm/tlb.c
index 0da23109f817..964130f8f89d 100644
--- a/arch/avr32/mm/tlb.c
+++ b/arch/avr32/mm/tlb.c
@@ -8,6 +8,7 @@
* published by the Free Software Foundation.
*/
#include <linux/mm.h>
+#include <linux/sched.h>
#include <asm/mmu_context.h>
--
Kirill A. Shutemov
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
2015-01-28 20:45 ` Kirill A. Shutemov
@ 2015-01-28 21:18 ` Guenter Roeck
-1 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 21:18 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 10:45:44PM +0200, Kirill A. Shutemov wrote:
[ ... ]
> Could you try this for arm?
>
> diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
> index f40354198bad..bb4ae035e5e3 100644
Applied. Test will take a couple of hours to complete since
the testbed is busy right now. You can check the status anytime
at http://server.roeck-us.net:8010/builders.
Guenter
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 21:18 ` Guenter Roeck
0 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 21:18 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 10:45:44PM +0200, Kirill A. Shutemov wrote:
[ ... ]
> Could you try this for arm?
>
> diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
> index f40354198bad..bb4ae035e5e3 100644
Applied. Test will take a couple of hours to complete since
the testbed is busy right now. You can check the status anytime
at http://server.roeck-us.net:8010/builders.
Guenter
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 2/4] mm: split up mm_struct to separate header file
2015-01-28 13:17 ` Kirill A. Shutemov
@ 2015-01-29 0:30 ` Kirill A. Shutemov
-1 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-29 0:30 UTC (permalink / raw)
To: Guenter Roeck; +Cc: Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 03:17:42PM +0200, Kirill A. Shutemov wrote:
> We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
> pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
> <asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
> definition.
>
> This patch move mm_struct definition into separate header file in order
> to fix circular header dependencies.
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Guenter, below is update for the patch. It doesn't fix all the issues, but
you should see an improvement. I'll continue with this tomorrow.
BTW, any idea where I can get hexagon cross compiler?
>From 8bb23d8b6b4b81485294966193a262f918a0a195 Mon Sep 17 00:00:00 2001
From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Date: Wed, 28 Jan 2015 13:30:24 +0200
Subject: [PATCH] mm: split up mm_struct to separate header file
We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
<asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
definition.
This patch move mm_struct definition into separate header file in order
to fix circular header dependencies.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
---
arch/arm/include/asm/pgtable-3level.h | 9 +-
arch/arm/include/asm/pgtable.h | 3 -
arch/arm/kernel/ftrace.c | 1 +
arch/arm/mm/mm.h | 1 +
arch/arm64/kernel/efi.c | 1 +
arch/avr32/include/asm/pgtable.h | 5 -
arch/avr32/mm/tlb.c | 1 +
arch/c6x/kernel/dma.c | 1 -
arch/cris/arch-v10/mm/init.c | 1 +
arch/cris/include/asm/pgtable.h | 7 +-
arch/cris/include/asm/tlbflush.h | 1 +
arch/frv/include/asm/pgtable.h | 33 +----
arch/frv/mm/pgalloc.c | 31 +++++
arch/ia64/include/asm/pgtable.h | 8 +-
arch/ia64/kernel/topology.c | 1 +
arch/m68k/include/asm/cacheflush_mm.h | 1 +
arch/m68k/include/asm/motorola_pgtable.h | 6 +-
arch/m68k/include/asm/pgtable_mm.h | 1 -
arch/microblaze/include/asm/pgtable.h | 1 -
arch/microblaze/kernel/kgdb.c | 1 +
arch/powerpc/include/asm/pgtable-ppc32.h | 1 -
arch/s390/include/asm/pgtable.h | 1 +
arch/sparc/include/asm/pgtable_32.h | 2 +-
arch/x86/include/asm/mmu_context.h | 1 +
arch/x86/include/asm/pgtable.h | 15 +--
drivers/iommu/amd_iommu_v2.c | 1 +
drivers/macintosh/adb-iop.c | 1 +
drivers/staging/android/ion/ion.c | 1 -
include/linux/bootmem.h | 1 +
include/linux/mm.h | 1 +
include/linux/mm_struct.h | 214 ++++++++++++++++++++++++++++++
include/linux/mm_types.h | 218 +------------------------------
include/linux/mmu_notifier.h | 1 +
include/linux/sched.h | 1 +
mm/init-mm.c | 1 +
mm/kmemcheck.c | 1 -
36 files changed, 287 insertions(+), 288 deletions(-)
create mode 100644 include/linux/mm_struct.h
diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h
index 370684dcc2da..7874de5ab7dd 100644
--- a/arch/arm/include/asm/pgtable-3level.h
+++ b/arch/arm/include/asm/pgtable-3level.h
@@ -269,7 +269,7 @@ static inline pmd_t pmd_modify(pmd_t pmd, pgprot_t newprot)
return pmd;
}
-static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
+static inline void __set_pmd_at(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd)
{
BUG_ON(addr >= TASK_SIZE);
@@ -284,9 +284,14 @@ static inline void set_pmd_at(struct mm_struct *mm, unsigned long addr,
pmd_val(pmd) |= PMD_SECT_AP2;
*pmdp = __pmd(pmd_val(pmd) | PMD_SECT_nG);
- flush_pmd_entry(pmdp);
}
+#define set_pmd_at(mm, addr, pmdp, pmd) \
+ do { \
+ __set_pmd_at(mm, addr, pmdp, pmd) \
+ flush_pmd_entry(pmdp) \
+ } while(0)
+
static inline int has_transparent_hugepage(void)
{
return 1;
diff --git a/arch/arm/include/asm/pgtable.h b/arch/arm/include/asm/pgtable.h
index f40354198bad..bb4ae035e5e3 100644
--- a/arch/arm/include/asm/pgtable.h
+++ b/arch/arm/include/asm/pgtable.h
@@ -24,9 +24,6 @@
#include <asm/memory.h>
#include <asm/pgtable-hwdef.h>
-
-#include <asm/tlbflush.h>
-
#ifdef CONFIG_ARM_LPAE
#include <asm/pgtable-3level.h>
#else
diff --git a/arch/arm/kernel/ftrace.c b/arch/arm/kernel/ftrace.c
index 709ee1d6d4df..b6f6c11e67a6 100644
--- a/arch/arm/kernel/ftrace.c
+++ b/arch/arm/kernel/ftrace.c
@@ -21,6 +21,7 @@
#include <asm/opcodes.h>
#include <asm/ftrace.h>
#include <asm/insn.h>
+#include <asm/tlbflush.h>
#ifdef CONFIG_THUMB2_KERNEL
#define NOP 0xf85deb04 /* pop.w {lr} */
diff --git a/arch/arm/mm/mm.h b/arch/arm/mm/mm.h
index ce727d47275c..b5e764e0d5a8 100644
--- a/arch/arm/mm/mm.h
+++ b/arch/arm/mm/mm.h
@@ -3,6 +3,7 @@
#include <linux/vmalloc.h>
#include <asm/pgtable.h>
+#include <asm/tlbflush.h>
/* the upper-most page table pointer */
extern pmd_t *top_pmd;
diff --git a/arch/arm64/kernel/efi.c b/arch/arm64/kernel/efi.c
index b42c7b480e1e..fbf0a6d6f691 100644
--- a/arch/arm64/kernel/efi.c
+++ b/arch/arm64/kernel/efi.c
@@ -17,6 +17,7 @@
#include <linux/export.h>
#include <linux/memblock.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/bootmem.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
diff --git a/arch/avr32/include/asm/pgtable.h b/arch/avr32/include/asm/pgtable.h
index 35800664076e..3af39532b25b 100644
--- a/arch/avr32/include/asm/pgtable.h
+++ b/arch/avr32/include/asm/pgtable.h
@@ -10,11 +10,6 @@
#include <asm/addrspace.h>
-#ifndef __ASSEMBLY__
-#include <linux/sched.h>
-
-#endif /* !__ASSEMBLY__ */
-
/*
* Use two-level page tables just as the i386 (without PAE)
*/
diff --git a/arch/avr32/mm/tlb.c b/arch/avr32/mm/tlb.c
index 0da23109f817..964130f8f89d 100644
--- a/arch/avr32/mm/tlb.c
+++ b/arch/avr32/mm/tlb.c
@@ -8,6 +8,7 @@
* published by the Free Software Foundation.
*/
#include <linux/mm.h>
+#include <linux/sched.h>
#include <asm/mmu_context.h>
diff --git a/arch/c6x/kernel/dma.c b/arch/c6x/kernel/dma.c
index ab7b12de144d..5a489f1eabbd 100644
--- a/arch/c6x/kernel/dma.c
+++ b/arch/c6x/kernel/dma.c
@@ -9,7 +9,6 @@
#include <linux/module.h>
#include <linux/dma-mapping.h>
#include <linux/mm.h>
-#include <linux/mm_types.h>
#include <linux/scatterlist.h>
#include <asm/cacheflush.h>
diff --git a/arch/cris/arch-v10/mm/init.c b/arch/cris/arch-v10/mm/init.c
index e7f8066105aa..800faf7f97e7 100644
--- a/arch/cris/arch-v10/mm/init.c
+++ b/arch/cris/arch-v10/mm/init.c
@@ -6,6 +6,7 @@
#include <linux/init.h>
#include <linux/bootmem.h>
#include <linux/mm.h>
+#include <linux/sched.h>
#include <asm/pgtable.h>
#include <asm/page.h>
#include <asm/types.h>
diff --git a/arch/cris/include/asm/pgtable.h b/arch/cris/include/asm/pgtable.h
index ceefc314d64d..88fd211af681 100644
--- a/arch/cris/include/asm/pgtable.h
+++ b/arch/cris/include/asm/pgtable.h
@@ -9,7 +9,7 @@
#include <asm-generic/pgtable-nopmd.h>
#ifndef __ASSEMBLY__
-#include <linux/sched.h>
+#include <linux/mm_types.h>
#include <asm/mmu.h>
#endif
#include <arch/pgtable.h>
@@ -232,10 +232,7 @@ static inline void pmd_set(pmd_t * pmdp, pte_t * ptep)
#define pgd_index(address) (((address) >> PGDIR_SHIFT) & (PTRS_PER_PGD-1))
/* to find an entry in a page-table-directory */
-static inline pgd_t * pgd_offset(const struct mm_struct *mm, unsigned long address)
-{
- return mm->pgd + pgd_index(address);
-}
+#define pgd_offset(mm, address) ((mm)->pgd + pgd_index(address))
/* to find an entry in a kernel page-table-directory */
#define pgd_offset_k(address) pgd_offset(&init_mm, address)
diff --git a/arch/cris/include/asm/tlbflush.h b/arch/cris/include/asm/tlbflush.h
index 20697e7ef4f2..149c8aa1d58a 100644
--- a/arch/cris/include/asm/tlbflush.h
+++ b/arch/cris/include/asm/tlbflush.h
@@ -2,6 +2,7 @@
#define _CRIS_TLBFLUSH_H
#include <linux/mm.h>
+#include <linux/sched.h>
#include <asm/processor.h>
#include <asm/pgtable.h>
#include <asm/pgalloc.h>
diff --git a/arch/frv/include/asm/pgtable.h b/arch/frv/include/asm/pgtable.h
index 93bcf2abd1a1..0c1c9ead8510 100644
--- a/arch/frv/include/asm/pgtable.h
+++ b/arch/frv/include/asm/pgtable.h
@@ -25,7 +25,6 @@
#include <linux/slab.h>
#include <linux/list.h>
#include <linux/spinlock.h>
-#include <linux/sched.h>
struct vm_area_struct;
#endif
@@ -477,36 +476,8 @@ static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
#define __HAVE_ARCH_PTE_SAME
#include <asm-generic/pgtable.h>
-/*
- * preload information about a newly instantiated PTE into the SCR0/SCR1 PGE cache
- */
-static inline void update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
-{
- struct mm_struct *mm;
- unsigned long ampr;
-
- mm = current->mm;
- if (mm) {
- pgd_t *pge = pgd_offset(mm, address);
- pud_t *pue = pud_offset(pge, address);
- pmd_t *pme = pmd_offset(pue, address);
-
- ampr = pme->ste[0] & 0xffffff00;
- ampr |= xAMPRx_L | xAMPRx_SS_16Kb | xAMPRx_S | xAMPRx_C |
- xAMPRx_V;
- } else {
- address = ULONG_MAX;
- ampr = 0;
- }
-
- asm volatile("movgs %0,scr0\n"
- "movgs %0,scr1\n"
- "movgs %1,dampr4\n"
- "movgs %1,dampr5\n"
- :
- : "r"(address), "r"(ampr)
- );
-}
+void update_mmu_cache(struct vm_area_struct *vma, unsigned long address,
+ pte_t *ptep);
#ifdef CONFIG_PROC_FS
extern char *proc_pid_status_frv_cxnr(struct mm_struct *mm, char *buffer);
diff --git a/arch/frv/mm/pgalloc.c b/arch/frv/mm/pgalloc.c
index 41907d25ed38..fc18d208030b 100644
--- a/arch/frv/mm/pgalloc.c
+++ b/arch/frv/mm/pgalloc.c
@@ -155,3 +155,34 @@ void check_pgt_cache(void)
quicklist_trim(0, pgd_dtor, 25, 16);
}
+/*
+ * preload information about a newly instantiated PTE into the SCR0/SCR1 PGE cache
+ */
+void update_mmu_cache(struct vm_area_struct *vma, unsigned long address,
+ pte_t *ptep)
+{
+ struct mm_struct *mm;
+ unsigned long ampr;
+
+ mm = current->mm;
+ if (mm) {
+ pgd_t *pge = pgd_offset(mm, address);
+ pud_t *pue = pud_offset(pge, address);
+ pmd_t *pme = pmd_offset(pue, address);
+
+ ampr = pme->ste[0] & 0xffffff00;
+ ampr |= xAMPRx_L | xAMPRx_SS_16Kb | xAMPRx_S | xAMPRx_C |
+ xAMPRx_V;
+ } else {
+ address = ULONG_MAX;
+ ampr = 0;
+ }
+
+ asm volatile("movgs %0,scr0\n"
+ "movgs %0,scr1\n"
+ "movgs %1,dampr4\n"
+ "movgs %1,dampr5\n"
+ :
+ : "r"(address), "r"(ampr)
+ );
+}
diff --git a/arch/ia64/include/asm/pgtable.h b/arch/ia64/include/asm/pgtable.h
index 7b6f8801df57..66bbf5618631 100644
--- a/arch/ia64/include/asm/pgtable.h
+++ b/arch/ia64/include/asm/pgtable.h
@@ -147,10 +147,8 @@
# ifndef __ASSEMBLY__
-#include <linux/sched.h> /* for mm_struct */
#include <linux/bitops.h>
#include <asm/cacheflush.h>
-#include <asm/mmu_context.h>
/*
* Next come the mappings that determine how mmap() protection bits
@@ -368,11 +366,7 @@ pgd_index (unsigned long address)
/* The offset in the 1-level directory is given by the 3 region bits
(61..63) and the level-1 bits. */
-static inline pgd_t*
-pgd_offset (const struct mm_struct *mm, unsigned long address)
-{
- return mm->pgd + pgd_index(address);
-}
+#define pgd_offset(mm, addr) ((mm)->pgd + pgd_index(addr))
/* In the kernel's mapped region we completely ignore the region number
(since we know it's in region number 5). */
diff --git a/arch/ia64/kernel/topology.c b/arch/ia64/kernel/topology.c
index 965ab42fabb0..4009738811f7 100644
--- a/arch/ia64/kernel/topology.c
+++ b/arch/ia64/kernel/topology.c
@@ -16,6 +16,7 @@
#include <linux/cpu.h>
#include <linux/kernel.h>
#include <linux/mm.h>
+#include <linux/sched.h>
#include <linux/node.h>
#include <linux/slab.h>
#include <linux/init.h>
diff --git a/arch/m68k/include/asm/cacheflush_mm.h b/arch/m68k/include/asm/cacheflush_mm.h
index fa2c3d681d84..e91674ddf62a 100644
--- a/arch/m68k/include/asm/cacheflush_mm.h
+++ b/arch/m68k/include/asm/cacheflush_mm.h
@@ -2,6 +2,7 @@
#define _M68K_CACHEFLUSH_H
#include <linux/mm.h>
+#include <linux/sched.h>
#ifdef CONFIG_COLDFIRE
#include <asm/mcfsim.h>
#endif
diff --git a/arch/m68k/include/asm/motorola_pgtable.h b/arch/m68k/include/asm/motorola_pgtable.h
index 0085aab80e5a..8074c399a048 100644
--- a/arch/m68k/include/asm/motorola_pgtable.h
+++ b/arch/m68k/include/asm/motorola_pgtable.h
@@ -192,11 +192,7 @@ static inline pte_t pte_mkspecial(pte_t pte) { return pte; }
#define pgd_index(address) ((address) >> PGDIR_SHIFT)
/* to find an entry in a page-table-directory */
-static inline pgd_t *pgd_offset(const struct mm_struct *mm,
- unsigned long address)
-{
- return mm->pgd + pgd_index(address);
-}
+#define pgd_offset(mm, address) ((mm)->pgd + pgd_index(address))
#define swapper_pg_dir kernel_pg_dir
extern pgd_t kernel_pg_dir[128];
diff --git a/arch/m68k/include/asm/pgtable_mm.h b/arch/m68k/include/asm/pgtable_mm.h
index 28a145bfbb71..575b39fd31a8 100644
--- a/arch/m68k/include/asm/pgtable_mm.h
+++ b/arch/m68k/include/asm/pgtable_mm.h
@@ -7,7 +7,6 @@
#ifndef __ASSEMBLY__
#include <asm/processor.h>
-#include <linux/sched.h>
#include <linux/threads.h>
/*
diff --git a/arch/microblaze/include/asm/pgtable.h b/arch/microblaze/include/asm/pgtable.h
index e53b8532353c..101cc7b4440a 100644
--- a/arch/microblaze/include/asm/pgtable.h
+++ b/arch/microblaze/include/asm/pgtable.h
@@ -66,7 +66,6 @@ extern int mem_init_done;
#ifdef __KERNEL__
#ifndef __ASSEMBLY__
-#include <linux/sched.h>
#include <linux/threads.h>
#include <asm/processor.h> /* For TASK_SIZE */
#include <asm/mmu.h>
diff --git a/arch/microblaze/kernel/kgdb.c b/arch/microblaze/kernel/kgdb.c
index 8736af5806ae..89e12562b30d 100644
--- a/arch/microblaze/kernel/kgdb.c
+++ b/arch/microblaze/kernel/kgdb.c
@@ -10,6 +10,7 @@
#include <linux/kdebug.h>
#include <linux/irq.h>
#include <linux/io.h>
+#include <linux/sched.h>
#include <asm/cacheflush.h>
#include <asm/asm-offsets.h>
#include <asm/kgdb.h>
diff --git a/arch/powerpc/include/asm/pgtable-ppc32.h b/arch/powerpc/include/asm/pgtable-ppc32.h
index cc3e621aa830..32632aa7e1c7 100644
--- a/arch/powerpc/include/asm/pgtable-ppc32.h
+++ b/arch/powerpc/include/asm/pgtable-ppc32.h
@@ -4,7 +4,6 @@
#include <asm-generic/pgtable-nopmd.h>
#ifndef __ASSEMBLY__
-#include <linux/sched.h>
#include <linux/threads.h>
#include <asm/io.h> /* For sub-arch specific PPC_PIN_SIZE */
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index fbb5ee3ae57c..578eb098b1be 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -29,6 +29,7 @@
#ifndef __ASSEMBLY__
#include <linux/sched.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/page-flags.h>
#include <linux/radix-tree.h>
#include <asm/bug.h>
diff --git a/arch/sparc/include/asm/pgtable_32.h b/arch/sparc/include/asm/pgtable_32.h
index f06b36a00a3b..91b963a887b7 100644
--- a/arch/sparc/include/asm/pgtable_32.h
+++ b/arch/sparc/include/asm/pgtable_32.h
@@ -14,7 +14,7 @@
#include <asm-generic/4level-fixup.h>
#include <linux/spinlock.h>
-#include <linux/swap.h>
+#include <linux/mm_types.h>
#include <asm/types.h>
#include <asm/pgtsrmmu.h>
#include <asm/vaddrs.h>
diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h
index 4b75d591eb5e..78a87a30ec50 100644
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -4,6 +4,7 @@
#include <asm/desc.h>
#include <linux/atomic.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <trace/events/tlb.h>
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 9d0ade00923e..594d09d07bb4 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -445,18 +445,9 @@ static inline int pte_present(pte_t a)
return pte_flags(a) & (_PAGE_PRESENT | _PAGE_PROTNONE);
}
-#define pte_accessible pte_accessible
-static inline bool pte_accessible(struct mm_struct *mm, pte_t a)
-{
- if (pte_flags(a) & _PAGE_PRESENT)
- return true;
-
- if ((pte_flags(a) & _PAGE_PROTNONE) &&
- mm_tlb_flush_pending(mm))
- return true;
-
- return false;
-}
+#define pte_accessible(mm, pte) \
+ (pte_flags(pte) & _PAGE_PRESENT) || \
+ ((pte_flags(pte) & _PAGE_PROTNONE) && mm_tlb_flush_pending(mm))
static inline int pte_hidden(pte_t pte)
{
diff --git a/drivers/iommu/amd_iommu_v2.c b/drivers/iommu/amd_iommu_v2.c
index 90f70d0e1141..d49cd95eaf7b 100644
--- a/drivers/iommu/amd_iommu_v2.c
+++ b/drivers/iommu/amd_iommu_v2.c
@@ -19,6 +19,7 @@
#include <linux/mmu_notifier.h>
#include <linux/amd-iommu.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/profile.h>
#include <linux/module.h>
#include <linux/sched.h>
diff --git a/drivers/macintosh/adb-iop.c b/drivers/macintosh/adb-iop.c
index f5f4da3d0b67..8916f1ad5f37 100644
--- a/drivers/macintosh/adb-iop.c
+++ b/drivers/macintosh/adb-iop.c
@@ -18,6 +18,7 @@
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/proc_fs.h>
+#include <linux/sched.h>
#include <asm/macintosh.h>
#include <asm/macints.h>
diff --git a/drivers/staging/android/ion/ion.c b/drivers/staging/android/ion/ion.c
index b8f1c491553e..1cb3c88f6423 100644
--- a/drivers/staging/android/ion/ion.c
+++ b/drivers/staging/android/ion/ion.c
@@ -27,7 +27,6 @@
#include <linux/miscdevice.h>
#include <linux/export.h>
#include <linux/mm.h>
-#include <linux/mm_types.h>
#include <linux/rbtree.h>
#include <linux/slab.h>
#include <linux/seq_file.h>
diff --git a/include/linux/bootmem.h b/include/linux/bootmem.h
index 0995c2de8162..4d90afe3fbb4 100644
--- a/include/linux/bootmem.h
+++ b/include/linux/bootmem.h
@@ -7,6 +7,7 @@
#include <linux/mmzone.h>
#include <linux/mm_types.h>
#include <asm/dma.h>
+#include <asm/mmu.h>
/*
* simple boot-time physical memory area allocator.
diff --git a/include/linux/mm.h b/include/linux/mm.h
index b976d9ffbcd6..543e9723d441 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -14,6 +14,7 @@
#include <linux/atomic.h>
#include <linux/debug_locks.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/range.h>
#include <linux/pfn.h>
#include <linux/bit_spinlock.h>
diff --git a/include/linux/mm_struct.h b/include/linux/mm_struct.h
new file mode 100644
index 000000000000..0a233c232a39
--- /dev/null
+++ b/include/linux/mm_struct.h
@@ -0,0 +1,214 @@
+#ifndef _LINUX_MM_STRUCT_H
+#define _LINUX_MM_STRUCT_H
+
+#include <linux/auxvec.h>
+#include <linux/rbtree.h>
+#include <linux/rwsem.h>
+#include <linux/types.h>
+#include <linux/uprobes.h>
+
+#include <asm/mmu.h>
+
+struct kioctx_table;
+struct vm_area_struct;
+
+#ifndef AT_VECTOR_SIZE_ARCH
+#define AT_VECTOR_SIZE_ARCH 0
+#endif
+#define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1))
+
+enum {
+ MM_FILEPAGES,
+ MM_ANONPAGES,
+ MM_SWAPENTS,
+ NR_MM_COUNTERS
+};
+
+#if NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS && defined(CONFIG_MMU)
+#define SPLIT_RSS_COUNTING
+/* per-thread cached information, */
+struct task_rss_stat {
+ int events; /* for synchronization threshold */
+ int count[NR_MM_COUNTERS];
+};
+#endif /* USE_SPLIT_PTE_PTLOCKS */
+
+struct mm_rss_stat {
+ atomic_long_t count[NR_MM_COUNTERS];
+};
+
+struct mm_struct {
+ struct vm_area_struct *mmap; /* list of VMAs */
+ struct rb_root mm_rb;
+ u32 vmacache_seqnum; /* per-thread vmacache */
+#ifdef CONFIG_MMU
+ unsigned long (*get_unmapped_area) (struct file *filp,
+ unsigned long addr, unsigned long len,
+ unsigned long pgoff, unsigned long flags);
+#endif
+ unsigned long mmap_base; /* base of mmap area */
+ unsigned long mmap_legacy_base; /* base of mmap area in bottom-up allocations */
+ unsigned long task_size; /* size of task vm space */
+ unsigned long highest_vm_end; /* highest vma end address */
+ pgd_t * pgd;
+ atomic_t mm_users; /* How many users with user space? */
+ atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
+ atomic_long_t nr_ptes; /* PTE page table pages */
+ atomic_long_t nr_pmds; /* PMD page table pages */
+ int map_count; /* number of VMAs */
+
+ spinlock_t page_table_lock; /* Protects page tables and some counters */
+ struct rw_semaphore mmap_sem;
+
+ struct list_head mmlist; /* List of maybe swapped mm's. These are globally strung
+ * together off init_mm.mmlist, and are protected
+ * by mmlist_lock
+ */
+
+
+ unsigned long hiwater_rss; /* High-watermark of RSS usage */
+ unsigned long hiwater_vm; /* High-water virtual memory usage */
+
+ unsigned long total_vm; /* Total pages mapped */
+ unsigned long locked_vm; /* Pages that have PG_mlocked set */
+ unsigned long pinned_vm; /* Refcount permanently increased */
+ unsigned long shared_vm; /* Shared pages (files) */
+ unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE */
+ unsigned long stack_vm; /* VM_GROWSUP/DOWN */
+ unsigned long def_flags;
+ unsigned long start_code, end_code, start_data, end_data;
+ unsigned long start_brk, brk, start_stack;
+ unsigned long arg_start, arg_end, env_start, env_end;
+
+ unsigned long saved_auxv[AT_VECTOR_SIZE]; /* for /proc/PID/auxv */
+
+ /*
+ * Special counters, in some configurations protected by the
+ * page_table_lock, in other configurations by being atomic.
+ */
+ struct mm_rss_stat rss_stat;
+
+ struct linux_binfmt *binfmt;
+
+ cpumask_var_t cpu_vm_mask_var;
+
+ /* Architecture-specific MM context */
+ mm_context_t context;
+
+ unsigned long flags; /* Must use atomic bitops to access the bits */
+
+ struct core_state *core_state; /* coredumping support */
+#ifdef CONFIG_AIO
+ spinlock_t ioctx_lock;
+ struct kioctx_table __rcu *ioctx_table;
+#endif
+#ifdef CONFIG_MEMCG
+ /*
+ * "owner" points to a task that is regarded as the canonical
+ * user/owner of this mm. All of the following must be true in
+ * order for it to be changed:
+ *
+ * current == mm->owner
+ * current->mm != mm
+ * new_owner->mm == mm
+ * new_owner->alloc_lock is held
+ */
+ struct task_struct __rcu *owner;
+#endif
+
+ /* store ref to file /proc/<pid>/exe symlink points to */
+ struct file *exe_file;
+#ifdef CONFIG_MMU_NOTIFIER
+ struct mmu_notifier_mm *mmu_notifier_mm;
+#endif
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
+ pgtable_t pmd_huge_pte; /* protected by page_table_lock */
+#endif
+#ifdef CONFIG_CPUMASK_OFFSTACK
+ struct cpumask cpumask_allocation;
+#endif
+#ifdef CONFIG_NUMA_BALANCING
+ /*
+ * numa_next_scan is the next time that the PTEs will be marked
+ * pte_numa. NUMA hinting faults will gather statistics and migrate
+ * pages to new nodes if necessary.
+ */
+ unsigned long numa_next_scan;
+
+ /* Restart point for scanning and setting pte_numa */
+ unsigned long numa_scan_offset;
+
+ /* numa_scan_seq prevents two threads setting pte_numa */
+ int numa_scan_seq;
+#endif
+#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
+ /*
+ * An operation with batched TLB flushing is going on. Anything that
+ * can move process memory needs to flush the TLB when moving a
+ * PROT_NONE or PROT_NUMA mapped page.
+ */
+ bool tlb_flush_pending;
+#endif
+ struct uprobes_state uprobes_state;
+#ifdef CONFIG_X86_INTEL_MPX
+ /* address of the bounds directory */
+ void __user *bd_addr;
+#endif
+};
+
+static inline void mm_init_cpumask(struct mm_struct *mm)
+{
+#ifdef CONFIG_CPUMASK_OFFSTACK
+ mm->cpu_vm_mask_var = &mm->cpumask_allocation;
+#endif
+ cpumask_clear(mm->cpu_vm_mask_var);
+}
+
+/* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
+static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
+{
+ return mm->cpu_vm_mask_var;
+}
+
+#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
+/*
+ * Memory barriers to keep this state in sync are graciously provided by
+ * the page table locks, outside of which no page table modifications happen.
+ * The barriers below prevent the compiler from re-ordering the instructions
+ * around the memory barriers that are already present in the code.
+ */
+static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
+{
+ barrier();
+ return mm->tlb_flush_pending;
+}
+static inline void set_tlb_flush_pending(struct mm_struct *mm)
+{
+ mm->tlb_flush_pending = true;
+
+ /*
+ * Guarantee that the tlb_flush_pending store does not leak into the
+ * critical section updating the page tables
+ */
+ smp_mb__before_spinlock();
+}
+/* Clearing is done after a TLB flush, which also provides a barrier. */
+static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+{
+ barrier();
+ mm->tlb_flush_pending = false;
+}
+#else
+static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
+{
+ return false;
+}
+static inline void set_tlb_flush_pending(struct mm_struct *mm)
+{
+}
+static inline void clear_tlb_flush_pending(struct mm_struct *mm)
+{
+}
+#endif
+
+#endif
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 5dfdd5ed5254..80797b1c1e26 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1,24 +1,16 @@
#ifndef _LINUX_MM_TYPES_H
#define _LINUX_MM_TYPES_H
-#include <linux/auxvec.h>
-#include <linux/types.h>
-#include <linux/threads.h>
-#include <linux/list.h>
-#include <linux/spinlock.h>
-#include <linux/rbtree.h>
-#include <linux/rwsem.h>
#include <linux/completion.h>
#include <linux/cpumask.h>
-#include <linux/uprobes.h>
+#include <linux/list.h>
#include <linux/page-flags-layout.h>
-#include <asm/page.h>
-#include <asm/mmu.h>
+#include <linux/rbtree.h>
+#include <linux/spinlock.h>
+#include <linux/threads.h>
+#include <linux/types.h>
-#ifndef AT_VECTOR_SIZE_ARCH
-#define AT_VECTOR_SIZE_ARCH 0
-#endif
-#define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1))
+#include <asm/page.h>
struct address_space;
struct mem_cgroup;
@@ -326,203 +318,7 @@ struct core_state {
struct completion startup;
};
-enum {
- MM_FILEPAGES,
- MM_ANONPAGES,
- MM_SWAPENTS,
- NR_MM_COUNTERS
-};
-
-#if USE_SPLIT_PTE_PTLOCKS && defined(CONFIG_MMU)
-#define SPLIT_RSS_COUNTING
-/* per-thread cached information, */
-struct task_rss_stat {
- int events; /* for synchronization threshold */
- int count[NR_MM_COUNTERS];
-};
-#endif /* USE_SPLIT_PTE_PTLOCKS */
-
-struct mm_rss_stat {
- atomic_long_t count[NR_MM_COUNTERS];
-};
-
-struct kioctx_table;
-struct mm_struct {
- struct vm_area_struct *mmap; /* list of VMAs */
- struct rb_root mm_rb;
- u32 vmacache_seqnum; /* per-thread vmacache */
-#ifdef CONFIG_MMU
- unsigned long (*get_unmapped_area) (struct file *filp,
- unsigned long addr, unsigned long len,
- unsigned long pgoff, unsigned long flags);
-#endif
- unsigned long mmap_base; /* base of mmap area */
- unsigned long mmap_legacy_base; /* base of mmap area in bottom-up allocations */
- unsigned long task_size; /* size of task vm space */
- unsigned long highest_vm_end; /* highest vma end address */
- pgd_t * pgd;
- atomic_t mm_users; /* How many users with user space? */
- atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
- atomic_long_t nr_ptes; /* PTE page table pages */
- atomic_long_t nr_pmds; /* PMD page table pages */
- int map_count; /* number of VMAs */
-
- spinlock_t page_table_lock; /* Protects page tables and some counters */
- struct rw_semaphore mmap_sem;
-
- struct list_head mmlist; /* List of maybe swapped mm's. These are globally strung
- * together off init_mm.mmlist, and are protected
- * by mmlist_lock
- */
-
-
- unsigned long hiwater_rss; /* High-watermark of RSS usage */
- unsigned long hiwater_vm; /* High-water virtual memory usage */
-
- unsigned long total_vm; /* Total pages mapped */
- unsigned long locked_vm; /* Pages that have PG_mlocked set */
- unsigned long pinned_vm; /* Refcount permanently increased */
- unsigned long shared_vm; /* Shared pages (files) */
- unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE */
- unsigned long stack_vm; /* VM_GROWSUP/DOWN */
- unsigned long def_flags;
- unsigned long start_code, end_code, start_data, end_data;
- unsigned long start_brk, brk, start_stack;
- unsigned long arg_start, arg_end, env_start, env_end;
-
- unsigned long saved_auxv[AT_VECTOR_SIZE]; /* for /proc/PID/auxv */
-
- /*
- * Special counters, in some configurations protected by the
- * page_table_lock, in other configurations by being atomic.
- */
- struct mm_rss_stat rss_stat;
-
- struct linux_binfmt *binfmt;
-
- cpumask_var_t cpu_vm_mask_var;
-
- /* Architecture-specific MM context */
- mm_context_t context;
-
- unsigned long flags; /* Must use atomic bitops to access the bits */
-
- struct core_state *core_state; /* coredumping support */
-#ifdef CONFIG_AIO
- spinlock_t ioctx_lock;
- struct kioctx_table __rcu *ioctx_table;
-#endif
-#ifdef CONFIG_MEMCG
- /*
- * "owner" points to a task that is regarded as the canonical
- * user/owner of this mm. All of the following must be true in
- * order for it to be changed:
- *
- * current == mm->owner
- * current->mm != mm
- * new_owner->mm == mm
- * new_owner->alloc_lock is held
- */
- struct task_struct __rcu *owner;
-#endif
-
- /* store ref to file /proc/<pid>/exe symlink points to */
- struct file *exe_file;
-#ifdef CONFIG_MMU_NOTIFIER
- struct mmu_notifier_mm *mmu_notifier_mm;
-#endif
-#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && !USE_SPLIT_PMD_PTLOCKS
- pgtable_t pmd_huge_pte; /* protected by page_table_lock */
-#endif
-#ifdef CONFIG_CPUMASK_OFFSTACK
- struct cpumask cpumask_allocation;
-#endif
-#ifdef CONFIG_NUMA_BALANCING
- /*
- * numa_next_scan is the next time that the PTEs will be marked
- * pte_numa. NUMA hinting faults will gather statistics and migrate
- * pages to new nodes if necessary.
- */
- unsigned long numa_next_scan;
-
- /* Restart point for scanning and setting pte_numa */
- unsigned long numa_scan_offset;
-
- /* numa_scan_seq prevents two threads setting pte_numa */
- int numa_scan_seq;
-#endif
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
- /*
- * An operation with batched TLB flushing is going on. Anything that
- * can move process memory needs to flush the TLB when moving a
- * PROT_NONE or PROT_NUMA mapped page.
- */
- bool tlb_flush_pending;
-#endif
- struct uprobes_state uprobes_state;
-#ifdef CONFIG_X86_INTEL_MPX
- /* address of the bounds directory */
- void __user *bd_addr;
-#endif
-};
-
-static inline void mm_init_cpumask(struct mm_struct *mm)
-{
-#ifdef CONFIG_CPUMASK_OFFSTACK
- mm->cpu_vm_mask_var = &mm->cpumask_allocation;
-#endif
- cpumask_clear(mm->cpu_vm_mask_var);
-}
-
-/* Future-safe accessor for struct mm_struct's cpu_vm_mask. */
-static inline cpumask_t *mm_cpumask(struct mm_struct *mm)
-{
- return mm->cpu_vm_mask_var;
-}
-
-#if defined(CONFIG_NUMA_BALANCING) || defined(CONFIG_COMPACTION)
-/*
- * Memory barriers to keep this state in sync are graciously provided by
- * the page table locks, outside of which no page table modifications happen.
- * The barriers below prevent the compiler from re-ordering the instructions
- * around the memory barriers that are already present in the code.
- */
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
- barrier();
- return mm->tlb_flush_pending;
-}
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
-{
- mm->tlb_flush_pending = true;
-
- /*
- * Guarantee that the tlb_flush_pending store does not leak into the
- * critical section updating the page tables
- */
- smp_mb__before_spinlock();
-}
-/* Clearing is done after a TLB flush, which also provides a barrier. */
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
-{
- barrier();
- mm->tlb_flush_pending = false;
-}
-#else
-static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
-{
- return false;
-}
-static inline void set_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-static inline void clear_tlb_flush_pending(struct mm_struct *mm)
-{
-}
-#endif
-
-struct vm_special_mapping
-{
+struct vm_special_mapping {
const char *name;
struct page **pages;
};
diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
index 95243d28a0ee..779e32567e6d 100644
--- a/include/linux/mmu_notifier.h
+++ b/include/linux/mmu_notifier.h
@@ -4,6 +4,7 @@
#include <linux/list.h>
#include <linux/spinlock.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/srcu.h>
struct mmu_notifier;
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 22ee0d5d7f8c..99001775ffa0 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -25,6 +25,7 @@ struct sched_param {
#include <linux/errno.h>
#include <linux/nodemask.h>
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/preempt_mask.h>
#include <asm/page.h>
diff --git a/mm/init-mm.c b/mm/init-mm.c
index a56a851908d2..310ae8c7d9c6 100644
--- a/mm/init-mm.c
+++ b/mm/init-mm.c
@@ -1,4 +1,5 @@
#include <linux/mm_types.h>
+#include <linux/mm_struct.h>
#include <linux/rbtree.h>
#include <linux/rwsem.h>
#include <linux/spinlock.h>
diff --git a/mm/kmemcheck.c b/mm/kmemcheck.c
index cab58bb592d8..c40d065013d2 100644
--- a/mm/kmemcheck.c
+++ b/mm/kmemcheck.c
@@ -1,5 +1,4 @@
#include <linux/gfp.h>
-#include <linux/mm_types.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include "slab.h"
--
2.2.2
--
Kirill A. Shutemov
^ permalink raw reply related [flat|nested] 28+ messages in thread
* Re: [PATCH 2/4] mm: split up mm_struct to separate header file
@ 2015-01-29 0:30 ` Kirill A. Shutemov
0 siblings, 0 replies; 28+ messages in thread
From: Kirill A. Shutemov @ 2015-01-29 0:30 UTC (permalink / raw)
To: Guenter Roeck; +Cc: Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 03:17:42PM +0200, Kirill A. Shutemov wrote:
> We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
> pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
> <asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
> definition.
>
> This patch move mm_struct definition into separate header file in order
> to fix circular header dependencies.
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Guenter, below is update for the patch. It doesn't fix all the issues, but
you should see an improvement. I'll continue with this tomorrow.
BTW, any idea where I can get hexagon cross compiler?
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 2/4] mm: split up mm_struct to separate header file
2015-01-29 0:30 ` Kirill A. Shutemov
@ 2015-01-29 2:02 ` Guenter Roeck
-1 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-29 2:02 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On 01/28/2015 04:30 PM, Kirill A. Shutemov wrote:
> On Wed, Jan 28, 2015 at 03:17:42PM +0200, Kirill A. Shutemov wrote:
>> We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
>> pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
>> <asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
>> definition.
>>
>> This patch move mm_struct definition into separate header file in order
>> to fix circular header dependencies.
>>
>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>
> Guenter, below is update for the patch. It doesn't fix all the issues, but
> you should see an improvement. I'll continue with this tomorrow.
>
I'll give it a try.
> BTW, any idea where I can get hexagon cross compiler?
>
http://www.mentor.com/embedded-software/sourcery-tools/sourcery-codebench/editions/lite-edition/
It is 32 bit, so you'll have to install some 32 bit libraries.
Guenter
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 2/4] mm: split up mm_struct to separate header file
@ 2015-01-29 2:02 ` Guenter Roeck
0 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-29 2:02 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On 01/28/2015 04:30 PM, Kirill A. Shutemov wrote:
> On Wed, Jan 28, 2015 at 03:17:42PM +0200, Kirill A. Shutemov wrote:
>> We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
>> pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
>> <asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
>> definition.
>>
>> This patch move mm_struct definition into separate header file in order
>> to fix circular header dependencies.
>>
>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>
> Guenter, below is update for the patch. It doesn't fix all the issues, but
> you should see an improvement. I'll continue with this tomorrow.
>
I'll give it a try.
> BTW, any idea where I can get hexagon cross compiler?
>
http://www.mentor.com/embedded-software/sourcery-tools/sourcery-codebench/editions/lite-edition/
It is 32 bit, so you'll have to install some 32 bit libraries.
Guenter
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 2/4] mm: split up mm_struct to separate header file
2015-01-29 0:30 ` Kirill A. Shutemov
@ 2015-01-29 4:22 ` Guenter Roeck
-1 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-29 4:22 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On 01/28/2015 04:30 PM, Kirill A. Shutemov wrote:
> On Wed, Jan 28, 2015 at 03:17:42PM +0200, Kirill A. Shutemov wrote:
>> We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
>> pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
>> <asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
>> definition.
>>
>> This patch move mm_struct definition into separate header file in order
>> to fix circular header dependencies.
>>
>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>
> Guenter, below is update for the patch. It doesn't fix all the issues, but
> you should see an improvement. I'll continue with this tomorrow.
Yes, this is much better than before.
Build results:
total: 134 pass: 122 fail: 12
Failed builds:
arm:imx_v6_v7_defconfig
arm:s3c6400_defconfig
cris:artpec_3_defconfig
cris:etraxfs_defconfig
hexagon:defconfig
m68k:m5475evb_defconfig
mips:allmodconfig
s390:defconfig
sparc64:defconfig
sparc64:allmodconfig
unicore32:defconfig
xtensa:allmodconfig
Qemu tests:
total: 30 pass: 28 fail: 2
Failed tests:
sparc64:sparc_smp_defconfig
sparc64:sparc_nosmp_defconfig
Some of the problems are inherited from mmotm.
Here are the mmotm build results (v3.19-rc6-462-g995c249):
total: 134 pass: 128 fail: 6
Failed builds:
arm:imx_v6_v7_defconfig
arm:s3c6400_defconfig
mips:allmodconfig
sparc64:allmodconfig
unicore32:defconfig
xtensa:allmodconfig
Qemu tests:
total: 30 pass: 30 fail: 0
With this, we can conclude that your patch series still has problems
(at least) with cris, hexagon, m68k, s390, sparc, and unicore32.
Guenter
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 2/4] mm: split up mm_struct to separate header file
@ 2015-01-29 4:22 ` Guenter Roeck
0 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-29 4:22 UTC (permalink / raw)
To: Kirill A. Shutemov; +Cc: Andrew Morton, linux-mm, linux-kernel
On 01/28/2015 04:30 PM, Kirill A. Shutemov wrote:
> On Wed, Jan 28, 2015 at 03:17:42PM +0200, Kirill A. Shutemov wrote:
>> We want to use __PAGETABLE_PMD_FOLDED in mm_struct to drop nr_pmds if
>> pmd is folded. __PAGETABLE_PMD_FOLDED is defined in <asm/pgtable.h>, but
>> <asm/pgtable.h> itself wants <linux/mm_types.h> for struct page
>> definition.
>>
>> This patch move mm_struct definition into separate header file in order
>> to fix circular header dependencies.
>>
>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>
> Guenter, below is update for the patch. It doesn't fix all the issues, but
> you should see an improvement. I'll continue with this tomorrow.
Yes, this is much better than before.
Build results:
total: 134 pass: 122 fail: 12
Failed builds:
arm:imx_v6_v7_defconfig
arm:s3c6400_defconfig
cris:artpec_3_defconfig
cris:etraxfs_defconfig
hexagon:defconfig
m68k:m5475evb_defconfig
mips:allmodconfig
s390:defconfig
sparc64:defconfig
sparc64:allmodconfig
unicore32:defconfig
xtensa:allmodconfig
Qemu tests:
total: 30 pass: 28 fail: 2
Failed tests:
sparc64:sparc_smp_defconfig
sparc64:sparc_nosmp_defconfig
Some of the problems are inherited from mmotm.
Here are the mmotm build results (v3.19-rc6-462-g995c249):
total: 134 pass: 128 fail: 6
Failed builds:
arm:imx_v6_v7_defconfig
arm:s3c6400_defconfig
mips:allmodconfig
sparc64:allmodconfig
unicore32:defconfig
xtensa:allmodconfig
Qemu tests:
total: 30 pass: 30 fail: 0
With this, we can conclude that your patch series still has problems
(at least) with cris, hexagon, m68k, s390, sparc, and unicore32.
Guenter
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 21:19 ` Guenter Roeck
0 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 21:19 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 11:16:47PM +0200, Kirill A. Shutemov wrote:
>
> Fixlet for AVR32:
>
> diff --git a/arch/avr32/include/asm/pgtable.h b/arch/avr32/include/asm/pgtable.h
> index 35800664076e..3af39532b25b 100644
Also applied.
Guenter
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH 0/4] Introduce <linux/mm_struct.h>
@ 2015-01-28 21:19 ` Guenter Roeck
0 siblings, 0 replies; 28+ messages in thread
From: Guenter Roeck @ 2015-01-28 21:19 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Kirill A. Shutemov, Andrew Morton, linux-mm, linux-kernel
On Wed, Jan 28, 2015 at 11:16:47PM +0200, Kirill A. Shutemov wrote:
>
> Fixlet for AVR32:
>
> diff --git a/arch/avr32/include/asm/pgtable.h b/arch/avr32/include/asm/pgtable.h
> index 35800664076e..3af39532b25b 100644
Also applied.
Guenter
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2015-01-29 4:22 UTC | newest]
Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-01-28 13:17 [PATCH 0/4] Introduce <linux/mm_struct.h> Kirill A. Shutemov
2015-01-28 13:17 ` Kirill A. Shutemov
2015-01-28 13:17 ` [PATCH 1/4] mm: move enum tlb_flush_reason into <trace/events/tlb.h> Kirill A. Shutemov
2015-01-28 13:17 ` Kirill A. Shutemov
2015-01-28 13:17 ` [PATCH 2/4] mm: split up mm_struct to separate header file Kirill A. Shutemov
2015-01-28 13:17 ` Kirill A. Shutemov
2015-01-29 0:30 ` Kirill A. Shutemov
2015-01-29 0:30 ` Kirill A. Shutemov
2015-01-29 2:02 ` Guenter Roeck
2015-01-29 2:02 ` Guenter Roeck
2015-01-29 4:22 ` Guenter Roeck
2015-01-29 4:22 ` Guenter Roeck
2015-01-28 13:17 ` [PATCH 3/4] mm: define __PAGETABLE_{PMD,PUD}_FOLDED to zero or one Kirill A. Shutemov
2015-01-28 13:17 ` Kirill A. Shutemov
2015-01-28 13:17 ` [PATCH 4/4] mm: do not add nr_pmds into mm_struct if PMD is folded Kirill A. Shutemov
2015-01-28 13:17 ` Kirill A. Shutemov
2015-01-28 17:06 ` [PATCH 0/4] Introduce <linux/mm_struct.h> Guenter Roeck
2015-01-28 17:06 ` Guenter Roeck
2015-01-28 18:50 ` Guenter Roeck
2015-01-28 18:50 ` Guenter Roeck
2015-01-28 20:45 ` Kirill A. Shutemov
2015-01-28 20:45 ` Kirill A. Shutemov
2015-01-28 21:18 ` Guenter Roeck
2015-01-28 21:18 ` Guenter Roeck
2015-01-28 21:16 ` Kirill A. Shutemov
2015-01-28 21:16 ` Kirill A. Shutemov
2015-01-28 21:19 Guenter Roeck
2015-01-28 21:19 ` Guenter Roeck
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.