From: Peter Zijlstra <a.p.zijlstra@chello.nl> To: Andrea Arcangeli <aarcange@redhat.com>, Avi Kivity <avi@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, Rik van Riel <riel@redhat.com>, Ingo Molnar <mingo@elte.hu>, akpm@linux-foundation.org, Linus Torvalds <torvalds@linux-foundation.org> Cc: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, Benjamin Herrenschmidt <benh@kernel.crashing.org>, David Miller <davem@davemloft.net>, Hugh Dickins <hugh.dickins@tiscali.co.uk>, Mel Gorman <mel@csn.ul.ie>, Nick Piggin <npiggin@suse.de>, Peter Zijlstra <a.p.zijlstra@chello.nl>, Paul McKenney <paulmck@linux.vnet.ibm.com>, Yanmin Zhang <yanmin_zhang@linux.intel.com>, Stephen Rothwell <sfr@canb.auug.org.au>, Martin Schwidefsky <schwidefsky@de.ibm.com> Subject: [PATCH 10/20] s390: preemptible mmu_gather Date: Sat, 28 Aug 2010 16:16:47 +0200 [thread overview] Message-ID: <20100828142456.102992099@chello.nl> (raw) In-Reply-To: 20100828141637.421594670@chello.nl [-- Attachment #1: martin-mm-preempt-tlb-gather-s390.patch --] [-- Type: text/plain, Size: 2682 bytes --] From: Martin Schwidefsky <schwidefsky@de.ibm.com> Adapt the stand-alone s390 mmu_gather implementation to the new preemptible mmu_gather interface. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> LKML-Reference: <20100409105800.0d0f93ac@mschwide.boeblingen.de.ibm.com> --- arch/s390/include/asm/tlb.h | 43 +++++++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 18 deletions(-) Index: linux-2.6/arch/s390/include/asm/tlb.h =================================================================== --- linux-2.6.orig/arch/s390/include/asm/tlb.h +++ linux-2.6/arch/s390/include/asm/tlb.h @@ -28,44 +28,50 @@ #include <asm/smp.h> #include <asm/tlbflush.h> -#ifndef CONFIG_SMP -#define TLB_NR_PTRS 1 -#else -#define TLB_NR_PTRS 508 -#endif - struct mmu_gather { struct mm_struct *mm; unsigned int fullmm; unsigned int nr_ptes; unsigned int nr_pxds; - void *array[TLB_NR_PTRS]; + unsigned int max; + void **array; + void *local[8]; }; -DECLARE_PER_CPU(struct mmu_gather, mmu_gathers); - -static inline struct mmu_gather *tlb_gather_mmu(struct mm_struct *mm, - unsigned int full_mm_flush) +static inline void __tlb_alloc_pages(struct mmu_gather *tlb) { - struct mmu_gather *tlb = &get_cpu_var(mmu_gathers); + unsigned long addr = __get_free_pages(GFP_ATOMIC, 0); + + if (addr) { + tlb->array = (void *) addr; + tlb->max = PAGE_SIZE / sizeof(void *); + } +} +static inline void tlb_gather_mmu(struct mmu_gather *tlb, + struct mm_struct *mm, + unsigned int full_mm_flush) +{ tlb->mm = mm; + tlb->max = ARRAY_SIZE(tlb->local); + tlb->array = tlb->local; tlb->fullmm = full_mm_flush; - tlb->nr_ptes = 0; - tlb->nr_pxds = TLB_NR_PTRS; if (tlb->fullmm) __tlb_flush_mm(mm); - return tlb; + else + __tlb_alloc_pages(tlb); + tlb->nr_ptes = 0; + tlb->nr_pxds = tlb->max; } static inline void tlb_flush_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end) { - if (!tlb->fullmm && (tlb->nr_ptes > 0 || tlb->nr_pxds < TLB_NR_PTRS)) + if (!tlb->fullmm && (tlb->nr_ptes > 0 || tlb->nr_pxds < tlb->max)) __tlb_flush_mm(tlb->mm); while (tlb->nr_ptes > 0) pte_free(tlb->mm, tlb->array[--tlb->nr_ptes]); - while (tlb->nr_pxds < TLB_NR_PTRS) + while (tlb->nr_pxds < tlb->max) /* pgd_free frees the pointer as region or segment table */ pgd_free(tlb->mm, tlb->array[tlb->nr_pxds++]); } @@ -78,7 +84,8 @@ static inline void tlb_finish_mmu(struct /* keep the page table cache within bounds */ check_pgt_cache(); - put_cpu_var(mmu_gathers); + if (tlb->array != tlb->local) + free_pages((unsigned long) tlb->array, 0); } /*
WARNING: multiple messages have this Message-ID (diff)
From: Peter Zijlstra <a.p.zijlstra@chello.nl> To: Andrea Arcangeli <aarcange@redhat.com>, Avi Kivity <avi@redhat.com>, Thomas Gleixner <tglx@linutronix.de>, Rik van Riel <riel@redhat.com>, Ingo Molnar <mingo@elte.hu>, akpm@linux-fou Cc: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, Benjamin Herrenschmidt <benh@kernel.crashing.org>, David Miller <davem@davemloft.net>, Hugh Dickins <hugh.dickins@tiscali.co.uk>, Mel Gorman <mel@csn.ul.ie>, Nick Piggin <npiggin@suse.de>, Peter Zijlstra <a.p.zijlstra@chello.nl>, Paul McKenney <paulmck@linux.vnet.ibm.com>, Yanmin Zhang <yanmin_zhang@linux.intel.com>, Stephen Rothwell <sfr@canb.auug.org.au>, Martin Schwidefsky <schwidefsky@de.ibm.com> Subject: [PATCH 10/20] s390: preemptible mmu_gather Date: Sat, 28 Aug 2010 16:16:47 +0200 [thread overview] Message-ID: <20100828142456.102992099@chello.nl> (raw) In-Reply-To: 20100828141637.421594670@chello.nl [-- Attachment #1: martin-mm-preempt-tlb-gather-s390.patch --] [-- Type: text/plain, Size: 2680 bytes --] From: Martin Schwidefsky <schwidefsky@de.ibm.com> Adapt the stand-alone s390 mmu_gather implementation to the new preemptible mmu_gather interface. Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl> LKML-Reference: <20100409105800.0d0f93ac@mschwide.boeblingen.de.ibm.com> --- arch/s390/include/asm/tlb.h | 43 +++++++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 18 deletions(-) Index: linux-2.6/arch/s390/include/asm/tlb.h =================================================================== --- linux-2.6.orig/arch/s390/include/asm/tlb.h +++ linux-2.6/arch/s390/include/asm/tlb.h @@ -28,44 +28,50 @@ #include <asm/smp.h> #include <asm/tlbflush.h> -#ifndef CONFIG_SMP -#define TLB_NR_PTRS 1 -#else -#define TLB_NR_PTRS 508 -#endif - struct mmu_gather { struct mm_struct *mm; unsigned int fullmm; unsigned int nr_ptes; unsigned int nr_pxds; - void *array[TLB_NR_PTRS]; + unsigned int max; + void **array; + void *local[8]; }; -DECLARE_PER_CPU(struct mmu_gather, mmu_gathers); - -static inline struct mmu_gather *tlb_gather_mmu(struct mm_struct *mm, - unsigned int full_mm_flush) +static inline void __tlb_alloc_pages(struct mmu_gather *tlb) { - struct mmu_gather *tlb = &get_cpu_var(mmu_gathers); + unsigned long addr = __get_free_pages(GFP_ATOMIC, 0); + + if (addr) { + tlb->array = (void *) addr; + tlb->max = PAGE_SIZE / sizeof(void *); + } +} +static inline void tlb_gather_mmu(struct mmu_gather *tlb, + struct mm_struct *mm, + unsigned int full_mm_flush) +{ tlb->mm = mm; + tlb->max = ARRAY_SIZE(tlb->local); + tlb->array = tlb->local; tlb->fullmm = full_mm_flush; - tlb->nr_ptes = 0; - tlb->nr_pxds = TLB_NR_PTRS; if (tlb->fullmm) __tlb_flush_mm(mm); - return tlb; + else + __tlb_alloc_pages(tlb); + tlb->nr_ptes = 0; + tlb->nr_pxds = tlb->max; } static inline void tlb_flush_mmu(struct mmu_gather *tlb, unsigned long start, unsigned long end) { - if (!tlb->fullmm && (tlb->nr_ptes > 0 || tlb->nr_pxds < TLB_NR_PTRS)) + if (!tlb->fullmm && (tlb->nr_ptes > 0 || tlb->nr_pxds < tlb->max)) __tlb_flush_mm(tlb->mm); while (tlb->nr_ptes > 0) pte_free(tlb->mm, tlb->array[--tlb->nr_ptes]); - while (tlb->nr_pxds < TLB_NR_PTRS) + while (tlb->nr_pxds < tlb->max) /* pgd_free frees the pointer as region or segment table */ pgd_free(tlb->mm, tlb->array[tlb->nr_pxds++]); } @@ -78,7 +84,8 @@ static inline void tlb_finish_mmu(struct /* keep the page table cache within bounds */ check_pgt_cache(); - put_cpu_var(mmu_gathers); + if (tlb->array != tlb->local) + free_pages((unsigned long) tlb->array, 0); } /*
next prev parent reply other threads:[~2010-08-28 14:28 UTC|newest] Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top 2010-08-28 14:16 [PATCH 00/20] mm: Preemptibility -v4 Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 01/20] powerpc: Use call_rcu_sched() for pagetables Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-31 6:10 ` Benjamin Herrenschmidt 2010-08-28 14:16 ` [PATCH 02/20] mm: Improve page_lock_anon_vma() comment Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 03/20] mm: Rename drop_anon_vma to put_anon_vma Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 15:08 ` Pekka Enberg 2010-08-28 14:16 ` [PATCH 04/20] mm: Move anon_vma ref out from under CONFIG_KSM Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 05/20] mm: Simplify anon_vma refcounts Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 15:13 ` Pekka Enberg 2010-08-28 14:16 ` [PATCH 06/20] mm: Use refcounts for page_lock_anon_vma() Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 07/20] mm: Preemptible mmu_gather Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 08/20] powerpc: " Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-31 6:26 ` Benjamin Herrenschmidt 2010-08-31 6:31 ` Benjamin Herrenschmidt 2010-08-31 9:14 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 09/20] sparc: " Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra [this message] 2010-08-28 14:16 ` [PATCH 10/20] s390: preemptible mmu_gather Peter Zijlstra 2010-08-28 14:16 ` [PATCH 11/20] arm: Preemptible mmu_gather Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 12/20] sh: " Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 13/20] um: " Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 14/20] ia64: " Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-30 15:44 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 15/20] mm, powerpc: Move the RCU page-table freeing into generic code Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 16/20] lockdep, mutex: Provide mutex_lock_nest_lock Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 17/20] mutex: Provide mutex_is_contended Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 18/20] mm: Convert i_mmap_lock and anon_vma->lock to mutexes Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 19/20] mm: Extended batches for generic mmu_gather Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:16 ` [PATCH 20/20] mm: Optimize page_lock_anon_vma() fast-path Peter Zijlstra 2010-08-28 14:16 ` Peter Zijlstra 2010-08-28 14:32 ` [PATCH 00/20] mm: Preemptibility -v4 Peter Zijlstra 2010-08-28 22:28 ` David Miller 2010-08-28 22:41 ` Peter Zijlstra 2010-08-28 14:56 ` Piotr Hosowicz 2010-08-28 15:10 ` Peter Zijlstra 2010-08-28 15:17 ` Piotr Hosowicz 2010-08-28 15:23 ` Peter Zijlstra 2010-08-28 16:01 ` Piotr Hosowicz 2010-08-29 12:46 ` Piotr Hosowicz 2010-08-29 13:37 ` Peter Zijlstra 2010-08-29 13:43 ` Piotr Hosowicz 2010-08-31 14:02 ` Piotr Hosowicz 2010-08-31 14:14 ` Piotr Hosowicz 2010-09-02 14:53 ` Piotr Hosowicz 2010-08-28 15:19 ` Pekka Enberg 2010-08-28 15:27 ` Peter Zijlstra [not found] ` <AANLkTikSm2Mq8hGNac9rpFH-3pvryw2kW57EP45Ny6Vp@mail.gmail.com> 2010-09-14 5:36 ` Alex,Shi 2010-09-14 7:42 ` Peter Zijlstra 2010-10-18 11:24 [PATCH 00/20] mm: Preemptibility -v5 Peter Zijlstra 2010-10-18 11:24 ` [PATCH 10/20] s390: preemptible mmu_gather Peter Zijlstra 2010-10-18 11:24 ` Peter Zijlstra
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20100828142456.102992099@chello.nl \ --to=a.p.zijlstra@chello.nl \ --cc=aarcange@redhat.com \ --cc=akpm@linux-foundation.org \ --cc=avi@redhat.com \ --cc=benh@kernel.crashing.org \ --cc=davem@davemloft.net \ --cc=hugh.dickins@tiscali.co.uk \ --cc=linux-arch@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mel@csn.ul.ie \ --cc=mingo@elte.hu \ --cc=npiggin@suse.de \ --cc=paulmck@linux.vnet.ibm.com \ --cc=riel@redhat.com \ --cc=schwidefsky@de.ibm.com \ --cc=sfr@canb.auug.org.au \ --cc=tglx@linutronix.de \ --cc=torvalds@linux-foundation.org \ --cc=yanmin_zhang@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.