From: Rusty Russell <rusty@rustcorp.com.au>
To: lkml - Kernel Mailing List <linux-kernel@vger.kernel.org>
Cc: Zachary Amsden <zach@vmware.com>,
Jeremy Fitzhardinge <jeremy@xensource.com>,
Ingo Molnar <mingo@elte.hu>,
Andrew Morton <akpm@linux-foundation.org>,
Andi Kleen <ak@suse.de>
Subject: [PATCH 6/8] Allow per-cpu variables to be page-aligned
Date: Wed, 07 Mar 2007 00:00:20 +1100 [thread overview]
Message-ID: <1173186021.4644.38.camel@localhost.localdomain> (raw)
In-Reply-To: <1173185909.4644.36.camel@localhost.localdomain>
Xen wants page-aligned GDT (and PDA must not cross a page-boundary,
but that doesn't happen at the moment since it's so close to start of
page). Let's allow page-alignment in general for per-cpu data.
Because larger alignments can use more room, we increase the max
per-cpu memory to 64k rather than 32k: it's getting a little tight.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
diff -r 213b1ec27001 arch/alpha/kernel/vmlinux.lds.S
--- a/arch/alpha/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/alpha/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -69,7 +69,7 @@ SECTIONS
. = ALIGN(8);
SECURITY_INIT
- . = ALIGN(64);
+ . = ALIGN(8192);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/arm/kernel/vmlinux.lds.S
--- a/arch/arm/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/arm/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -59,7 +59,7 @@ SECTIONS
usr/built-in.o(.init.ramfs)
__initramfs_end = .;
#endif
- . = ALIGN(64);
+ . = ALIGN(4096);
__per_cpu_start = .;
*(.data.percpu)
__per_cpu_end = .;
diff -r 213b1ec27001 arch/cris/arch-v32/vmlinux.lds.S
--- a/arch/cris/arch-v32/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/cris/arch-v32/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -91,6 +91,7 @@ SECTIONS
}
SECURITY_INIT
+ . = ALIGN (8192);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/frv/kernel/vmlinux.lds.S
--- a/arch/frv/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/frv/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -57,6 +57,7 @@ SECTIONS
__alt_instructions_end = .;
.altinstr_replacement : { *(.altinstr_replacement) }
+ . = ALIGN(4096);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/i386/kernel/vmlinux.lds.S
--- a/arch/i386/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/i386/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -196,7 +196,7 @@ SECTIONS
__initramfs_end = .;
}
#endif
- . = ALIGN(L1_CACHE_BYTES);
+ . = ALIGN(4096);
.data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) {
__per_cpu_start = .;
*(.data.percpu)
diff -r 213b1ec27001 arch/m32r/kernel/vmlinux.lds.S
--- a/arch/m32r/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/m32r/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -110,7 +110,7 @@ SECTIONS
__initramfs_end = .;
#endif
- . = ALIGN(32);
+ . = ALIGN(4096);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/mips/kernel/vmlinux.lds.S
--- a/arch/mips/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/mips/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -119,7 +119,7 @@ SECTIONS
.init.ramfs : { *(.init.ramfs) }
__initramfs_end = .;
#endif
- . = ALIGN(32);
+ . = ALIGN(_PAGE_SIZE);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/parisc/kernel/vmlinux.lds.S
--- a/arch/parisc/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/parisc/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -181,7 +181,7 @@ SECTIONS
.init.ramfs : { *(.init.ramfs) }
__initramfs_end = .;
#endif
- . = ALIGN(32);
+ . = ALIGN(ASM_PAGE_SIZE);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/powerpc/kernel/vmlinux.lds.S
--- a/arch/powerpc/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/powerpc/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -139,11 +139,7 @@ SECTIONS
__initramfs_end = .;
}
#endif
-#ifdef CONFIG_PPC32
- . = ALIGN(32);
-#else
- . = ALIGN(128);
-#endif
+ . = ALIGN(PAGE_SIZE);
.data.percpu : {
__per_cpu_start = .;
*(.data.percpu)
diff -r 213b1ec27001 arch/ppc/kernel/vmlinux.lds.S
--- a/arch/ppc/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/ppc/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -130,7 +130,7 @@ SECTIONS
__ftr_fixup : { *(__ftr_fixup) }
__stop___ftr_fixup = .;
- . = ALIGN(32);
+ . = ALIGN(4096);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/s390/kernel/vmlinux.lds.S
--- a/arch/s390/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/s390/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -99,7 +99,7 @@ SECTIONS
. = ALIGN(2);
__initramfs_end = .;
#endif
- . = ALIGN(256);
+ . = ALIGN(4096);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/sh/kernel/vmlinux.lds.S
--- a/arch/sh/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/sh/kernel/vmlinux.lds.S Tue Mar 06 19:02:25 2007 +1100
@@ -54,7 +54,7 @@ SECTIONS
. = ALIGN(PAGE_SIZE);
.data.page_aligned : { *(.data.page_aligned) }
- . = ALIGN(L1_CACHE_BYTES);
+ . = ALIGN(PAGE_SIZE);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/sh64/kernel/vmlinux.lds.S
--- a/arch/sh64/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/sh64/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -85,7 +85,7 @@ SECTIONS
. = ALIGN(PAGE_SIZE);
.data.page_aligned : C_PHYS(.data.page_aligned) { *(.data.page_aligned) }
- . = ALIGN(L1_CACHE_BYTES);
+ . = ALIGN(PAGE_SIZE);
__per_cpu_start = .;
.data.percpu : C_PHYS(.data.percpu) { *(.data.percpu) }
__per_cpu_end = . ;
diff -r 213b1ec27001 arch/sparc/kernel/vmlinux.lds.S
--- a/arch/sparc/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/sparc/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -65,7 +65,7 @@ SECTIONS
__initramfs_end = .;
#endif
- . = ALIGN(32);
+ . = ALIGN(4096);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/x86_64/kernel/vmlinux.lds.S
--- a/arch/x86_64/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/x86_64/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -194,7 +194,7 @@ SECTIONS
__initramfs_end = .;
#endif
- . = ALIGN(CONFIG_X86_L1_CACHE_BYTES);
+ . = ALIGN(4096);
__per_cpu_start = .;
.data.percpu : AT(ADDR(.data.percpu) - LOAD_OFFSET) { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 arch/xtensa/kernel/vmlinux.lds.S
--- a/arch/xtensa/kernel/vmlinux.lds.S Tue Mar 06 19:01:59 2007 +1100
+++ b/arch/xtensa/kernel/vmlinux.lds.S Tue Mar 06 19:02:03 2007 +1100
@@ -198,7 +198,7 @@ SECTIONS
__ftr_fixup : { *(__ftr_fixup) }
__stop___ftr_fixup = .;
- . = ALIGN(32);
+ . = ALIGN(4096);
__per_cpu_start = .;
.data.percpu : { *(.data.percpu) }
__per_cpu_end = .;
diff -r 213b1ec27001 include/linux/percpu.h
--- a/include/linux/percpu.h Tue Mar 06 19:01:59 2007 +1100
+++ b/include/linux/percpu.h Tue Mar 06 23:32:35 2007 +1100
@@ -11,7 +11,7 @@
/* Enough to cover all DEFINE_PER_CPUs in kernel, including modules. */
#ifndef PERCPU_ENOUGH_ROOM
-#define PERCPU_ENOUGH_ROOM 32768
+#define PERCPU_ENOUGH_ROOM 65536
#endif
/*
diff -r 213b1ec27001 init/main.c
--- a/init/main.c Tue Mar 06 19:01:59 2007 +1100
+++ b/init/main.c Tue Mar 06 19:02:03 2007 +1100
@@ -368,12 +368,12 @@ static void __init setup_per_cpu_areas(v
unsigned long nr_possible_cpus = num_possible_cpus();
/* Copy section for each CPU (we discard the original) */
- size = ALIGN(__per_cpu_end - __per_cpu_start, SMP_CACHE_BYTES);
+ size = ALIGN(__per_cpu_end - __per_cpu_start, PAGE_SIZE);
#ifdef CONFIG_MODULES
if (size < PERCPU_ENOUGH_ROOM)
size = PERCPU_ENOUGH_ROOM;
#endif
- ptr = alloc_bootmem(size * nr_possible_cpus);
+ ptr = alloc_bootmem_pages(size * nr_possible_cpus);
for_each_possible_cpu(i) {
__per_cpu_offset[i] = ptr - __per_cpu_start;
diff -r 213b1ec27001 kernel/module.c
--- a/kernel/module.c Tue Mar 06 19:01:59 2007 +1100
+++ b/kernel/module.c Tue Mar 06 19:02:03 2007 +1100
@@ -544,10 +544,10 @@ static void *percpu_modalloc(unsigned lo
unsigned int i;
void *ptr;
- if (align > SMP_CACHE_BYTES) {
- printk(KERN_WARNING "%s: per-cpu alignment %li > %i\n",
- name, align, SMP_CACHE_BYTES);
- align = SMP_CACHE_BYTES;
+ if (align > PAGE_SIZE) {
+ printk(KERN_WARNING "%s: per-cpu alignment %li > %li\n",
+ name, align, PAGE_SIZE);
+ align = PAGE_SIZE;
}
ptr = __per_cpu_start;
@@ -628,7 +628,7 @@ static int percpu_modinit(void)
pcpu_size = kmalloc(sizeof(pcpu_size[0]) * pcpu_num_allocated,
GFP_KERNEL);
/* Static in-kernel percpu data (used). */
- pcpu_size[0] = -ALIGN(__per_cpu_end-__per_cpu_start, SMP_CACHE_BYTES);
+ pcpu_size[0] = -ALIGN(__per_cpu_end-__per_cpu_start, PAGE_SIZE);
/* Free room. */
pcpu_size[1] = PERCPU_ENOUGH_ROOM + pcpu_size[0];
if (pcpu_size[1] < 0) {
next prev parent reply other threads:[~2007-03-06 13:01 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-03-06 12:39 [PATCH 0/8] x86 boot, pda and gdt cleanups Rusty Russell
2007-03-06 12:53 ` [PATCH 1/8] Remove cpu_gdt_table: use boot_gdt_table until migration to per-cpu Rusty Russell
2007-03-06 12:54 ` [PATCH 2/8] Remove NR_CPUS from asm-generic/percpu.h Rusty Russell
2007-03-06 12:55 ` [PATCH 3/8] Use per-cpu variables for GDT, PDA Rusty Russell
2007-03-06 12:57 ` [PATCH 4/8] Cleanup setup_pda Rusty Russell
2007-03-06 12:58 ` [PATCH 5/8] Cleanup GDT access Rusty Russell
2007-03-06 13:00 ` Rusty Russell [this message]
2007-03-06 13:01 ` [PATCH 7/8] Page-align the GDT Rusty Russell
2007-03-06 13:03 ` [PATCH 8/8] Convert PDA into the percpu section Rusty Russell
2007-03-06 13:10 ` Ingo Molnar
2007-03-07 0:12 ` Rusty Russell
2007-03-07 0:35 ` Jeremy Fitzhardinge
2007-03-06 18:28 ` Jeremy Fitzhardinge
2007-03-06 19:34 ` Andi Kleen
2007-03-06 18:37 ` Jeremy Fitzhardinge
2007-03-07 0:33 ` Rusty Russell
2007-03-07 11:55 ` Rusty Russell
2007-03-13 17:15 ` Jeremy Fitzhardinge
2007-03-14 2:27 ` Rusty Russell
2007-03-06 13:15 ` [PATCH 6/8] Allow per-cpu variables to be page-aligned Ingo Molnar
2007-03-07 0:16 ` Rusty Russell
2007-03-07 0:44 ` H. Peter Anvin
2007-03-06 18:17 ` Jeremy Fitzhardinge
2007-03-07 0:29 ` Rusty Russell
2007-03-06 18:16 ` [PATCH 5/8] Cleanup GDT access Jeremy Fitzhardinge
2007-03-06 18:14 ` [PATCH 3/8] Use per-cpu variables for GDT, PDA Jeremy Fitzhardinge
2007-03-06 13:21 ` [PATCH 2/8] Remove NR_CPUS from asm-generic/percpu.h Ingo Molnar
2007-03-06 13:20 ` [PATCH 1/8] Remove cpu_gdt_table: use boot_gdt_table until migration to per-cpu Ingo Molnar
2007-03-06 13:26 ` Ingo Molnar
2007-03-07 0:22 ` Rusty Russell
2007-03-13 20:48 ` [PATCH 0/8] x86 boot, pda and gdt cleanups Jeremy Fitzhardinge
2007-03-14 2:25 ` Rusty Russell
2007-03-14 4:39 ` Jeremy Fitzhardinge
2007-03-14 6:54 ` Rusty Russell
2007-03-14 23:55 ` Rusty Russell
2007-03-15 1:57 ` Jeremy Fitzhardinge
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1173186021.4644.38.camel@localhost.localdomain \
--to=rusty@rustcorp.com.au \
--cc=ak@suse.de \
--cc=akpm@linux-foundation.org \
--cc=jeremy@xensource.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=zach@vmware.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).