linux-next: build failure after merge of the final tree (slab tree related)

linux-next: build failure after merge of the final tree (slab tree related)

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 916 bytes --]

Hi all,

After merging the final tree, today's linux-next build (powerpc
allnoconfig) failed like this:

mm/slub.c: In function 'alloc_kmem_cache_cpus':
mm/slub.c:2094: error: 'PERCPU_DYNAMIC_EARLY_SIZE' undeclared (first use in this function)
mm/slub.c:2094: error: (Each undeclared identifier is reported only once
mm/slub.c:2094: error: for each function it appears in.)
mm/slub.c:2094: error: bit-field '<anonymous>' width not an integer constant

Caused by commit e18d65f0500b95d8724b17d8ea9f1116cf390bbe ("slub: Remove
static kmem_cache_cpu array for boot"). PERCPU_DYNAMIC_EARLY_SIZE is only
defined for SMP (and only in linux/percpu.h which is not explicitly
included).  This build does not have CONFIG_SMP set.

I have used the version of the slab tree from next-20100823 for today.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Pekka Enberg]

On 24.8.2010 5.07, Stephen Rothwell wrote:
> Hi all,
>
> After merging the final tree, today's linux-next build (powerpc
> allnoconfig) failed like this:
>
> mm/slub.c: In function 'alloc_kmem_cache_cpus':
> mm/slub.c:2094: error: 'PERCPU_DYNAMIC_EARLY_SIZE' undeclared (first use in this function)
> mm/slub.c:2094: error: (Each undeclared identifier is reported only once
> mm/slub.c:2094: error: for each function it appears in.)
> mm/slub.c:2094: error: bit-field '<anonymous>' width not an integer constant
>
> Caused by commit e18d65f0500b95d8724b17d8ea9f1116cf390bbe ("slub: Remove
> static kmem_cache_cpu array for boot"). PERCPU_DYNAMIC_EARLY_SIZE is only
> defined for SMP (and only in linux/percpu.h which is not explicitly
> included).  This build does not have CONFIG_SMP set.
>
> I have used the version of the slab tree from next-20100823 for today.

Thanks for the report. The problem should be fixed by this commit:

http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commitdiff;h=5792949c540c9fd719c3fc0d6edc97374222e792

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

On Tue, 24 Aug 2010, Pekka Enberg wrote:

> Thanks for the report. The problem should be fixed by this commit:

Its not that easy. __alloc_percpu falls back to kzalloc() on UP and this
can result in unique bootstrap problems with UP since the bootstrap array
is no longer there. Does the UP kernel boot?

Why did this ever build on my UP configuration tests? Hmmm... I only
tested x86_64 UP. This was 32 bit UP I guess.

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Pekka Enberg]

On 24.8.2010 20.59, Christoph Lameter wrote:
> On Tue, 24 Aug 2010, Pekka Enberg wrote:
>
>> Thanks for the report. The problem should be fixed by this commit:
>
> Its not that easy. __alloc_percpu falls back to kzalloc() on UP and this
> can result in unique bootstrap problems with UP since the bootstrap array
> is no longer there. Does the UP kernel boot?

No, I get this under kvm:

[    0.000000] Linux version 2.6.36-rc2+ (penberg@tiger) (gcc version 
4.4.1 (Ubuntu 4.4.1-4ubuntu9) ) #103 Tue Aug 24 21:27:28 EEST 2010
[    0.000000] Command line: notsc nolapic nosmp noacpi pci=conf1 
earlyprintk=ttyS0,keep
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
[    0.000000]  BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000f0000 - 00000000000fffff (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 0000000004000000 (usable)
[    0.000000] console [earlyser0] enabled
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] DMI not present or invalid.
[    0.000000] No AGP bridge found
[    0.000000] last_pfn = 0x4000 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106
[    0.000000] CPU MTRRs all blank - virtualized system.
[    0.000000] Scanning 1 areas for low memory corruption
[    0.000000] modified physical RAM map:
[    0.000000]  modified: 0000000000000000 - 0000000000010000 (reserved)
[    0.000000]  modified: 0000000000010000 - 000000000009fc00 (usable)
[    0.000000]  modified: 000000000009fc00 - 00000000000a0000 (reserved)
[    0.000000]  modified: 00000000000f0000 - 00000000000fffff (reserved)
[    0.000000]  modified: 0000000000100000 - 0000000004000000 (usable)
[    0.000000] init_memory_mapping: 0000000000000000-0000000004000000
[    0.000000] ACPI Error: A valid RSDP was not found 
(20100702/tbxfroot-219)
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 0:1945621, boot clock
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   empty
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[2] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009f
[    0.000000]     0: 0x00000100 -> 0x00004000
[    0.000000] No local APIC present
[    0.000000] APIC: disable apic facility
[    0.000000] APIC: switched to apic NOOP
[    0.000000] PM: Registered nosave memory: 000000000009f000 - 
00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 
00000000000f0000
[    0.000000] PM: Registered nosave memory: 00000000000f0000 - 
00000000000ff000
[    0.000000] PM: Registered nosave memory: 00000000000ff000 - 
0000000000100000
[    0.000000] Allocating PCI resources starting at 4000000 (gap: 
4000000:fc000000)
[    0.000000] Booting paravirtualized kernel on KVM
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on. 
Total pages: 16047
[    0.000000] Kernel command line: notsc nolapic nosmp noacpi pci=conf1 
earlyprintk=ttyS0,keep
[    0.000000] notsc: Kernel compiled with CONFIG_X86_TSC, cannot 
disable TSC completely.
[    0.000000] PID hash table entries: 256 (order: -1, 2048 bytes)
[    0.000000] Dentry cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.000000] Inode-cache hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Subtract (23 early reservations)
[    0.000000]   #0 [0001000000 - 0001b751a0]   TEXT DATA BSS
[    0.000000]   #1 [000009f000 - 0000100000]   BIOS reserved
[    0.000000]   #2 [0000010000 - 0000012000]      TRAMPOLINE
[    0.000000]   #3 [0000012000 - 0000016000]     ACPI WAKEUP
[    0.000000]   #4 [0001b751c0 - 0001b761c0]         BOOTMEM
[    0.000000]   #5 [0001f761c0 - 0001f761d8]         BOOTMEM
[    0.000000]   #6 [0002377000 - 0002378000]         BOOTMEM
[    0.000000]   #7 [0002378000 - 0002379000]         BOOTMEM
[    0.000000]   #8 [0002400000 - 0002600000]        MEMMAP 0
[    0.000000]   #9 [0001b761c0 - 0001b762c0]         BOOTMEM
[    0.000000]   #10 [0001b762c0 - 0001b766c0]         BOOTMEM
[    0.000000]   #11 [0001b77000 - 0001b78000]         BOOTMEM
[    0.000000]   #12 [0001b766c0 - 0001b767d8]         BOOTMEM
[    0.000000]   #13 [0001b76800 - 0001b76868]         BOOTMEM
[    0.000000]   #14 [0001b76880 - 0001b768e8]         BOOTMEM
[    0.000000]   #15 [0001b76900 - 0001b76968]         BOOTMEM
[    0.000000]   #16 [0001b76980 - 0001b769e8]         BOOTMEM
[    0.000000]   #17 [0001b76a00 - 0001b76a20]         BOOTMEM
[    0.000000]   #18 [0001b76a40 - 0001b76a7d]         BOOTMEM
[    0.000000]   #19 [0001b76a80 - 0001b76abd]         BOOTMEM
[    0.000000]   #20 [0001b78000 - 0001b78800]         BOOTMEM
[    0.000000]   #21 [0001b78800 - 0001b88800]         BOOTMEM
[    0.000000]   #22 [0001b88800 - 0001b90800]         BOOTMEM
[    0.000000] Memory: 51156k/65536k available (6550k kernel code, 452k 
absent, 13928k reserved, 3761k data, 764k init)
[    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache 
size=232 realsize=256 order=0 offset=0 flags=42000
[    0.000000]
[    0.000000] Pid: 0, comm: swapper Not tainted 2.6.36-rc2+ #103
[    0.000000] Call Trace:
[    0.000000]  [<ffffffff81659a35>] panic+0x91/0x19b
[    0.000000]  [<ffffffff81126e59>] kmem_cache_open+0x1e9/0x200
[    0.000000]  [<ffffffff81a3141a>] kmem_cache_init+0x57/0x300
[    0.000000]  [<ffffffff81a14a97>] start_kernel+0x1ca/0x396
[    0.000000]  [<ffffffff81a14325>] x86_64_start_reservations+0x12c/0x130
[    0.000000]  [<ffffffff81a14423>] x86_64_start_kernel+0xfa/0x109

> Why did this ever build on my UP configuration tests? Hmmm... I only
> tested x86_64 UP. This was 32 bit UP I guess.

I could reproduce the build problem with x86-64 and !CONFIG_SMP so it's 
not related to 32-bit.

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

On Tue, 24 Aug 2010, Pekka Enberg wrote:

> On 24.8.2010 20.59, Christoph Lameter wrote:
> > On Tue, 24 Aug 2010, Pekka Enberg wrote:
> >
> > > Thanks for the report. The problem should be fixed by this commit:
> >
> > Its not that easy. __alloc_percpu falls back to kzalloc() on UP and this
> > can result in unique bootstrap problems with UP since the bootstrap array
> > is no longer there. Does the UP kernel boot?
>
> No, I get this under kvm:

> [    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache
> size=232 realsize=256 order=0 offset=0 flags=42000

alloc per cpu result in kmalloc which fails.

Tejon: Is there some way we could get a reserved per cpu area under UP
instead of fallback to slab allocations during bootup?

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

Hi Pekka,

On Tue, 24 Aug 2010 20:41:36 +0300 Pekka Enberg <penberg@cs.helsinki.fi> wrote:
>
> Thanks for the report. The problem should be fixed by this commit:
> 
> http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commitdiff;h=5792949c540c9fd719c3fc0d6edc97374222e792

A small point:  I did *not* review that patch.  I *did* report the bug
(and seemingly provided the commit message).

Thanks for the build fix.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 490 bytes --]

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Pekka Enberg]

On 25.8.2010 3.13, Stephen Rothwell wrote:
> Hi Pekka,
>
> On Tue, 24 Aug 2010 20:41:36 +0300 Pekka Enberg<penberg@cs.helsinki.fi>  wrote:
>>
>> Thanks for the report. The problem should be fixed by this commit:
>>
>> http://git.kernel.org/?p=linux/kernel/git/penberg/slab-2.6.git;a=commitdiff;h=5792949c540c9fd719c3fc0d6edc97374222e792
>
> A small point:  I did *not* review that patch.  I *did* report the bug
> (and seemingly provided the commit message).

Oh, sorry about that.

> Thanks for the build fix.

I dropped slub cleanups from -next until we've resolved the UP boot time 
crash problem.

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Tejun Heo]

Hello,

On 08/24/2010 08:53 PM, Christoph Lameter wrote:
>> [    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache
>> size=232 realsize=256 order=0 offset=0 flags=42000
> 
> alloc per cpu result in kmalloc which fails.
> 
> Tejon: Is there some way we could get a reserved per cpu area under UP
> instead of fallback to slab allocations during bootup?

Eh... nasty.  Maybe we can create a alloc_percpu_early() function
which doesn't allow freeing of allocate memory and just redirect to
bootmem on UP?

-- 
tejun

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Pekka Enberg]

On 8/25/10 11:18 AM, Tejun Heo wrote:
> Hello,
>
> On 08/24/2010 08:53 PM, Christoph Lameter wrote:
>>> [    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache
>>> size=232 realsize=256 order=0 offset=0 flags=42000
>>
>> alloc per cpu result in kmalloc which fails.
>>
>> Tejon: Is there some way we could get a reserved per cpu area under UP
>> instead of fallback to slab allocations during bootup?
>
> Eh... nasty.  Maybe we can create a alloc_percpu_early() function
> which doesn't allow freeing of allocate memory and just redirect to
> bootmem on UP?

Yeah, I was thinking about that too.

				Pekka

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

On Wed, 25 Aug 2010, Pekka Enberg wrote:

> On 8/25/10 11:18 AM, Tejun Heo wrote:
> > Hello,
> >
> > On 08/24/2010 08:53 PM, Christoph Lameter wrote:
> > > > [    0.000000] Kernel panic - not syncing: Cannot create slab kmem_cache
> > > > size=232 realsize=256 order=0 offset=0 flags=42000
> > >
> > > alloc per cpu result in kmalloc which fails.
> > >
> > > Tejon: Is there some way we could get a reserved per cpu area under UP
> > > instead of fallback to slab allocations during bootup?
> >
> > Eh... nasty.  Maybe we can create a alloc_percpu_early() function
> > which doesn't allow freeing of allocate memory and just redirect to
> > bootmem on UP?
>
> Yeah, I was thinking about that too.

Another solution is to allocate an order 1 (compound) page for each early
cache. Then resize the kmalloc array after everything is up. kfree() will
then redirect to the page allocator.

But this is a slab allocator specific solution.

We now have the situation that alloc_percpu can only be used on early boot
on SMP machines. A general solution would be better I think.

If the early alloc_percpu stuff would work consistently then it could also
be used to avoid the boot_pageset in the page allocator f.e.

Can we just get rid of the special UP case and just run the percpu
subsystem even for UP?

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

On Wed, 25 Aug 2010, Pekka Enberg wrote:

> I dropped slub cleanups from -next until we've resolved the UP boot time crash
> problem.
>

It works here with this patch. I'd rather see a general solution so that
we can use allocpercpu for bootstrapping any allocator without such
special bandaids.



Subject: Slub: UP bandaid

Since the percpu allocator does not provide early allocation in UP
mode (only in SMP configurations) use __get_free_page() to improvise
a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

Signed-off-by: Christoph Lameter <cl@linux.com>

---
 mm/slub.c |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

Index: linux-2.6/mm/slub.c
===================================================================
--- linux-2.6.orig/mm/slub.c	2010-08-24 20:07:12.766010774 -0500
+++ linux-2.6/mm/slub.c	2010-08-24 20:15:46.304130417 -0500
@@ -2064,8 +2064,24 @@

 static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
 {
+#ifdef CONFIG_SMP
+	/*
+	 * Will use reserve that does not require slab operation during
+	 * early boot.
+	 */
 	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
 			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
+#else
+	/*
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);
+	else
+#endif

 	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

On Wed, 25 Aug 2010, Tejun Heo wrote:

> Eh... nasty.  Maybe we can create a alloc_percpu_early() function
> which doesn't allow freeing of allocate memory and just redirect to
> bootmem on UP?

The whole early alloc stuff does also does not go over too well with the
page allocator (never had trouble on KVM):

Decompressing Linux... Parsing ELF... done.
Booting the kernel.
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.36-rc2 (root@rd-rsync) (gcc version 4.4.4 (Debian
4.4.4-5) ) #1 SMP Wed Aug 25 13:26:59 CDT 2010
Command line: ro root=/dev/mapper/vgubuntu-root console=tty0
console=ttyS1,57600 idle=mwait earlyprintk=ttyS1,57600
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
 BIOS-e820: 0000000000100000 - 00000000cf699000 (usable)
 BIOS-e820: 00000000cf699000 - 00000000cf6af000 (reserved)
 BIOS-e820: 00000000cf6af000 - 00000000cf6ce000 (ACPI data)
 BIOS-e820: 00000000cf6ce000 - 00000000d0000000 (reserved)
 BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
 BIOS-e820: 00000000fe000000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 00000001b0000000 (usable)
bootconsole [earlyser0] enabled
NX (Execute Disable) protection: active
DMI 2.6 present.
No AGP bridge found
last_pfn = 0x1b0000 max_arch_pfn = 0x400000000
last_pfn = 0xcf699 max_arch_pfn = 0x400000000
found SMP MP-table at [ffff8800000fe710] fe710
init_memory_mapping: 0000000000000000-00000000cf699000
init_memory_mapping: 0000000100000000-00000001b0000000
RAMDISK: 37b98000 - 37ff0000
ACPI: RSDP 00000000000f1630 00024 (v02 DELL  )
ACPI: XSDT 00000000000f1734 0009C (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: FACP 00000000cf6c3f9c 000F4 (v03 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: DSDT 00000000cf6af000 0320F (v01 DELL   PE_SC3   00000001 INTL
20050624)
ACPI: FACS 00000000cf6c6000 00040
ACPI: APIC 00000000cf6c3478 0015E (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: SPCR 00000000cf6c35d8 00050 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: HPET 00000000cf6c362c 00038 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: DM__ 00000000cf6c3668 001A8 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: MCFG 00000000cf6c38c4 0003C (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: WD__ 00000000cf6c3904 00134 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: SLIC 00000000cf6c3a3c 00176 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: ERST 00000000cf6b2390 00270 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: HEST 00000000cf6b2600 0027C (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: BERT 00000000cf6b2210 00030 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: EINJ 00000000cf6b2240 00150 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: SRAT 00000000cf6c3bc0 00370 (v01 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: TCPA 00000000cf6c3f34 00064 (v02 DELL   PE_SC3   00000001 DELL
00000001)
ACPI: SSDT 00000000cf6c7000 04BE4 (v01  INTEL PPM RCM  80000001 INTL
20061109)
SRAT: PXM 1 -> APIC 0x10 -> Node 0
SRAT: PXM 2 -> APIC 0x00 -> Node 1
SRAT: PXM 1 -> APIC 0x12 -> Node 0
SRAT: PXM 2 -> APIC 0x02 -> Node 1
SRAT: PXM 1 -> APIC 0x14 -> Node 0
SRAT: PXM 2 -> APIC 0x04 -> Node 1
SRAT: PXM 1 -> APIC 0x16 -> Node 0
SRAT: PXM 2 -> APIC 0x06 -> Node 1
SRAT: PXM 1 -> APIC 0x11 -> Node 0
SRAT: PXM 2 -> APIC 0x01 -> Node 1
SRAT: PXM 1 -> APIC 0x13 -> Node 0
SRAT: PXM 2 -> APIC 0x03 -> Node 1
SRAT: PXM 1 -> APIC 0x15 -> Node 0
SRAT: PXM 2 -> APIC 0x05 -> Node 1
SRAT: PXM 1 -> APIC 0x17 -> Node 0
SRAT: PXM 2 -> APIC 0x07 -> Node 1
SRAT: Node 1 PXM 2 0-c0000000
SRAT: Node 0 PXM 1 c0000000-d0000000
SRAT: Node 0 PXM 1 100000000-1b0000000
SRAT: Node 0 [c0000000,d0000000) + [100000000,1b0000000) ->
[c0000000,1b0000000)
Initmem setup node 0 00000000c0000000-00000001b0000000
  NODE_DATA [0000000100000000 - 0000000100004fff]
Initmem setup node 1 0000000000000000-00000000c0000000
  NODE_DATA [00000000016ee780 - 00000000016f377f]
Zone PFN ranges:
  DMA      0x00000001 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x001b0000
Movable zone start PFN for each node
early_node_map[4] active PFN ranges
    1: 0x00000001 -> 0x000000a0
    1: 0x00000100 -> 0x000c0000
    0: 0x000c0000 -> 0x000cf699
    0: 0x00100000 -> 0x001b0000
------------[ cut here ]------------
WARNING: at mm/percpu.c:285 pcpu_mem_alloc+0x31/0x78()
Hardware name: PowerEdge R610
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.36-rc2 #1
Call Trace:
 [<ffffffff810305fa>] warn_slowpath_common+0x80/0x98
 [<ffffffff81030627>] warn_slowpath_null+0x15/0x17
 [<ffffffff810a226c>] pcpu_mem_alloc+0x31/0x78
 [<ffffffff810a2fe1>] pcpu_alloc+0x243/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
 [<ffffffff815d29df>] start_kernel+0x88/0x351
 [<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
 [<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
---[ end trace 4eaa2a86a8e2da22 ]---
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G        W   2.6.36-rc2 #1
Call Trace:
 [<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
 [<ffffffff815d29df>] start_kernel+0x88/0x351
 [<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
 [<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
---[ end trace 4eaa2a86a8e2da22 ]---
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G        W   2.6.36-rc2 #1
Call Trace:
 [<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a
 [<ffffffff815d29df>] start_kernel+0x88/0x351
 [<ffffffff815d22a3>] x86_64_start_reservations+0xb3/0xb7
 [<ffffffff815d238b>] x86_64_start_kernel+0xe4/0xeb
PERCPU: allocation failed, size=96 align=8, failed to allocate new chunk
Pid: 0, comm: swapper Tainted: G        W   2.6.36-rc2 #1
Call Trace:
 [<ffffffff810a35ca>] pcpu_alloc+0x82c/0x8d9
 [<ffffffff81030c85>] ? release_console_sem+0x185/0x1b6
 [<ffffffff810a3692>] __alloc_percpu+0xb/0xd
 [<ffffffff81601d6f>] free_area_init_node+0x235/0x350
 [<ffffffff815e9c65>] free_area_init_nodes+0x4c1/0x50b
 [<ffffffff815eb13c>] ? sparse_init+0x292/0x2cf
 [<ffffffff815e2d50>] paging_init+0x53/0x5a
 [<ffffffff815d642f>] setup_arch+0x82c/0x8b8
 [<ffffffff813ad816>] ? printk+0x3c/0x3e
 [<ffffffff8104f548>] ? clockevents_register_notifier+0x3e/0x4a

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

Err... Sorry there was patch under testing in there (remove the boot
pageset from the page allocator and use early allocpercpu) which resulted
in the reserve area becoming a bit tight. Sorry.

Re: linux-next: build failure after merge of the final tree (slab tree related)

[David Rientjes]

On Wed, 25 Aug 2010, Christoph Lameter wrote:

> Since the percpu allocator does not provide early allocation in UP
> mode (only in SMP configurations) use __get_free_page() to improvise
> a compound page allocation that can be later freed via kfree().
> 
> Compound pages will be released when the cpu caches are resized.
> 
> Signed-off-by: Christoph Lameter <cl@linux.com>
> 
> ---
>  mm/slub.c |    6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> Index: linux-2.6/mm/slub.c
> ===================================================================
> --- linux-2.6.orig/mm/slub.c	2010-08-24 20:07:12.766010774 -0500
> +++ linux-2.6/mm/slub.c	2010-08-24 20:15:46.304130417 -0500
> @@ -2064,8 +2064,24 @@
> 
>  static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
>  {
> +#ifdef CONFIG_SMP
> +	/*
> +	 * Will use reserve that does not require slab operation during
> +	 * early boot.
> +	 */
>  	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
>  			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
> +#else
> +	/*
> +	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
> +	 * operations. So we cannot use that before the slab allocator is up
> +	 * Simply get the smallest possible compound page. The page will be
> +	 * released via kfree() when the cpu caches are resized later.
> +	 */
> +	if (slab_state < UP)
> +		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);

__get_free_pages() takes an order argument.

> +	else
> +#endif
> 
>  	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);
> 

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

On Wed, 25 Aug 2010, David Rientjes wrote:

> > +	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
> > +	 * operations. So we cannot use that before the slab allocator is up
> > +	 * Simply get the smallest possible compound page. The page will be
> > +	 * released via kfree() when the cpu caches are resized later.
> > +	 */
> > +	if (slab_state < UP)
> > +		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);
>
> __get_free_pages() takes an order argument.

Right. Patch not refreshed after the last tinzy winzy change. Seems that I
got a bit rusty.

Subject: Slub: UP bandaid

Since the percpu allocator does not provide early allocation in UP
mode (only in SMP configurations) use __get_free_page() to improvise
a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

Signed-off-by: Christoph Lameter <cl@linux.com>



---
 mm/slub.c |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)

Index: linux-2.6/mm/slub.c
===================================================================
--- linux-2.6.orig/mm/slub.c	2010-08-25 09:00:59.000000000 -0500
+++ linux-2.6/mm/slub.c	2010-08-25 10:59:30.000000000 -0500
@@ -2091,8 +2091,24 @@ init_kmem_cache_node(struct kmem_cache_n

 static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
 {
+#ifdef CONFIG_SMP
+	/*
+	 * Will use reserve that does not require slab operation during
+	 * early boot.
+	 */
 	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
 			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
+#else
+	/*
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)__get_free_pages(GFP_NOWAIT, 1);
+	else
+#endif

 	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);

Re: linux-next: build failure after merge of the final tree (slab tree related)

[David Rientjes]

On Wed, 25 Aug 2010, Christoph Lameter wrote:

> > > +	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
> > > +	 * operations. So we cannot use that before the slab allocator is up
> > > +	 * Simply get the smallest possible compound page. The page will be
> > > +	 * released via kfree() when the cpu caches are resized later.
> > > +	 */
> > > +	if (slab_state < UP)
> > > +		s->cpu_slab = (__percpu void *)__get_free_page(GFP_NOWAIT, 1);
> >
> > __get_free_pages() takes an order argument.
> 
> Right. Patch not refreshed after the last tinzy winzy change. Seems that I
> got a bit rusty.
> 
> Subject: Slub: UP bandaid
> 
> Since the percpu allocator does not provide early allocation in UP
> mode (only in SMP configurations) use __get_free_page() to improvise
> a compound page allocation that can be later freed via kfree().
> 
> Compound pages will be released when the cpu caches are resized.
> 
> Signed-off-by: Christoph Lameter <cl@linux.com>

Acked-by: David Rientjes <rientjes@google.com>

I'm really hoping that we can remove this hack soon when the percpu 
allocator can handle these allocations on UP without any specialized slab 
behavior.

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Tejun Heo]

Hello,

On 08/25/2010 03:50 PM, Christoph Lameter wrote:
> Can we just get rid of the special UP case and just run the percpu
> subsystem even for UP?

Yeah, maybe.  Then we also can guarantee that percpu allocator always
honors alignment (which wq code currently requires and papers over
with similarly ugly workaround).  It would add a mostly redundant
allocator code tho.  I'll look into how easily it can be done.

Thanks.

-- 
tejun

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Christoph Lameter]

On Wed, 25 Aug 2010, David Rientjes wrote:

> I'm really hoping that we can remove this hack soon when the percpu
> allocator can handle these allocations on UP without any specialized slab
> behavior.

So do I. Here is a slightly less hacky version through using
kmalloc_large instead:


Subject: Slub: UP bandaid

Since the percpu allocator does not provide early allocation in UP
mode (only in SMP configurations) use __get_free_page() to improvise
a compound page allocation that can be later freed via kfree().

Compound pages will be released when the cpu caches are resized.

Acked-by: David Rientjes <rientjes@google.com>
Signed-off-by: Christoph Lameter <cl@linux.com>
---
 mm/slub.c |   16 ++++++++++++++++
 1 file changed, 16 insertions(+)

Index: linux-2.6/mm/slub.c
===================================================================
--- linux-2.6.orig/mm/slub.c	2010-08-26 09:19:35.000000000 -0500
+++ linux-2.6/mm/slub.c	2010-08-26 09:36:29.000000000 -0500
@@ -2103,8 +2103,24 @@ init_kmem_cache_node(struct kmem_cache_n

 static inline int alloc_kmem_cache_cpus(struct kmem_cache *s)
 {
+#ifdef CONFIG_SMP
+	/*
+	 * Will use reserve that does not require slab operation during
+	 * early boot.
+	 */
 	BUILD_BUG_ON(PERCPU_DYNAMIC_EARLY_SIZE <
 			SLUB_PAGE_SHIFT * sizeof(struct kmem_cache_cpu));
+#else
+	/*
+	 * Special hack for UP mode. allocpercpu() falls back to kmalloc
+	 * operations. So we cannot use that before the slab allocator is up
+	 * Simply get the smallest possible compound page. The page will be
+	 * released via kfree() when the cpu caches are resized later.
+	 */
+	if (slab_state < UP)
+		s->cpu_slab = (__percpu void *)kmalloc_large(PAGE_SIZE << 1, GFP_NOWAIT);
+	else
+#endif

 	s->cpu_slab = alloc_percpu(struct kmem_cache_cpu);

Re: linux-next: build failure after merge of the final tree (slab tree related)

[Pekka Enberg]

On 26.8.2010 17.41, Christoph Lameter wrote:
> On Wed, 25 Aug 2010, David Rientjes wrote:
>
>> I'm really hoping that we can remove this hack soon when the percpu
>> allocator can handle these allocations on UP without any specialized slab
>> behavior.
>
> So do I. Here is a slightly less hacky version through using
> kmalloc_large instead:

Applied and queued to -next.

[PATCH 1/3] vmalloc: pcpu_get/free_vm_areas() aren't needed on UP

[Tejun Heo]

These functions are used only by percpu memory allocator on SMP.
Don't build them on UP.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Nick Piggin <npiggin@kernel.dk>
---
So, something like these three patches.

Nick, can I route this one through percpu tree?

Thanks.

 include/linux/vmalloc.h |    2 ++
 mm/vmalloc.c            |    2 ++
 2 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index de05e96..3d510e8 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -115,10 +115,12 @@ extern rwlock_t vmlist_lock;
 extern struct vm_struct *vmlist;
 extern __init void vm_area_register_early(struct vm_struct *vm, size_t align);

+#ifdef CONFIG_SMP
 struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,
 				     const size_t *sizes, int nr_vms,
 				     size_t align, gfp_t gfp_mask);

 void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms);
+#endif

 #endif /* _LINUX_VMALLOC_H */
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index b7e314b..eb57a63 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -2052,6 +2052,7 @@ void free_vm_area(struct vm_struct *area)
 }
 EXPORT_SYMBOL_GPL(free_vm_area);

+#ifdef CONFIG_SMP
 static struct vmap_area *node_to_va(struct rb_node *n)
 {
 	return n ? rb_entry(n, struct vmap_area, rb_node) : NULL;
@@ -2332,6 +2333,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 		free_vm_area(vms[i]);
 	kfree(vms);
 }
+#endif	/* CONFIG_SMP */

 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)
-- 
1.7.1

[PATCH 2/3] percpu: reduce PCPU_MIN_UNIT_SIZE to 32k

[Tejun Heo]

In preparation of enabling percpu allocator for UP, reduce
PCPU_MIN_UNIT_SIZE to 32k.  On UP, the first chunk doesn't have to
include static percpu variables and chunk size can be smaller which is
important as UP percpu allocator will use contiguous kernel memory to
populate chunks.

PCPU_MIN_UNIT_SIZE also determines the maximum supported allocation
size but 32k should still be enough.

Signed-off-by: Tejun Heo <tj@kernel.org>
---
 include/linux/percpu.h |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/include/linux/percpu.h b/include/linux/percpu.h
index 49466b1..fc8130a 100644
--- a/include/linux/percpu.h
+++ b/include/linux/percpu.h
@@ -42,7 +42,7 @@
 #ifdef CONFIG_SMP

 /* minimum unit size, also is the maximum supported allocation size */
-#define PCPU_MIN_UNIT_SIZE		PFN_ALIGN(64 << 10)
+#define PCPU_MIN_UNIT_SIZE		PFN_ALIGN(32 << 10)

 /*
  * Percpu allocator can serve percpu allocations before slab is
-- 
1.7.1

[PATCH 3/3] percpu: use percpu allocator on UP too

[Tejun Heo]

On UP, percpu allocations were redirected to kmalloc.  This has the
following problems.

* For certain amount of allocations (determined by
  PERCPU_DYNAMIC_EARLY_SLOTS and PERCPU_DYNAMIC_EARLY_SIZE), percpu
  allocator can be used before the usual kernel memory allocator is
  brought online.  On SMP, this is used to initialize the kernel
  memory allocator.

* percpu allocator honors alignment upto PAGE_SIZE but kmalloc()
  doesn't.  For example, workqueue makes use of larger alignments for
  cpu_workqueues.

Currently, users of percpu allocators need to handle UP differently,
which is somewhat fragile and ugly.  Other than small amount of
memory, there isn't much to lose by enabling percpu allocator on UP.
It can simply use kernel memory based chunk allocation which was added
for SMP archs w/o MMUs.

This patch removes mm/percpu_up.c, builds mm/percpu.c on UP too and
makes UP build use percpu-km.  As percpu addresses and kernel
addresses are always identity mapped and static percpu variables don't
need any special treatment, nothing is arch dependent and mm/percpu.c
implements generic setup_per_cpu_areas() for UP.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
---
 include/linux/percpu.h |   29 ++++-------------------
 mm/Kconfig             |    8 ++++++
 mm/Makefile            |    7 +----
 mm/percpu-km.c         |    2 +-
 mm/percpu.c            |   60 ++++++++++++++++++++++++++++++++++++++++++++---
 mm/percpu_up.c         |   30 ------------------------
 6 files changed, 71 insertions(+), 65 deletions(-)
 delete mode 100644 mm/percpu_up.c

diff --git a/include/linux/percpu.h b/include/linux/percpu.h
index fc8130a..aeeeef1 100644
--- a/include/linux/percpu.h
+++ b/include/linux/percpu.h
@@ -39,8 +39,6 @@
 	preempt_enable();				\
 } while (0)

-#ifdef CONFIG_SMP
-
 /* minimum unit size, also is the maximum supported allocation size */
 #define PCPU_MIN_UNIT_SIZE		PFN_ALIGN(32 << 10)

@@ -137,37 +135,20 @@ extern int __init pcpu_page_first_chunk(size_t reserved_size,
  * dynamically allocated. Non-atomic access to the current CPU's
  * version should probably be combined with get_cpu()/put_cpu().
  */
+#ifdef CONFIG_SMP
 #define per_cpu_ptr(ptr, cpu)	SHIFT_PERCPU_PTR((ptr), per_cpu_offset((cpu)))
+#else
+#define per_cpu_ptr(ptr, cpu)	({ (void)(cpu); VERIFY_PERCPU_PTR((ptr)); })
+#endif

 extern void __percpu *__alloc_reserved_percpu(size_t size, size_t align);
 extern bool is_kernel_percpu_address(unsigned long addr);

-#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA
+#if !defined(CONFIG_SMP) || !defined(CONFIG_HAVE_SETUP_PER_CPU_AREA)
 extern void __init setup_per_cpu_areas(void);
 #endif
 extern void __init percpu_init_late(void);

-#else /* CONFIG_SMP */
-
-#define per_cpu_ptr(ptr, cpu) ({ (void)(cpu); VERIFY_PERCPU_PTR((ptr)); })
-
-/* can't distinguish from other static vars, always false */
-static inline bool is_kernel_percpu_address(unsigned long addr)
-{
-	return false;
-}
-
-static inline void __init setup_per_cpu_areas(void) { }
-
-static inline void __init percpu_init_late(void) { }
-
-static inline void *pcpu_lpage_remapped(void *kaddr)
-{
-	return NULL;
-}
-
-#endif /* CONFIG_SMP */
-
 extern void __percpu *__alloc_percpu(size_t size, size_t align);
 extern void free_percpu(void __percpu *__pdata);
 extern phys_addr_t per_cpu_ptr_to_phys(void *addr);
diff --git a/mm/Kconfig b/mm/Kconfig
index f4e516e..01a5744 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -301,3 +301,11 @@ config NOMMU_INITIAL_TRIM_EXCESS
 	  of 1 says that all excess pages should be trimmed.

 	  See Documentation/nommu-mmap.txt for more information.
+
+#
+# UP and nommu archs use km based percpu allocator
+#
+config NEED_PER_CPU_KM
+	depends on !SMP
+	bool
+	default y
diff --git a/mm/Makefile b/mm/Makefile
index 34b2546..f73f75a 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -11,7 +11,7 @@ obj-y			:= bootmem.o filemap.o mempool.o oom_kill.o fadvise.o \
 			   maccess.o page_alloc.o page-writeback.o \
 			   readahead.o swap.o truncate.o vmscan.o shmem.o \
 			   prio_tree.o util.o mmzone.o vmstat.o backing-dev.o \
-			   page_isolation.o mm_init.o mmu_context.o \
+			   page_isolation.o mm_init.o mmu_context.o percpu.o \
 			   $(mmu-y)
 obj-y += init-mm.o

@@ -36,11 +36,6 @@ obj-$(CONFIG_FAILSLAB) += failslab.o
 obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o
 obj-$(CONFIG_FS_XIP) += filemap_xip.o
 obj-$(CONFIG_MIGRATION) += migrate.o
-ifdef CONFIG_SMP
-obj-y += percpu.o
-else
-obj-y += percpu_up.o
-endif
 obj-$(CONFIG_QUICKLIST) += quicklist.o
 obj-$(CONFIG_CGROUP_MEM_RES_CTLR) += memcontrol.o page_cgroup.o
 obj-$(CONFIG_MEMORY_FAILURE) += memory-failure.o
diff --git a/mm/percpu-km.c b/mm/percpu-km.c
index df68085..7037bc7 100644
--- a/mm/percpu-km.c
+++ b/mm/percpu-km.c
@@ -27,7 +27,7 @@
  *   chunk size is not aligned.  percpu-km code will whine about it.
  */

-#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
+#if defined(CONFIG_SMP) && defined(CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK)
 #error "contiguous percpu allocation is incompatible with paged first chunk"
 #endif

diff --git a/mm/percpu.c b/mm/percpu.c
index 58c572b..fa70122 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -76,6 +76,7 @@
 #define PCPU_SLOT_BASE_SHIFT		5	/* 1-31 shares the same slot */
 #define PCPU_DFL_MAP_ALLOC		16	/* start a map with 16 ents */

+#ifdef CONFIG_SMP
 /* default addr <-> pcpu_ptr mapping, override in asm/percpu.h if necessary */
 #ifndef __addr_to_pcpu_ptr
 #define __addr_to_pcpu_ptr(addr)					\
@@ -89,6 +90,11 @@
 			 (unsigned long)pcpu_base_addr -		\
 			 (unsigned long)__per_cpu_start)
 #endif
+#else	/* CONFIG_SMP */
+/* on UP, it's always identity mapped */
+#define __addr_to_pcpu_ptr(addr)	(void __percpu *)(addr)
+#define __pcpu_ptr_to_addr(ptr)		(void __force *)(ptr)
+#endif	/* CONFIG_SMP */

 struct pcpu_chunk {
 	struct list_head	list;		/* linked to pcpu_slot lists */
@@ -949,6 +955,7 @@ EXPORT_SYMBOL_GPL(free_percpu);
  */
 bool is_kernel_percpu_address(unsigned long addr)
 {
+#ifdef CONFIG_SMP
 	const size_t static_size = __per_cpu_end - __per_cpu_start;
 	void __percpu *base = __addr_to_pcpu_ptr(pcpu_base_addr);
 	unsigned int cpu;
@@ -959,6 +966,8 @@ bool is_kernel_percpu_address(unsigned long addr)
 		if ((void *)addr >= start && (void *)addr < start + static_size)
 			return true;
         }
+#endif
+	/* on UP, can't distinguish from other static vars, always false */
 	return false;
 }

@@ -1066,6 +1075,8 @@ void __init pcpu_free_alloc_info(struct pcpu_alloc_info *ai)
 	free_bootmem(__pa(ai), ai->__ai_size);
 }

+#if defined(CONFIG_SMP) && (defined(CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK) || \
+			    defined(CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK))
 /**
  * pcpu_build_alloc_info - build alloc_info considering distances between CPUs
  * @reserved_size: the size of reserved percpu area in bytes
@@ -1220,6 +1231,8 @@ static struct pcpu_alloc_info * __init pcpu_build_alloc_info(

 	return ai;
 }
+#endif	/* CONFIG_SMP && (CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK ||
+			  CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK) */

 /**
  * pcpu_dump_alloc_info - print out information about pcpu_alloc_info
@@ -1363,7 +1376,9 @@ int __init pcpu_setup_first_chunk(const struct pcpu_alloc_info *ai,

 	/* sanity checks */
 	PCPU_SETUP_BUG_ON(ai->nr_groups <= 0);
+#ifdef CONFIG_SMP
 	PCPU_SETUP_BUG_ON(!ai->static_size);
+#endif
 	PCPU_SETUP_BUG_ON(!base_addr);
 	PCPU_SETUP_BUG_ON(ai->unit_size < size_sum);
 	PCPU_SETUP_BUG_ON(ai->unit_size & ~PAGE_MASK);
@@ -1488,6 +1503,8 @@ int __init pcpu_setup_first_chunk(const struct pcpu_alloc_info *ai,
 	return 0;
 }

+#ifdef CONFIG_SMP
+
 const char *pcpu_fc_names[PCPU_FC_NR] __initdata = {
 	[PCPU_FC_AUTO]	= "auto",
 	[PCPU_FC_EMBED]	= "embed",
@@ -1758,8 +1775,9 @@ out_free_ar:
 }
 #endif /* CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK */

+#ifndef	CONFIG_HAVE_SETUP_PER_CPU_AREA
 /*
- * Generic percpu area setup.
+ * Generic SMP percpu area setup.
  *
  * The embedding helper is used because its behavior closely resembles
  * the original non-dynamic generic percpu area setup.  This is
@@ -1770,7 +1788,6 @@ out_free_ar:
  * on the physical linear memory mapping which uses large page
  * mappings on applicable archs.
  */
-#ifndef CONFIG_HAVE_SETUP_PER_CPU_AREA
 unsigned long __per_cpu_offset[NR_CPUS] __read_mostly;
 EXPORT_SYMBOL(__per_cpu_offset);

@@ -1799,13 +1816,48 @@ void __init setup_per_cpu_areas(void)
 				    PERCPU_DYNAMIC_RESERVE, PAGE_SIZE, NULL,
 				    pcpu_dfl_fc_alloc, pcpu_dfl_fc_free);
 	if (rc < 0)
-		panic("Failed to initialized percpu areas.");
+		panic("Failed to initialize percpu areas.");

 	delta = (unsigned long)pcpu_base_addr - (unsigned long)__per_cpu_start;
 	for_each_possible_cpu(cpu)
 		__per_cpu_offset[cpu] = delta + pcpu_unit_offsets[cpu];
 }
-#endif /* CONFIG_HAVE_SETUP_PER_CPU_AREA */
+#endif	/* CONFIG_HAVE_SETUP_PER_CPU_AREA */
+
+#else	/* CONFIG_SMP */
+
+/*
+ * UP percpu area setup.
+ *
+ * UP always uses km-based percpu allocator with identity mapping.
+ * Static percpu variables are indistinguishable from the usual static
+ * variables and don't require any special preparation.
+ */
+void __init setup_per_cpu_areas(void)
+{
+	const size_t unit_size =
+		roundup_pow_of_two(max_t(size_t, PCPU_MIN_UNIT_SIZE,
+					 PERCPU_DYNAMIC_RESERVE));
+	struct pcpu_alloc_info *ai;
+	void *fc;
+
+	ai = pcpu_alloc_alloc_info(1, 1);
+	fc = __alloc_bootmem(unit_size, PAGE_SIZE, __pa(MAX_DMA_ADDRESS));
+	if (!ai || !fc)
+		panic("Failed to allocate memory for percpu areas.");
+
+	ai->dyn_size = unit_size;
+	ai->unit_size = unit_size;
+	ai->atom_size = unit_size;
+	ai->alloc_size = unit_size;
+	ai->groups[0].nr_units = 1;
+	ai->groups[0].cpu_map[0] = 0;
+
+	if (pcpu_setup_first_chunk(ai, fc) < 0)
+		panic("Failed to initialize percpu areas.");
+}
+
+#endif	/* CONFIG_SMP */

 /*
  * First and reserved chunks are initialized with temporary allocation
diff --git a/mm/percpu_up.c b/mm/percpu_up.c
deleted file mode 100644
index db884fa..0000000
--- a/mm/percpu_up.c
+++ /dev/null
@@ -1,30 +0,0 @@
-/*
- * mm/percpu_up.c - dummy percpu memory allocator implementation for UP
- */
-
-#include <linux/module.h>
-#include <linux/percpu.h>
-#include <linux/slab.h>
-
-void __percpu *__alloc_percpu(size_t size, size_t align)
-{
-	/*
-	 * Can't easily make larger alignment work with kmalloc.  WARN
-	 * on it.  Larger alignment should only be used for module
-	 * percpu sections on SMP for which this path isn't used.
-	 */
-	WARN_ON_ONCE(align > SMP_CACHE_BYTES);
-	return (void __percpu __force *)kzalloc(size, GFP_KERNEL);
-}
-EXPORT_SYMBOL_GPL(__alloc_percpu);
-
-void free_percpu(void __percpu *p)
-{
-	kfree(this_cpu_ptr(p));
-}
-EXPORT_SYMBOL_GPL(free_percpu);
-
-phys_addr_t per_cpu_ptr_to_phys(void *addr)
-{
-	return __pa(addr);
-}
-- 
1.7.1

[PATCH RESEND 1/3] vmalloc: pcpu_get/free_vm_areas() aren't needed on UP

[Tejun Heo]

These functions are used only by percpu memory allocator on SMP.
Don't build them on UP.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Nick Piggin <npiggin@kernel.dk>
---
(resending w/ Nick's new email address)

So, something like these three patches.  Also available in the
following git tree.

  git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git review-up

Nick, can I route this one through percpu tree?

Thanks.

 include/linux/vmalloc.h |    2 ++
 mm/vmalloc.c            |    2 ++
 2 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index de05e96..3d510e8 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -115,10 +115,12 @@ extern rwlock_t vmlist_lock;
 extern struct vm_struct *vmlist;
 extern __init void vm_area_register_early(struct vm_struct *vm, size_t align);

+#ifdef CONFIG_SMP
 struct vm_struct **pcpu_get_vm_areas(const unsigned long *offsets,
 				     const size_t *sizes, int nr_vms,
 				     size_t align, gfp_t gfp_mask);

 void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms);
+#endif

 #endif /* _LINUX_VMALLOC_H */
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index b7e314b..eb57a63 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -2052,6 +2052,7 @@ void free_vm_area(struct vm_struct *area)
 }
 EXPORT_SYMBOL_GPL(free_vm_area);

+#ifdef CONFIG_SMP
 static struct vmap_area *node_to_va(struct rb_node *n)
 {
 	return n ? rb_entry(n, struct vmap_area, rb_node) : NULL;
@@ -2332,6 +2333,7 @@ void pcpu_free_vm_areas(struct vm_struct **vms, int nr_vms)
 		free_vm_area(vms[i]);
 	kfree(vms);
 }
+#endif	/* CONFIG_SMP */

 #ifdef CONFIG_PROC_FS
 static void *s_start(struct seq_file *m, loff_t *pos)
-- 
1.7.1

Re: [PATCH 1/3] vmalloc: pcpu_get/free_vm_areas() aren't needed on UP

[Christoph Lameter]

Reviewed-by: Chrsitoph Lameter <cl@linux.com>

Re: [PATCH 2/3] percpu: reduce PCPU_MIN_UNIT_SIZE to 32k

[Christoph Lameter]

Reviewed-by: Christoph Lameter <cl@linux.com>

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

[Christoph Lameter]

As much as I can review looks okay.

Reviewed-by: Christoph Lameter <cl@linux.com>

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

[Pekka Enberg]

On 3.9.2010 19.26, Tejun Heo wrote:
> On UP, percpu allocations were redirected to kmalloc.  This has the
> following problems.
>
> * For certain amount of allocations (determined by
>    PERCPU_DYNAMIC_EARLY_SLOTS and PERCPU_DYNAMIC_EARLY_SIZE), percpu
>    allocator can be used before the usual kernel memory allocator is
>    brought online.  On SMP, this is used to initialize the kernel
>    memory allocator.
>
> * percpu allocator honors alignment upto PAGE_SIZE but kmalloc()
>    doesn't.  For example, workqueue makes use of larger alignments for
>    cpu_workqueues.
>
> Currently, users of percpu allocators need to handle UP differently,
> which is somewhat fragile and ugly.  Other than small amount of
> memory, there isn't much to lose by enabling percpu allocator on UP.
> It can simply use kernel memory based chunk allocation which was added
> for SMP archs w/o MMUs.
>
> This patch removes mm/percpu_up.c, builds mm/percpu.c on UP too and
> makes UP build use percpu-km.  As percpu addresses and kernel
> addresses are always identity mapped and static percpu variables don't
> need any special treatment, nothing is arch dependent and mm/percpu.c
> implements generic setup_per_cpu_areas() for UP.
>
> Signed-off-by: Tejun Heo<tj@kernel.org>
> Cc: Christoph Lameter<cl@linux-foundation.org>
> Cc: Pekka Enberg<penberg@cs.helsinki.fi>

Acked-by: Pekka Enberg <penberg@kernel.org>

Is this going into some public append-only branch I could cherry-pick 
the changeset from to my 'slub/cleanups' branch?

			Pekka

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

[Tejun Heo]

Hello,

On 09/04/2010 08:54 AM, Pekka Enberg wrote:
> Is this going into some public append-only branch I could
> cherry-pick the changeset from to my 'slub/cleanups' branch?

I'll put it into percpu#for-next once Linus pulls in the currently
pending set of fixes.  I'll let you know.

Thanks.

-- 
tejun

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

[Tejun Heo]

On 09/04/2010 11:47 AM, Tejun Heo wrote:
> Hello,
> 
> On 09/04/2010 08:54 AM, Pekka Enberg wrote:
>> Is this going into some public append-only branch I could
>> cherry-pick the changeset from to my 'slub/cleanups' branch?
> 
> I'll put it into percpu#for-next once Linus pulls in the currently
> pending set of fixes.  I'll let you know.

Alright, now in percpu#for-next tree.

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git for-next

The branch currenly contains only those three patches, so please feel
free to pull from it.

Thanks.

-- 
tejun

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

[Tejun Heo]

On 09/08/2010 11:17 AM, Tejun Heo wrote:
> On 09/04/2010 11:47 AM, Tejun Heo wrote:
>> Hello,
>>
>> On 09/04/2010 08:54 AM, Pekka Enberg wrote:
>>> Is this going into some public append-only branch I could
>>> cherry-pick the changeset from to my 'slub/cleanups' branch?
>>
>> I'll put it into percpu#for-next once Linus pulls in the currently
>> pending set of fixes.  I'll let you know.
> 
> Alright, now in percpu#for-next tree.
> 
>  git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git for-next
> 
> The branch currenly contains only those three patches, so please feel
> free to pull from it.

Branch updated to fix build for s390 and add missing memory clearing
for km- allocator.  You'll at least want to pull upto commit
fc1481a956181d0360d3eb129965302489895a1b.

Thanks.

-- 
tejun

Re: [PATCH 3/3] percpu: use percpu allocator on UP too

[Pekka Enberg]

On 10.9.2010 17.59, Tejun Heo wrote:
> On 09/08/2010 11:17 AM, Tejun Heo wrote:
>> On 09/04/2010 11:47 AM, Tejun Heo wrote:
>>> Hello,
>>>
>>> On 09/04/2010 08:54 AM, Pekka Enberg wrote:
>>>> Is this going into some public append-only branch I could
>>>> cherry-pick the changeset from to my 'slub/cleanups' branch?
>>>
>>> I'll put it into percpu#for-next once Linus pulls in the currently
>>> pending set of fixes.  I'll let you know.
>>
>> Alright, now in percpu#for-next tree.
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git for-next
>>
>> The branch currenly contains only those three patches, so please feel
>> free to pull from it.
>
> Branch updated to fix build for s390 and add missing memory clearing
> for km- allocator.  You'll at least want to pull upto commit
> fc1481a956181d0360d3eb129965302489895a1b.

I cherry-picked patches from the branch and reverted SLUB bandaid. SLUB 
works fine here on UP now so I'm putting it in linux-next. Thanks Tejun!

			Pekka

linux-next: build failure after merge of the final tree (slab tree related)

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 2980 bytes --]

Hi all,

After merging the final tree, today's linux-next build (powerpc
allnoconfig) failed like this:

In file included from include/linux/xattr.h:14:0,
                 from include/linux/cgroup.h:21,
                 from include/linux/memcontrol.h:22,
                 from include/linux/swap.h:8,
                 from include/linux/suspend.h:4,
                 from arch/powerpc/kernel/asm-offsets.c:24:
include/linux/slab.h:587:21: error: redefinition of 'kmalloc_node'
 static inline void *kmalloc_node(size_t size, gfp_t flags, int node)
                     ^
include/linux/slab.h:429:30: note: previous definition of 'kmalloc_node' was here
 static __always_inline void *kmalloc_node(size_t size, gfp_t flags, int node)
                              ^
include/linux/slab.h:592:21: error: redefinition of '__kmalloc_node'
 static inline void *__kmalloc_node(size_t size, gfp_t flags, int node)
                     ^
In file included from include/linux/xattr.h:14:0,
                 from include/linux/cgroup.h:21,
                 from include/linux/memcontrol.h:22,
                 from include/linux/swap.h:8,
                 from include/linux/suspend.h:4,
                 from arch/powerpc/kernel/asm-offsets.c:24:
include/linux/slab.h:302:30: note: previous definition of '__kmalloc_node' was here
 static __always_inline void *__kmalloc_node(size_t size, gfp_t flags, int node)
                              ^
In file included from include/linux/xattr.h:14:0,
                 from include/linux/cgroup.h:21,
                 from include/linux/memcontrol.h:22,
                 from include/linux/swap.h:8,
                 from include/linux/suspend.h:4,
                 from arch/powerpc/kernel/asm-offsets.c:24:
include/linux/slab.h:599:21: error: redefinition of 'kmem_cache_alloc_node'
 static inline void *kmem_cache_alloc_node(struct kmem_cache *cachep,
                     ^
In file included from include/linux/xattr.h:14:0,
                 from include/linux/cgroup.h:21,
                 from include/linux/memcontrol.h:22,
                 from include/linux/swap.h:8,
                 from include/linux/suspend.h:4,
                 from arch/powerpc/kernel/asm-offsets.c:24:
include/linux/slab.h:307:30: note: previous definition of 'kmem_cache_alloc_node' was here
 static __always_inline void *kmem_cache_alloc_node(struct kmem_cache *s, gfp_t flags, int node)
                              ^

Probably caused by commit 1a1d0328975f ("mm/sl[aou]b: Move kmalloc_node
functions to common code") from the slab tree.

This build has CONFIG_NUMA and CONFIG_SLOB undefined.

I have reverted that commit and commits eab6f38ab23c ("mm/sl[aou]b: Move
kmalloc definitions to slab.h"), c8f12ea82f92 ("sl[aou]b: Remove
unnecessary #includes") and 8876832d0466 ("slub: remove
verify_mem_not_deleted()") that followed it.
-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]