Linux-Next Archive on lore.kernel.org
 help / color / Atom feed
From: Sachin Sant <sachinp@linux.vnet.ibm.com>
To: Linux-Next Mailing List <linux-next@vger.kernel.org>,
	Kirill Tkhai <ktkhai@virtuozzo.com>
Cc: linuxppc-dev@lists.ozlabs.org
Subject: [5.6.0-rc2-next-20200218/powerpc] Boot failure on POWER9
Date: Tue, 18 Feb 2020 16:15:23 +0530
Message-ID: <3381CD91-AB3D-4773-BA04-E7A072A63968@linux.vnet.ibm.com> (raw)


[-- Attachment #1: Type: text/plain, Size: 3407 bytes --]

Todays next fails to boot on a POWER9 PowerVM logical partition
with following trace:

[    8.767660] random: systemd: uninitialized urandom read (16 bytes read)
[    8.768629] BUG: Kernel NULL pointer dereference on read at 0x000073b0
[    8.768635] Faulting instruction address: 0xc0000000003d55f4
[    8.768641] Oops: Kernel access of bad area, sig: 11 [#1]
[    8.768645] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries
[    8.768650] Modules linked in:
[    8.768655] CPU: 19 PID: 1 Comm: systemd Not tainted 5.6.0-rc2-next-20200218-autotest #1
[    8.768660] NIP:  c0000000003d55f4 LR: c0000000003d5b94 CTR: 0000000000000000
[    8.768666] REGS: c0000008b37836d0 TRAP: 0300   Not tainted  (5.6.0-rc2-next-20200218-autotest)
[    8.768671] MSR:  8000000000009033 <SF,EE,ME,IR,DR,RI,LE>  CR: 24004844  XER: 00000000
[    8.768679] CFAR: c00000000000dec4 DAR: 00000000000073b0 DSISR: 40000000 IRQMASK: 1
[    8.768679] GPR00: c0000000003d5b94 c0000008b3783960 c00000000155d400 c0000008b301f500
[    8.768679] GPR04: 0000000000000dc0 0000000000000002 c0000000003443d8 c0000008bb398620
[    8.768679] GPR08: 00000008ba2f0000 0000000000000001 0000000000000000 0000000000000000
[    8.768679] GPR12: 0000000024004844 c00000001ec52a00 0000000000000000 0000000000000000
[    8.768679] GPR16: c0000008a1b20048 c000000001595898 c000000001750c18 0000000000000002
[    8.768679] GPR20: c000000001750c28 c000000001624470 0000000fffffffe0 5deadbeef0000122
[    8.768679] GPR24: 0000000000000001 0000000000000dc0 0000000000000002 c0000000003443d8
[    8.768679] GPR28: c0000008b301f500 c0000008bb398620 0000000000000000 c00c000002287180
[    8.768727] NIP [c0000000003d55f4] ___slab_alloc+0x1f4/0x760
[    8.768732] LR [c0000000003d5b94] __slab_alloc+0x34/0x60
[    8.768735] Call Trace:
[    8.768739] [c0000008b3783960] [c0000000003d5734] ___slab_alloc+0x334/0x760 (unreliable)
[    8.768745] [c0000008b3783a40] [c0000000003d5b94] __slab_alloc+0x34/0x60
[    8.768751] [c0000008b3783a70] [c0000000003d6fa0] __kmalloc_node+0x110/0x490
[    8.768757] [c0000008b3783af0] [c0000000003443d8] kvmalloc_node+0x58/0x110
[    8.768763] [c0000008b3783b30] [c0000000003fee38] mem_cgroup_css_online+0x108/0x270
[    8.768769] [c0000008b3783b90] [c000000000235aa8] online_css+0x48/0xd0
[    8.768775] [c0000008b3783bc0] [c00000000023eaec] cgroup_apply_control_enable+0x2ec/0x4d0
[    8.768781] [c0000008b3783ca0] [c000000000242318] cgroup_mkdir+0x228/0x5f0
[    8.768786] [c0000008b3783d10] [c00000000051e170] kernfs_iop_mkdir+0x90/0xf0
[    8.768792] [c0000008b3783d50] [c00000000043dc00] vfs_mkdir+0x110/0x230
[    8.768797] [c0000008b3783da0] [c000000000441c90] do_mkdirat+0xb0/0x1a0
[    8.768804] [c0000008b3783e20] [c00000000000b278] system_call+0x5c/0x68
[    8.768808] Instruction dump:
[    8.768811] 7c421378 e95f0000 714a0001 4082fff0 4bffff64 60000000 60000000 faa10088
[    8.768818] 3ea2000c 3ab57070 7b4a1f24 7d55502a <e94a73b0> 2faa0000 409e0394 3d02002a
[    8.768826] ---[ end trace 631af2cb73507891 ]---
[    8.770876]
[    9.770887] Kernel panic - not syncing: Fatal exception

Bisect reveals the problem was introduced in next-20200217 by following commit 

commit a75056fc1e7c 
mm/memcontrol.c: allocate shrinker_map on appropriate NUMA node

I can boot the kernel successfully if the patch is reverted. 

Boot log attached.

Thanks
-Sachin


[-- Attachment #2: next-20200218.log --]
[-- Type: application/octet-stream, Size: 19761 bytes --]

             reply index

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-18 10:45 Sachin Sant [this message]
2020-02-18 10:50 ` Kirill Tkhai
2020-02-18 11:01   ` Kirill Tkhai
2020-02-18 11:35     ` Kirill Tkhai
2020-02-18 11:40     ` Sachin Sant
2020-02-18 11:55       ` Michal Hocko
2020-02-18 14:00         ` Sachin Sant
2020-02-18 14:26           ` Michal Hocko
2020-02-18 15:11             ` Sachin Sant
2020-02-18 15:24               ` Michal Hocko
2020-02-22  3:38                 ` Christopher Lameter
2020-02-24  8:58                   ` Michal Hocko
2020-02-26 18:25                     ` Christopher Lameter
2020-02-26 18:41                       ` Michal Hocko
2020-02-26 18:44                         ` Christopher Lameter
2020-02-26 19:01                           ` Michal Hocko
2020-02-26 20:31                             ` David Rientjes
2020-02-26 20:52                               ` Michal Hocko
2020-02-26 21:45                         ` Vlastimil Babka
2020-02-26 22:29                           ` Vlastimil Babka
2020-02-27 12:12                             ` Michal Hocko
2020-02-27 16:00                               ` Sachin Sant
2020-02-27 16:16                                 ` Vlastimil Babka
2020-02-27 18:26                                   ` Michal Hocko
2020-03-10 15:01                                     ` Michal Hocko
2020-03-12 12:18                                       ` Michael Ellerman
2020-03-12 16:51                                         ` Sachin Sant
2020-03-13 10:48                                           ` Michael Ellerman
2020-03-13 11:12                                             ` Srikar Dronamraju
2020-03-13 11:35                                               ` Vlastimil Babka
2020-03-14  8:10                                                 ` Sachin Sant
2020-02-27 12:02                           ` Michal Hocko
2020-02-18 11:38   ` Sachin Sant
2020-02-18 11:53     ` Kirill Tkhai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3381CD91-AB3D-4773-BA04-E7A072A63968@linux.vnet.ibm.com \
    --to=sachinp@linux.vnet.ibm.com \
    --cc=ktkhai@virtuozzo.com \
    --cc=linux-next@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-Next Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-next/0 linux-next/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-next linux-next/ https://lore.kernel.org/linux-next \
		linux-next@vger.kernel.org
	public-inbox-index linux-next

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-next


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git